I’ve always been fascinated by what SDXL and its derivative fine-tuned models have brought to the table. The ability to generate stunning, high-quality images from simple text prompts feels like a superpower, especially when you can guide the AI to create art, realistic portraits, or even fantasy landscapes. However, there was one area that left me frustrated: generating scientifically accurate images of black holes.

Like many, I found myself captivated by the idea of visualizing the most enigmatic objects in our universe. But every time I prompted SDXL to generate a black hole, the results were more artistic than scientific. It became apparent that the internet—overflowing with inaccurate artistic renditions—had influenced the model's output. I knew I had to change that. If I couldn't find a scientifically accurate black hole model, I would create one.

The Quest for Scientific Accuracy

My journey began with the search for a dataset that could teach SDXL to generate black hole images grounded in scientific reality. This wasn’t just about creating something visually striking; it was about adhering to the laws of general relativity and the principles that govern these cosmic giants.

I started by generating over 200 black hole images, each carefully crafted with varying angles, different sizes of accretion disks, and unique backgrounds. Using software like Universe Sandbox, I simulated black holes of different masses, spins, and orientations, ensuring that space was accurately warped around them. I leaned heavily on scientifically accurate sources, such as NASA’s articles and research papers, to guide my simulations and ensure each image adhered to the latest scientific understanding.

Dataset Curation and Tagging

Creating the images was just the first step. The next was to meticulously tag each one to ensure the model would learn from the data effectively. I applied detailed tags that captured the orientation of the black hole, the specific characteristics of the accretion disk, and the nature of the background. These tags weren’t just labels—they were essential for helping the model grasp the complex physics at play.

By ensuring the dataset was scientifically accurate and well-tagged, I aimed to teach SDXL more than just what a black hole looks like; I wanted it to "understand" the physical phenomena that make black holes so awe-inspiring.

Training the LoRa: Fine-Tuning with Kohya's SDXL Trainer

With my dataset ready, I turned to Kohya’s SDXL LoRa Trainer to fine-tune the model.

Choosing the right parameters was critical. After experimenting with various settings, I found that a learning rate of 1e-5 worked best, balancing the need for the model to learn the new data while retaining its ability to generate high-quality images. I also ensured that the training was done in a manner that preserved the detail and accuracy of the images—factors like the accretion disk’s glow and the gravitational lensing effect were crucial to get right.

The trigger word for this LoRa model is “grgnt”—a shorthand for Gargantua, the famous black hole from the movie Interstellar.

Getting the Best Results in Generation

For those looking to generate their own black hole images with this LoRa, I recommend using the DMP++ SDE Karras sampler with 30+ steps. This setup seems to produce the most accurate and visually striking results. In my experiments, a CFG scale between 3 and 7 worked best, though you might need to tweak this depending on your specific prompts and desired outcomes.

Looking Ahead: Next Steps and Future Plans

While I’m thrilled with what this model can do, I’m already planning the next iteration. My focus will be on expanding the dataset with even more varied black holes and refining the caption tagging to capture even more nuances. The goal is to make this LoRa even more versatile and accurate, enabling anyone to create their dream rendition of a black hole.

Access and the Path Forward

As of now, I’m releasing this LoRa in early access. I hope to gather enough support—both in terms of credits and feedback—to continue refining this model. The ultimate goal is to move towards a more powerful training setup, potentially involving FLUX training for even better results.

If I don’t reach the support goal within a week, I’ll open up access to everyone, though further training may be delayed or never happen due to my current lack of powerful hardware. I rely heavily on tools like CIVIT AI LoRa Trainer, and without the necessary resources, future iterations of this model might remain just a dream.

I’m excited to see what the community will create with this model, and I hope it serves as a valuable tool for those who, like me, are fascinated by the mysteries of the cosmos and want to visualize them with scientific accuracy.

Thank you for your support, and here’s to many more journeys into the heart of a black hole!