12/29/2024 Illustrious/noob update:
Happy New Years!
Finally getting back into training these loras. Updated this one to "noobaiXLNAIXL_epsilonPred11Version", using the same 135 images. I... can't remember what step count or epoch I trained to, but it seems to be good, maybe a tad overtrained. Let me know how it works, since training for Illustrious seems a bit more nuanced than Pony. I used Jinkiesnoob for these example images which worked really well, but also the base noob model and even waiNSFWillustrious worked really well from my brief testing.
The robot hand claws are considerably more consistent, just prompt for "robotic hand, 3 fingers" with some weight, and you'll get it far more often.
Older Descriptions:
4/1/2024 PDXL Update:
I updated this model for use with PDXL. Please see the "About this version" section to the right for details. It's pretty much the same model, just trained on better tags, at a better resolution, and with a better optimizer. Also, I might have overtrained this a tad, so consider lowering the strength if you have any issues with artist mixes or what-not. Hope you enjoy.
-One of the important things to note: I did manually tag all images where the assaultrons have their original "3 finger" robotic claw hands, so prompt for "robotic hand, 3 fingers" if you want to try and invoke that. NOTE: It's super inconsistent, but you can sometimes get the right results if you prompt hard enough.
RobCo™ has done it again with their latest in engineering: The Assaultron!
This is a lora trained on 136 hand picked images of assaultrons from the Fallout series. There are no 3d/sfm images included, it's exclusively digital art. It gets the 'face' and single eye pretty well, and can do the claw-arms too... sometimes. It was trained on danbooru-style tags, so use those when trying to prompt - I've included the tags in a .txt file in the "Training Images" download below.
It works pretty well with most ckpts, but I recommend Anything 4.5, I used it for all of the example images.
NOTES:
-Most assaultrons trained in this lora were green, but there were a good amount of blue ones too (Ada). You can change the color of the robot by emphasizing tags like: blue_armor, blue_skin, etc ...or use other colors. See example images.
-Unlike other models I've made, this one can't really do a "male version" assaultron, but you're welcome to go and try.
-The claw arms are the most inconsistent part of this, sometimes it will get confused on whether to do a normal hand or the three claw bits.
Example prompt: (See example images, most images still have their metadata. Just drag the images onto the PNG info tab and import it.)
Copyright RobCo™ Patent Pending 2072
Description
FAQ
Comments (21)
I wonder how well training it on actual game models or screenshots might help for more 3d realism.
You could basically make images from any angle or lighting style by using the model. You could even use a ton of poses from the Fallout modding scene. There are also a few sexy/thick variants of the assaultron as mods that could be used.
Additionally, you could use a lot of different colors as well. You could generate a pretty solid data set for this I think
@OtherPerspective I know there's some thicc Ada and Assaultron body mods that I've used before. Would love to have a Lora based on those, maybe one of these days I'll reinstall Fallout 4 and try it myself. There's some mods that let you hide the UI which should also help for screenshots.
@Tacot360 You can just use freecam mode, I think that hides the UI, but if not there is a command for it. Also, you could use the creation kit to make different environments with different backgrounds and lighting conditions (a lot of work, but would greatly help with variety and such.) Since the hands are an issue you could use the robot dlc and replace them with guns or other weapons. I think that would be cool too.
Having both the sexy and default version in the data set at the same time might be worth trying, but I don't know how that would work. There is a mod that adds genitals to the robots as well: https://www.nexusmods.com/fallout4/mods/26991 as well as a bunch of other stuff.
For poses, there are a tone of static pose packs and custom animations on the mod sites. There are also probably mods that add more paint jobs and the ability to change eye color etc.
For different lighting and backgrounds though it might be better to just use blender for that. It would be much easier to animate light sources moving around and changing color and you could just render a video of the character model changing poses and moving around with the lights changing and extract the frames to use for the data set. I don't know how the labelling or tagging works but I am guessing you might have to do that part manually so a lower framerate would make that easier.
I guess it all depends on how much work you're willing to put into it. Unfortunately it isn't an option for me because I don't have the specs for any actual training so I can't do much beyond using the loras and models provided here.
Would you be willing to share your data set by any chance?
I archive all the training images I use on my loras, so I still have the files, but why do you want them? Are you going to train your own assaultron model with different settings or something?
@ZoochMan I am thinking about it. Having your dataset as reference would be helpful but I intend to use primarily new images as well. The goal is to make a lora that produces different results but uses the same prompt language and stuff as your lora so they would be more compatible with each other if anyone wanted to do that.
Right now I am gathering images (creating my own using the character model) instead of artwork or existing images from other sources. If you'd be willing to share the data you used I might be able to integrate it as well if it would make better results but that's your call
@OtherPerspective Sounds good. I updated the "training data" to include the tags.txt and all the training images plus their tags as well. Looking forward to whatever you end up making!
@ZoochMan Awesome! Thanks for sharing. With any luck I will get some kind of success
@OtherPerspective Any luck on that model?
@ZoochMan I'm still working on the data set. I'm making a lot of custom images that should hopefully allow for dynamic generations and improvements to things like the hand claws and color limitations
Been working on my own assaultron lora off and on for some time now. I have finally seemed to make some decent progress with my latest attempt but I still can't get the claws right (and the headshape is also pretty off) but as far as detail and realism go I am pretty satisfied with my results so far lol: https://i.postimg.cc/rsq6KKCm/02116-439811379.png
@OtherPerspective Looks good! You could try training it at a higher resolution, and maybe train images where the claws are prevalent to a higher standard. Also, is this intended to be a realistic model lora?
Something I've been training my new loras on is a "furry art" model called Fluffyrock, and it's by far the best model I've seen for digital art so far. I could probably make a much better version of my assaultron model if I retrained it on that model at a higher resolution... But I'm lazy :P
@ZoochMan I'm trying to get a model with more consistent details of the assaultron. A lot of the fanart takes a lot of artistic liberties for obvious reasons but of course every artist makes different changes to the overall design. I want to have the basic "anatomy" of the assaultron be more accurate and consistent, and less sexualized by default (unless specified to be that way) and I want it to be capable of more realistic stuff too. Maybe it's too ambitious especially since I have never actually done any training on anything else before but it's at least a good way to learn.
As for the claws, I have them featured very directly in a lot of my images. I have made sure that they show up in very easy to read angles and have made sure that they interact with anything like they would practically and not like hands. I have a few close-ups and a few images with unique poses and interactions with the environment. I make sure they aren't too confusing to look at in each image, and I have any illustrations that affect their look too much lower priority during training.
At the moment, I am still expanding the dataset. I want to be able to generate good quality images that I can edit and improve and feed back into the AI dataset to "reinforce good behaviour" I guess. I need to supplement the dataset with something, as there aren't a lot of good sources for this character out there. Most of my dataset is made up of my own custom images I created in Garry's Mod of all places lmao. I can also iterate on the existing images by improving them in img2img and photoshop with each new improvement to the lora.
I am training on several models at once and so far the best I have gotten is Abyss orange 2 nsfw. Anything 4.5 was actually making the lora significantly worse for whatever reason. I will try more as well. Another thing that seems to have improved the results is using square images. Not by cropping or lowering the resolution, but simply by expanding the edges. With art with simple backgrounds, this is of course trivial to do in PS. I use outpainting or custom drawn areas for the rest. As long as they are square, it seems to help a lot, regardless of the resolution.
Unfortunately I can't train at a higher res since I use colab and 768 or higher causes out of memory errors unless you have a pro account, which I have no desire for at the moment.
https://postimg.cc/gallery/Nr3pmfB here are some more of the images btw. I used the randomizer addon with a lot of wildcards and generated a bunch overnight to see if I could find what works and what it is capable of. Claws are still pretty bad but a couple of these show them almost properly posed
@OtherPerspective Not too shabby. It definitely have a lot more detail, and is considerably more... faithful to the original design than mine :p
I've noticed that too, training on Anything 4.5 seems to... ruin loras. No idea why. I always used the NAI-anime model to train my models, but I'd used Anything 4.5 to generate the images with it. It just seems to work well that way.
Keep on working, it sounds like you're making good progress.
Hi, i'm kinda new to this whole ai art thing, and im wondering if im doing something wrong. im using your model and the same promps/negatives and im getting nothing that even remotely looks like an assaultron. help please ; ~ ;
Sure. First off, make sure you read the description of this post, then also make sure you've downloaded the Anything 4.5 model, and that you've selected it in your webui. Then get the additonal networks extension and enable my lora, or use the native lora capabilites that automatic1111 has. You're also gonna want to use the EasyNegative embedding, which makes the negative prompt work a lot better and easier.
After all that, it should output similar images to what I've posted here. Keep in mind, that I manually img2img all the examples images, and that you you should be generating at 512x786 resolution if you want the images to come out normally. Hope this helps. If you still have issues, just say.
@ZoochMan Followed your instructions and everything works perfectly now, i was messing up on the resolution, and i had no idea i could pre-select a lora through the extra networks in settings. Thanks so much.
@ZoochMan Wondering if it is Checkpoint-agnostic tho since there are other anime models that this can be ported in
@TomLucidor I've noticed that, when using loras made which were trained for one model, it can still work pretty well with other ones, even if the model is not anime-based, or digital art based at all. It's all pretty spotty and I haven't tested it extensively by any means.
Either way, I've kind of moved on from using most 'anime-based' models, so I don't really know.