This is mostly a test. It seems that Illustrious is already able to do multiple heads, but results are not great in some models; this is mostly to make results stable and consistent.

This is also a test to see how well training on Illustrious 0.1 works.

Seems like the two heads trigger makes three heads often, but i guess that's a feature.