The Workflow was setup to have a clean "GUI" showing only parameters that matter, so you might want to toggle off Link visibility, like in above screenshot.
V1.8 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama. Text to Audio/Song (examples below).
small update on Lora handling to better include Lora trigger words/phrases
Check this link with 1.5.Turbo XL Loras by Ryanontheinside: https://huggingface.co/ryanontheinside/models
1.5 XL Loras can be loaded on any XL Slot (Turbo, SFT or merges), it improve the sound very well, like more authentic guitars for rock&metal, etc. Also song structures follow the genre better.
Loras require a trigger word that can be found on the main page per Lora of above link. Use the trigger word in the new node "Pretext" within the Input section of the workflow, added around 25 of those triggers to a Note in the WF for copy&paste.
See post in discussion below for more info.
Key features:
Can use any Song, Artist as reference or any other description to generate tags and lyrics.
Will output up to 4 songs, each by Turbo, SFT, Turbo XL and SFT XL model (or any merge or Base model).
Keyscales, bpm and song duration can be randomized.
able to use dynamic prompts.
creates suitable songtitle and filenames with Ollama.
saves songs as MP3 with tags like lyrics, artist, genre, etc.
V1.7 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama. Text to Audio/Song.
replaced the "save Mp3" nodes to allow ID3 tags to include data like artist name, bpm, genre, lyrics, etc.
requires custom node: https://github.com/mattjohnpowell/comfyui-audio-expo
added a feature to save all relevant data (tags, lyrics, etc.) as a separate text (.txt) file
audio render processing remains unchanged to previous version
V1.6 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama.
updated the settings for XL models and added a 3rd System Prompt for tags to chose, with more descriptive song descriptions.
1.5 XL SFT pipeline now has an "Adaptive Projected Guidance" node and negative prompt.
** See below some tips which model and settings to start with.
V1.5 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama:
setup to create up to 4 tracks in a run, 2x Ace1.5 and 2x Ace1.5 XL, each with Turbo and SFT model, to compare (can be individually switched on/off)
VAE changed to tiled Audio VAE decode, uses less Vram.
V1.2 Ace Step 1.5. Turbo and SFT model with Ollama Text to Audio/Song
small update to GUI, system prompts and SFT sampler "engine"
V1.0 Ace Step 1.5. Turbo and SFT model with Ollama Text to Audio/Song
Ace Step uses TAGS and LYRICS to create a song. These can be generated by Ollama or by own prompts.
Download Files:
Ace Step 1.5 TURBO model: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/diffusion_models
Ace Step 1.5 SFT model: https://huggingface.co/ACE-Step/acestep-v15-sft/tree/main (download model.safetensor and rename it)
Ace Step 1.5 XL Models: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/diffusion_models
Ace Step 1.5 XL Model merges (i.e. merge of turbo with SFT): https://huggingface.co/Aryanne/acestep-v15-test-merges/tree/main/xl
Textencoder: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/text_encoders (Qwen_0.6b and Qwen_4b required, 1.7b is a smaller alternative to 4b)
VAE: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/vae
alternative VAE: https://huggingface.co/megagrump/Ace-Step-1.5-ScragVAE-ComfyUI
Ollama Models, required for tags, lyrics and songtitle, you can choose 1,2 or 3 different models, tags and lyrics might need a bigger model >7b, songtitle can use a smaller model:
https://ollama.com/mirage335/Llama-3-NeuralDaredevil-8B-abliterated-virtuoso (allround model, fast, usable for tags, lyrics and songtitle, recommended)
https://ollama.com/huihui_ai/qwen3-vl-abliterated (for tags and lyrics, able to use thinking)
https://ollama.com/artifish/llama3.2-uncensored (small and fast for songtitle and tags)
Alternative Turbo Models and merges (normal, non XL) :
Turbo continuous: https://huggingface.co/ACE-Step/acestep-v15-turbo-continuous/tree/main
Turbo-Shift1: https://huggingface.co/ACE-Step/acestep-v15-turbo-shift1/tree/main
Turbo-Shift3: https://huggingface.co/ACE-Step/acestep-v15-turbo-shift3/tree/main
Merges of SFT, Turbo and Base model: https://huggingface.co/Aryanne/acestep-v15-test-merges/tree/main
GGUF Models "normal" and XL: https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/tree/main
Which models to start with ?
If you just want to try it first before downloading all those models, start with 1 model only, recommend the Turbo or Turbo XL model.
My current choice for normal model: Turbo-SFT merge_ta_0.5 & Turbo-Shift1, using these settings:
Turbo-SFT_merge model with sampler: er_sde, scheduler: beta57 (or beta), 22 steps
Turbo-Shift1 model with sampler euler, scheduler: normal, 138 steps
XL Model settings:
XL Turbo-SFT merge model: sampler: er_sde, scheduler: sgm_uniform, 40 steps
alternative: sampler: res_s2, scheduler beta57 (requires RES4LYF custom nodes)
XL SFT model: sampler: euler (or res_2s), scheduler: normal, 46 steps, CFG = 7.3, Adaptive Projected Guidance: eta = 1.05, norm_thresh= 1.3, momentum=0.0. Increase norm_thresh as the main parameter. These settings deliver "stabil" output for XL SFT,Base and their merges. The merges sound way better, pure SFT or Base introduce a lot of noise. I bypassed ModelSamplingAuraflow (see node next to model loader node). I think the base-turbo XL model merge fits well in that slot.
Disable "generate_audio_codes" in "TextEncodeAceStep" node to get different results, it works very well for many genres and reduces process time.
Ollama Model: Llama-3-NeuralDaredevil-8b-abliterated
More infos on models see thread below in discussion.
Save Location:
📂 ComfyUI/
├── 📂 models/
│ ├── 📂 diffusion_models/
│ │ └── acestep_v1.5_turbo.safetensors
│ ├── 📂 text_encoders/
│ │ ├── qwen_0.6b_ace15.safetensors
│ │ └── qwen_4b_ace15.safetensors (or 1.7b)
│ └── 📂 vae/
│ └── ace_1.5_vae.safetensors
Custom Nodes used:
https://github.com/mattjohnpowell/comfyui-audio-expo (as of V1.7)
optional (use Beta57 scheduler for a bit more punch, requires RES4LYF): https://github.com/ClownsharkBatwing/RES4LYF
Examples various styles:
With Lora:
https://soundcloud.com/tele-joe/city-of-silent-sirens_gothic
https://soundcloud.com/tele-joe/forgotten-blazes_future-bass
No Lora:
https://soundcloud.com/tele-joe/burning-echoes-of-the-fallen
https://soundcloud.com/tele-joe/firestorms_unleashed_turbo_000
https://soundcloud.com/tele-joe/westsidevibesforever_funkhipho
Ollama help:
Install Ollama from https://ollama.com/
download a model: Go to a model page, chose a model , then hit the copy button, i.e. https://ollama.com/mirage335/Llama-3-NeuralDaredevil-8B-abliterated-virtuoso
open terminal and paste the model name, i.e.: ollama run huihui_ai/qwen3-vl-abliterated
model will be downloaded and can be selected in green comfy node "Ollama Connectivity". Hit "Reconnect" to refresh.
Description
Workflow setup to run AceStep 1.5 and 1.5 XL Models
update: Save Mp3 with ID3 tags and save tags & lyrics as separate .txt file
FAQ
Comments (16)
I'm glad you made this update but there are some issues. It told me to download a diffusion model "acestep_v1.5_xl_merge_base_turbo_ta_0.5.safetensors (1)" that seems to be named incorrectly. It doesn't output music except for SFT. The lyrics are going to the wrong directory outside of the output directory. I have a copy of the terminal log from Comfy if you need it.
that "acestep_v1.5_xl_merge_base_turbo_ta_0.5.safetensors" is to be used in the last slot where you see the red negativ prompt. That model represents a merge between the base and the turbo model (50/50) and delivers better quality than the pure base or sft model.
Although the pipeline name says "Turbo", "SFT", "Turbo XL", "SFT XL", I mainly use merge models in that slots, you can see the names in the model loader (and what I wrote in the notes saying "Which model to start with?")
regarding storage location for the lyrics file, you can change the "output_file_path" in the blue "save text" node.
I´ll check the log file seperately.
the log file shows some issue related to installed custom nodes that require Nvidia card, you seem to have AMD. But those custom nodes are not used in my workflow.
The settings in the workflow are set to use custom sampler from RES4LYF. Not sure if you have installed them. You can replace them with euler/beta setting, however for me the RES4LYF sound better, worth to install.
You can throw the log file into chatGPT or claude and ask for help, this usually works well.
@tremolo28Â I appreciate that your workflows work well with AMD gpu's. I have to wait for Ace Step to finally work with AMD. Much thanks for that. Work gets in the way of truly testing everything that's why it took me a while to try out this workflow. I did use that save text node in the older versions with success even though I just have set to song title and lyrics but I do have extension added to the files.
Another question is there a way to get Ace Step 1.5 to have more variation in female voices? All forms of Ace Step 1.5 seem to have the same female voice.
@tvhead1980Â the models have a bit of a mind on their own when it comes to sound of vocals, but there is definately variation, just be more descriptive within the tags (rough, gritty, soft, shouting, mellow voice/vocals, etc.} Also the genre has high influence on the vocal sounding and of course the language or accent.
@tremolo28Â Whenever AMD cards can make LoRa's I plan on making some specifically for voices.
Thanks
Ryanontheinside released some cool Loras for 1.5 XL model. Check this link: https://huggingface.co/ryanontheinside/models
1.5 XL Loras can be used on Turbo XL and SFT XL slot of the workflow and works well with Turbo XL model or with any Turbo XL merge model.
It requires to use the trigger words per Lora, which can be found on the main page per Lora (i.e: Dubstep Lora trigger = roti-dubst3pp).
Lora works without trigger already, but then it seems not to apply the trained sounds, just the trained genre song structure and articulations.
The same works as well with Non XL (normal 1.5) Loras, just load them on the normal model slots. For each XL genre Lora, there is a non XL Lora, check the filenames. They use the same trigger like the XL Lora.
i.e. Lora for Metal genre:
XL: https://huggingface.co/ryanontheinside/metal-acestep1.5-xl-v1
normal (non XL): https://huggingface.co/ryanontheinside/metal-acestep1.5-v1
trigger word for both: roti-m3t4al
Hello,
The workflow has been working well I am able to produce samples that I can use to make beats (quality isn't great but I can definitely work with it and its FREE!).
Only small issue I am having is the Loras.
Is there a way for me to change the weight?
Whenever I have the lora node active with a lora in the node I will get a cmd error "ERROR lora diffusion_model.decoder.layers.23.cross_attn.o_proj.weight shape '[2048, 2048]' is invalid for input of size 10485760".
If I dont use a lora but the node is on I will get a lora node error of still but not showing a weight error: "Exception in callback ProactorBasePipeTransport.call_connection_lost()
handle: <Handle ProactorBasePipeTransport.call_connection_lost()>
Traceback (most recent call last):
File "asyncio\eventspy", line 89, in _run
File "asyncio\proactor_events py", line 165, in callconnection_lost"
I am using the models that were suggested. Didnt touch the cfg shift or anything. I am also using the Ryan loras suggested for the XL, have done git clone method into the loras folder so it would have the path of Lora/Ace-Step/Funk-Ace-Step.
@ihy2ln Hi, you can change Lora weights in the Lora loader (Strength).
You are saying the lora loader causes issues even without any Lora loaded? The lora loader node is "Power Lora Loader (rgthree)", maybe check to update the rgthree custom nodes.
Can you share the Link of the Lora and tell in which Slot you loaded it?
@tremolo28Â https://huggingface.co/ryanontheinside/funk50-acestep1.5-dora-v2. loaded in the XL nodes. I was running into an issue where the lora node would not show any changes when clicking unless I refreshed the page. Also tried adding the nodes to other workflows same things happened. Did updates for rgthree on latest and for nightly both didnt work. Might be something on my end so I am doing a fresh install of comyui.
@tremolo28Â ok I got it working after the fresh install. RG3 and KJnode was acting up on my end fixed after a fresh install,
As for the models I had to use the additional ace step links cause the ones at top kept giving the error, I thought I could use the xl models as long as it was any xl model but seems I have to use more specific xl models.
also had to install the optional RES4LYF for the base 57 (through ComfyUI Manager for some reason the git hub didnt seem to work).
but after all that everything seems to be working as intended, tomorrow I will download the non xl version loras.
but as said before the workflow works out the box
@ihy2ln the file from the link is a Dora, which wont work in Lora loader. Check the files that have the naming like „genre“-acestep-1.5-xl-v1 for XL lora.
thank you for the fantastic wf, really love it.
do we need to tweak the parameters according to the lora we use in the wf or we can leave them untouched? notice that each lora sheet has something below, for example:
Recommended strength: 1.0
Recommended steps: 8
Recommended guidance: 7.0
Recommended shift: 3.0
@leoisleo glad you like it.
You dont need to tweak the parameters when using Loras, like I kept the values as is, the only thing I changed was to load the "pure" AceStep 1.5 Turbo model on slot 1 and the "pure" Turbo XL on slot 3 as I think those sound better with the Loras on (compared to the merges).
However you can tweak the values as you like it, just ensure a Turbo model requires CFG =1. SFT, Base and merges might go with CFG >1, but that only makes real sense for slot 4, where there is a negative prompt applied.