This is awesome! I started experimenting with LLMs and Ableton over the break too. I used a few off the shelf VST templates to create a VST that leverages an LLM that can create LilyPond format music. This creates multi track VST output (MIDI) from the parsed LilyPond. I have a system prompt that explains LilyPond to the LLM (it already knows anyhow), and then I give it a prompt like "Megaman, 64 bars, ice man level", and it pumps out 4-5 tracks of MIDI that is just what I asked for. I get nice sheet music on top of that.
I like this because I'm really in the creative process still, it feels like a tool like "Scaler" (nice tool btw) where you're picking origin notes, but really putting the song together yourself. It can suggest "Synth Bass" but I'm the one assigning it from one of my other VSTs sounds, mixing, picking, etc.. And if all goes well playing some lead on the guitar or seaboard and then deleting it all and starting over.
Excellent! I tried to use Claude on the Ableton file format about a year ago and it left me quite frustrated -- but now I have a new reason to look at this again.
Generally, it would be nice of Ableton to release an official documentation of their API.
I've been vibe-coding a diff tool[1] for Ableton Live project files in my spare time, though the project is still far from complete. It's meant to generate human-readable, meaningful summary text that shows the differences between two versions of a project file (.als). With this diff tool, I can then use Git to properly version-control Ableton Live project files.
So far I've completed roughly 70% of the Ableton Live project-file XML parsing, though some parts like Session View and the Groove Pool are not finished yet.
As for using Claude or other agents to parse Ableton Live's XML, my original plan was to build an automated workflow with ableton-mcp: have Claude use ableton-mcp to make edits in a blank project (for example, add an EQ8 or modify some automation), save the project file, then have Claude compare the modified project file with the original blank project and write the corresponding parsing code. But ableton-mcp still lacks many features[2], and the XML schema of .als files is inconsistent, so I ended up doing most of the review and verification manually.
It’s a method like any other. The process took him two days, after the LLM suggested which track to mix (like he was working for the LLM instead of the other way around?). Two days is more than enough to learn how to make a mashup in Ableton without LLM’s.
Seems though that working with those AI tools appeals to the author, that they learned and had fun, so I guess that makes it a good usecase for them specifically?
Author here, I think you’re spot on. If I set out specifically to make a mashup and spent the same number of hours working toward that goal I think I could have gotten to the same place. This was a journey that started with wrapping my head around MCP and LLM local software interoperability, and my Ableton knowledge leveled up a lot along the way.
I think this tooling could be useful in the hands of more capable musicians / audio engineers / etc. as there are often repetitive tasks in DAWs and it could potentially unlock new workflows that would have been too tedious without knowing how to program.
Awesome stuff. Seems to me like one great use case of an MCP for Ableton would be to develop muscle / workflow memory for music production workflows. Adjacently, I've started using Perplexity a bunch for this sort of thing because it indexes YouTube tutorial transcripts. Any thoughts on how to design MCPs for learning Ableton better?
One thing I don’t cover in the blog post is ensuring your MCP tool calls are well documented. (If it’s an existing MCP you can do this with a README or instructions file.) I saw a jump in efficiency when I manually edited the docstrings with examples of when each tool would be used, how to call it, and better argument descriptions.
That's cool! Have you thought about some docstring augmentation loop -- either by having the agent log the tool uses or by auto querying (e.g. perplexity API) for example use cases in the wild?
I set up my own Ableton MCP rig last year to do this. It works extremely well. I have had Claude organize, rename, colour, and prepare 100+ track songs for mixing. Unlike the OP I have not used it for any kind of creative use, although I did experiment with giving it access to my vst library and trying commands like "put a vocal compressor on the lead vox, with a high ratio and a fast attack", with absolutely terrible results.
The organizational and assistive ability though was/is outstanding, and using it in my own projects honestly made me sad that we're seeing AI integration everywhere that automates the creative work and leaves you with the tedious work. I don't expect we'll see AI updates to DAWs from vendors anytime soon with features like "speak into your mic and tell your DAW to group all the snares, color code them different shades of red, and rename them", but that's the part of the process I wish we were trying to get AI to tackle.
This is awesome! I started experimenting with LLMs and Ableton over the break too. I used a few off the shelf VST templates to create a VST that leverages an LLM that can create LilyPond format music. This creates multi track VST output (MIDI) from the parsed LilyPond. I have a system prompt that explains LilyPond to the LLM (it already knows anyhow), and then I give it a prompt like "Megaman, 64 bars, ice man level", and it pumps out 4-5 tracks of MIDI that is just what I asked for. I get nice sheet music on top of that.
I like this because I'm really in the creative process still, it feels like a tool like "Scaler" (nice tool btw) where you're picking origin notes, but really putting the song together yourself. It can suggest "Synth Bass" but I'm the one assigning it from one of my other VSTs sounds, mixing, picking, etc.. And if all goes well playing some lead on the guitar or seaboard and then deleting it all and starting over.
Excellent! I tried to use Claude on the Ableton file format about a year ago and it left me quite frustrated -- but now I have a new reason to look at this again.
Generally, it would be nice of Ableton to release an official documentation of their API.
The trick would have been to ask Claude to write a program that can parse/serialize from/to Ableton's XML format in the way you need it to. It's just XML, but a ton of it: https://github.com/stevengt/ableton-xml-analysis/blob/dev/ed...
I've been vibe-coding a diff tool[1] for Ableton Live project files in my spare time, though the project is still far from complete. It's meant to generate human-readable, meaningful summary text that shows the differences between two versions of a project file (.als). With this diff tool, I can then use Git to properly version-control Ableton Live project files.
So far I've completed roughly 70% of the Ableton Live project-file XML parsing, though some parts like Session View and the Groove Pool are not finished yet.
As for using Claude or other agents to parse Ableton Live's XML, my original plan was to build an automated workflow with ableton-mcp: have Claude use ableton-mcp to make edits in a blank project (for example, add an EQ8 or modify some automation), save the project file, then have Claude compare the modified project file with the original blank project and write the corresponding parsing code. But ableton-mcp still lacks many features[2], and the XML schema of .als files is inconsistent, so I ended up doing most of the review and verification manually.
[1]: https://github.com/krfantasy/alsdiff [2]: https://github.com/ahujasid/ableton-mcp/pulls
Thankfully session view is for crazy people that can be ignored :)
There's also one for Bitwig: https://github.com/WeModulate/bitwig-mcp-server
There goes my productivity for the weekend. :-D
This is super cool. Using LLMs to mix together two songs that don't belong mixed together is a great usecase.
It’s a method like any other. The process took him two days, after the LLM suggested which track to mix (like he was working for the LLM instead of the other way around?). Two days is more than enough to learn how to make a mashup in Ableton without LLM’s.
Seems though that working with those AI tools appeals to the author, that they learned and had fun, so I guess that makes it a good usecase for them specifically?
Author here, I think you’re spot on. If I set out specifically to make a mashup and spent the same number of hours working toward that goal I think I could have gotten to the same place. This was a journey that started with wrapping my head around MCP and LLM local software interoperability, and my Ableton knowledge leveled up a lot along the way.
I think this tooling could be useful in the hands of more capable musicians / audio engineers / etc. as there are often repetitive tasks in DAWs and it could potentially unlock new workflows that would have been too tedious without knowing how to program.
Awesome stuff. Seems to me like one great use case of an MCP for Ableton would be to develop muscle / workflow memory for music production workflows. Adjacently, I've started using Perplexity a bunch for this sort of thing because it indexes YouTube tutorial transcripts. Any thoughts on how to design MCPs for learning Ableton better?
One thing I don’t cover in the blog post is ensuring your MCP tool calls are well documented. (If it’s an existing MCP you can do this with a README or instructions file.) I saw a jump in efficiency when I manually edited the docstrings with examples of when each tool would be used, how to call it, and better argument descriptions.
That's cool! Have you thought about some docstring augmentation loop -- either by having the agent log the tool uses or by auto querying (e.g. perplexity API) for example use cases in the wild?
Would be forever grateful for a repo to learn from, even if its rough.
https://github.com/jhurliman/ableton-mcp/pull/1
Excited to try batch renaming clips and batch warping in very large sets. Haven’t been able to figure out ways to automate these tedious tasks.
I set up my own Ableton MCP rig last year to do this. It works extremely well. I have had Claude organize, rename, colour, and prepare 100+ track songs for mixing. Unlike the OP I have not used it for any kind of creative use, although I did experiment with giving it access to my vst library and trying commands like "put a vocal compressor on the lead vox, with a high ratio and a fast attack", with absolutely terrible results.
The organizational and assistive ability though was/is outstanding, and using it in my own projects honestly made me sad that we're seeing AI integration everywhere that automates the creative work and leaves you with the tedious work. I don't expect we'll see AI updates to DAWs from vendors anytime soon with features like "speak into your mic and tell your DAW to group all the snares, color code them different shades of red, and rename them", but that's the part of the process I wish we were trying to get AI to tackle.
related https://github.com/vroomai/live
Is there a FOSS version of Ableton?
I have yet to try it but I believe zrythm would be the closest.