Personal R&D · 2026 · Music tools

FLVSTX

A composer that lives inside FL Studio. Describe what you want and it writes chords, melody, bass, drums and eight more layer kinds into a piano roll you can edit, hear through built-in sounds, and drag straight onto your instruments.

CLAP + VST3 + standalone
Rust · nih-plug · egui
Claude Agent SDK
Ollama, local or remote
32 composer tools
Windows

The stills below are one turn, frozen. The video is the plugin working in real time.

Music models hand you finished audio you cannot open. A chat window hands you advice you still have to play in. FLVSTX sits between the two: the output is MIDI in your own project, layer by layer and section by section, written by something that can see the whole song and explain each choice in a musician's terms.

It is a CLAP and VST3 plugin plus a standalone app, written in Rust, with a small TypeScript sidecar that runs the agent. Generation, playback, editing and export all work with no model at all; the composer is the part that listens.

Ask for a change, read why it was made

I told it the chorus melody felt flat. It read the chorus and the verse, rewrote the line a fourth higher as one rising arc built from the verse motif, filled the hats out to sixteenths, and then pointed at the next problem: chords and pad running under the bass. The notes in the piano roll are the ones it wrote.

Claude, or a model on your own machine

The header switches between Claude and an Ollama server, local or across the network, with the model list read from that server and a Think switch for reasoning models. Each provider keeps its own conversation, so going back to Claude resumes where Claude left off. Local turns cost nothing and leave nothing.

Inside FL Studio, three ways out

FL Studio has no MIDI effect slot, so the plugin routes live MIDI through Patcher, drags a layer straight onto a channel, or hands a whole song to a piano-roll import script. A built-in General MIDI synth means the song is audible before a single instrument is chosen.

What makes it different

The model decides, the engine plays

A language model asked for raw notes writes stiff, plausible-looking MIDI. Here it decides form, harmony and motif, and a Rust rules engine realises them: voice-led chords, motif-developed melodies, bass that follows the kick, grooves with a pocket per style. Repeats are new performances, not copies, and sustained parts carry swells, pedal and 808 glides.

Plans before it writes

Producer mode reads the session and proposes a plan: key, form, and for every step the exact layer and section it will touch. Nothing changes until it is approved. Then each step goes to a specialist whose whole context is that craft: harmony and form, melody and topline, rhythm section, arrangement and mix.

Enforced by the host, not asked for in a prompt

A tool-use hook denies every write until the plan is approved, and denies a specialist writing outside its own layers. Progress is derived from a ledger of writes that actually landed, so the checklist cannot claim work nobody did. One checkpoint reverts a whole run, which is far more than undo can hold.

The plugin is the source of truth

The agent never holds the song. Every one of its 32 tools is a thin call back into the plugin's session over a local socket, the same operations the piano roll and the command line use. Edit a note by hand and the composer sees it on its next turn.

How a turn runs

Three processes, one song

Plugin · Rust

Holds the session, the piano roll, the synth and the undo history. Sends the message with a compact summary of the song, then answers the agent's tool calls by running the same operations the UI uses.

Sidecar · TypeScript

Runs the Claude Agent SDK session with the composer tools, the approval fence, the specialists and the ledger. It keeps no notes of its own: every read and write is a call back to the plugin.

Model · Claude or Ollama

Claude through the user's own login, or any tool-calling model on an Ollama server. The model picks the tools and the music; the engine turns its decisions into a performance.

The bug worth telling

Adding local models

Ollama speaks the same API the agent runtime does, so pointing one at the other should have been a base URL. The first local turn came back confident and wrong: the model described calling a tool, in invented XML, and the song never changed.

The server log had the answer in one line: truncating input prompt, limit=2050. That endpoint has no way to set a context window, so the composer's brief and all 32 tool definitions, about ten thousand tokens, were being cut to the last two thousand. The model had never seen a tool. It was improvising one from the user's sentence.

The fix is a small bridge inside the sidecar that translates each request to Ollama's native chat API, where the context window can be set and capped at what the model supports, and streams the reply back in the shape the runtime expects. Tool calls, tool results and reasoning all round-trip, and it is unit-tested against a stand-in server. The same turn then made a real tool call and the key changed.