AI guitar tone engine
Describe a tone in plain English — “Mayer’s Gravity, live” — and an agent dials it into the amp sims in real time.
I studied music theory and jazz guitar before I ever shipped software, and tone-chasing is the guitarist's oldest time sink — an hour of knob-twiddling to approximate a sound you can describe in one sentence. The sentence is the interface I actually wanted: plug in, say “Gravity, live version,” and play.
The hard part was never the AI. REAPER's built-in HTTP control surface has no verbs for plugin parameters at all — so before any model can touch a knob, something has to exist that can. The real build was the bridge: a long-running Lua script inside REAPER doing file-based IPC with a Python controller, plus a one-shot enumerator that dumps every parameter of every plugin — index, range, current value — into structured JSON. Build the system first; the model comes last.
A Claude agent with tool use sits on top of a tone knowledge base — per-plugin parameter maps with role aliases (so “treble” resolves to the right knob on each amp), 25 documented artist tones, a vocabulary file mapping words like “warm” and “scooped” to parameter tendencies, and genre defaults. The agent proposes settings as structured output; every value is validated against the enumerated map before it touches the live FX. A FastAPI server exposes it over REST and WebSocket to a chat-style web UI with live parameter state at 2 Hz.
One desktop shortcut takes the rig from cold to playing in about nine seconds — REAPER boots, the bridge heartbeats, the server launches, the browser connects.
It works — “make it brighter” reads the current state and adjusts; “something heavier” swaps amp sims; presets save and recall, including the six-effect vocal chain. The roadmap runs toward reference-audio tone matching (spectral analysis → parameter mapping) and hands-free control. It is also the most honest demo I have of the working method: same spec-first, knowledge-base-grounded, validate-everything architecture as the enterprise tools — built for the thing I loved first.
view the architecture
ai guitar tone engine — architecture (as it runs today)
strat -> scarlett 4i4 (hi-z) -> reaper (live sound template)
|
| hosts:
| track 1: jmx + nolly x amp sims
| track 2: 6-fx vocal chain
| track 3: analog lab (keys, midi)
v
bridge.lua (long-running reascript inside reaper)
^ file-ipc: _cmd_*.json / _resp_*.json
| heartbeat: _bridge_alive.json
v
python ToneController (get / set / snapshot / preset)
^
|
agent_cli.py (repl) web_server.py (fastapi :3000)
claude tool-use rest + websocket -> web ui
2 hz live param state
knowledge base injected into the system prompt:
plugins/ per-amp param names, ranges, role aliases
artists 25 named tones (gravity live, texas flood,
comfortably numb, polyphia clean lead...)
vocabulary tone words -> parameter tendencies
genres defaults per genre
delay slapback, dotted-eighth, ...
every parameter change is validated against the
enumerated map before it touches the live fx.Claude is the interface, not the system. The enumerated parameter map and the tone knowledge base are the actual product — the same rule as every tool at work, pointed at a Stratocaster.
the insight