From screenplay to one multi-voice MP3: parsing scripts for a multi-speaker TTS API on Cloudflare Workers

Most text-to-speech tools take one voice and one block of text. If you want a conversation, you generate each character separately, then line up the clips in an audio editor and fiddle with the gaps until it stops sounding like a voicemail tree. Newer TTS models can perform a whole scene in a single request: several speakers, one voice per speaker, with acting directions inline. I built a small web app on top of that, where you paste a script and download one MP3. Most of the interesting engineering wasn't the audio. It was turning the loose text people actually write into a strict payload the model accepts, without letting users break it. The stack is one Cloudflare Worker (plain JavaScript, no framework) serving static pages and a small JSON API. D1 holds users and generations, and R2 stores the MP3s. The model is Fish Audio's Drama 3, called over its HTTP API. The input format: a screenplay humans can type The format I settled on is as close to a radio script as possible: Narrator: [calm, low voice] The lighthouse had been dark for forty years. Mara: [whispering] Did you see that? Someone is up there. Jonas: [nervous laugh] It's the wind, Mara. Mara: [short pause] [firm] Then why did the door just open? One turn per line, as Name: line. Delivery directions go in [square brackets]. The model performs them rather than speaking them. … stresses a word. A line without a name belongs to the Narrator. That last rule matters more than it looks. People paste prose paragraphs between dialogue lines, and "this line has no speaker, so it's an error" is a terrible experience. Defaulting to a narrator means almost any text produces something. Parsing: one regex and a word-count heuristic Each line runs through a single regular expression: // "Name: line" or "Name [direction]: line". Names: up to 40 chars, no brackets/colons. const LINE_RE = /^([^\s:[\]|][^:[\]|]{0,39}?)\s*(\[[^\]]*\])?\s*:\s*(.*)$/; It accepts Mara: hi and Mara [whispering]: hi. If a direction sits before the colon, it gets moved into the line text, so both forms produce the same output. The hard case is a colon in ordinary prose. The plan she whispered was simple: run before dawn. shouldn't create a character called "The plan she whispered was simple". Two cheap guards cover most of this. The name can't contain brackets, colons, or |, and it must be five words or fewer: const m = LINE_RE.exec(line); let speaker = NARRATOR; let text = line; if (m && m[1].trim().split(/\s+/).length /g, '').trim(); } It's the same idea as escaping SQL. User text is data, and the delimiters belong to you. Brackets and are deliberately left alone, because those are features. Building the payload Once the turns are clean, the request body is short. Speaker indices follow the order characters first appear in, and reference_id is a positional array with one voice per index: export function buildFishPayload(turns, characters, voiceMap) { const index = new Map(characters.map((c, i) => [c, i])); const text = turns.map((t) => `${t.text}`).join('\n'); const referenceIds = characters.map((c) => voiceMap[c]); return { text, referenceIds }; } A useful side effect: the speaker tags depend only on character order, not on which voices are chosen. So the exact size of the request is known before casting, and the length check runs during validation, ahead of any expensive work. Auto-casting without duplicates Users can pick a voice per character, but most don't, at least on their first try. The app offers a fixed set of 16 curated stock voices (there's deliberately no voice cloning), and unassigned characters are filled in like this: Keep every voice the user chose explicitly. Give the Narrator the designated narrator voice, if nobody has taken it. Walk the rest of the pool round-robin, skipping voices already in use. Only when the pool runs out does a voice get reused. for (let tries = 0; tries < pool.length; tries++) { const cand = pool[(i + tries) % pool.length]; if (!used.has(cand)) { pick = cand; i = (i + tries + 1) % pool.length; break; } } The server also rejects any voice ID that isn't in that list, so a hand-crafted API request can't reach voices you haven't vetted. Calling the model: the header that fails silently The request is a plain fetch from the Worker: const res = await fetch('https://api.fish.audio/v1/tts', { method: 'POST', headers: { authorization: `Bearer ${env.FISH_AUDIO_API_KEY}`, 'content-type': 'application/json', model: 'drama-3', // missing header silently falls back to another model }, body: JSON.stringify({ text, reference_id: referenceIds, format: 'mp3', mp3_bitrate: 128, latency: 'normal', normalize: true, }), signal: AbortSignal.timeout(170_000), }); The comment on model is the most valuable line in the file. The provider's docs say a missing or misspelled model header doesn't fail. The request quietly falls back to a different model. Your tests pass, audio comes back, and you're running the wrong model. Pin it, and put a comment on it. Two more defensive checks: Treat a tiny response as a failure. A 200 with a body under 1 KB isn't a usable MP3, so the code throws instead of storing it. Never log the text or the key. Errors are logged as fish_status_ only. Scripts are user content. Knowing the cost before you call TTS pricing is usually per character of input, and a long script with lots of directions costs more than its audio length suggests. The app estimates from the exact string it's about to send, counted in UTF-8 bytes. That's never less than the character count, and it naturally charges more for non-Latin scripts, where a character is two to four bytes: export const billedBytes = (fishText) => new TextEncoder().encode(fishText).length; Users' credits are counted in seconds of audio, so the rule is "audio length or script size, whichever is greater". When a generation starts, a conservative amount is reserved, and the unused part is returned once the real length is known. A failed generation is fully refunded. If you build something similar, make the estimate and the final charge use the same function on both client and server, or users will see one number in the editor and another on their balance. Getting the real length doesn't need an audio decoder. The request asks for 128 kbps constant bitrate, which is 16,000 bytes per second: const seconds = Math.max(1, Math.ceil(audio.byteLength / 16000)); // 128 kbps CBR That's accurate enough for billing and needs no dependency. It only works because the bitrate is pinned in the request. With variable bitrate you'd have to parse frames. The tradeoff I accepted: one synchronous request The whole generation happens inside the user's HTTP request: validate, reserve, call the model, store to R2, settle. No queue, no polling. Pros: much less code, the response carries the finished generation, and there's no job state machine to get wrong. Cons: long scripts can run into request time limits, and if the client disconnects mid-generation, the Worker may be cancelled. The code handles that by blocking a second generation while one is pending, and a stale pending row stops blocking after five minutes. With a 6,000-character cap, a single scene stays a manageable size, so synchronous is the right call for now. If chapters get longer, the next step is a queue and a "your drama is ready" state. I'd rather add that when real usage needs it. Bonus gotcha: iOS won't play your sample MP3s The landing page has sample players. They worked everywhere except Safari on iOS, which wants 206 Partial Content responses to Range requests before it will play media. The Workers static-assets handler answered Range requests with a full 200. The fix is a small handler for /samples/*.mp3. It fetches the asset, parses a single bytes=start-end range (including the bytes=-N suffix form), and returns 206 with Content-Range, or 416 if the range can't be satisfied. Anything it doesn't understand falls back to a full 200, which RFC 9110 allows: headers.set('content-range', `bytes ${r.start}-${r.end}/${size}`); headers.set('content-length', String(r.end - r.start + 1)); return new Response(buf.slice(r.start, r.end + 1), { status: 206, headers }); If your audio "works on desktop but not on iPhone", check the status code before you check the codec. Checklist Default unnamed lines to a narrator instead of rejecting them. Show the parsed cast before spending money, so a misparse is cheap. Strip the provider's control tags from user text. Pin the model header, and comment why. Estimate cost from the exact payload, with the same function in both places. Pin the bitrate if you want duration without a decoder. Serve media with Range support. If you want to hear what a one-pass, multi-voice scene sounds like before writing any code, the AudioDrama samples are generated from the scripts shown on the page, unedited. Signing in gives you 2 free minutes to try your own script. What would you change in the parser? I'm especially curious how others handle Name: false positives in free-form text.