ElevenLabs Text-to-Speech (via Letta)

Generate natural speech from text - supports 90+ languages, multiple models for quality vs latency tradeoffs. Examples default to eleven_v4, the latest and highest-quality model.

This skill is adapted from ElevenLabs' text-to-speech skill (elevenlabs/skills, MIT; see LICENSE). Instead of calling ElevenLabs with your own key, you call Letta's hosted endpoint POST /v1/audio/speech. It forwards the request to ElevenLabs with Letta's key and charges the organization's Letta credits, so you do not need an ElevenLabs API key. The request body uses the same field names as ElevenLabs "Create speech" (the voice ID and output_format go in the body), and the response is the audio file itself, as with ElevenLabs.

Quick Start

base_url="${LETTA_BASE_URL%/}"

curl -sS -X POST "$base_url/v1/audio/speech" \
  -H "Authorization: Bearer $LETTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello, welcome to ElevenLabs!","voice_id":"JBFqnCBsd6RMkjVDRZzb","model_id":"eleven_v4"}' \
  --output output.mp3 \
  --dump-header speech-headers.txt \
  --write-out '%{http_code} %{content_type}\n'

Use the runtime-provided LETTA_BASE_URL and LETTA_API_KEY together. Build URLs relative to ${LETTA_BASE_URL%/} and do not hardcode https://api.letta.com. If either variable is missing, the user needs to authenticate with Letta Cloud; do not ask for an ElevenLabs key.

A 200 with an audio/... content type means output.mp3 is the speech. Any other status means output.mp3 holds a JSON error with a message instead; read it (see "Error Handling") and do not share the file.

Share the saved file the way your current surface supports: attach or upload it to the conversation or channel you are replying in. Keep credit amounts out of replies unless the user asks about cost.

Word timings

Use POST /v1/audio/speech/with-timestamps for captions, cueing visuals to words, or lip-sync checks. It accepts the same body as plain speech (text, voice_id, model_id, voice_settings, and other options), with the same Letta authentication and billing; no ElevenLabs key is needed.

The response is JSON { audio_base64, alignment, normalized_alignment, words }. Each word is { text, type: "word", start, end }, with times in seconds. Words retain punctuation and use normalized spoken text when available; missing alignment produces an empty words array.

base_url="${LETTA_BASE_URL%/}"

curl -fsS -X POST "$base_url/v1/audio/speech/with-timestamps" \
  -H "Authorization: Bearer $LETTA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello world!","voice_id":"JBFqnCBsd6RMkjVDRZzb","model_id":"eleven_v4","output_format":"mp3_44100_128"}' \
  --output speech.json &&
jq -r '.audio_base64' speech.json | base64 --decode > output.mp3 &&
jq '.words' speech.json > words.json

Plain /v1/audio/speech still returns raw audio bytes.

Models

eleven_turbo_v2_5 and eleven_turbo_v2 are superseded by the Flash models (same output, lower latency) — use eleven_flash_v2_5 / eleven_flash_v2 instead.

Eleven v4 notes: supports only stability and similarity_boost voice settings (no style or speed) and does not support SSML. Direct delivery with audio tags such as [whispers], [laughs], [sarcastic] (see "Audio Tags").

Voice IDs

The Letta endpoint accepts ElevenLabs premade voices and voices Letta provides. If your memory names your own voice ID, use it whenever you speak as yourself. Other voices (cloned or designed voices from someone's own ElevenLabs account) are rejected with 400; to use one, call ElevenLabs directly with the user's own ELEVENLABS_API_KEY secret and ElevenLabs' text-to-speech skill.

Premade voices:

• JBFqnCBsd6RMkjVDRZzb - George (male, warm storyteller)
• EXAVITQu4vr4xnSDxMaL - Sarah (female, mature, reassuring)
• cgSgspJ2msm6clMCkdW9 - Jessica (female, playful, conversational)
• onwK4e9ZLuTAKqWW03F9 - Daniel (male, steady broadcaster)
• SAz9YHcvj6GT2YYXdXww - River (neutral, relaxed)

Voice Settings

Fine-tune how the voice sounds:

• Stability: How consistent the voice stays. Lower values = more emotional range and variation, but can sound unstable. Higher = steady, predictable delivery.
• Similarity boost: How closely to match the original voice sample. Higher values sound more like the original but may amplify audio artifacts.
• Style: Exaggerates the voice's unique style characteristics (v2/v3 models; not supported on Eleven v4).
• Speed: Speech speed multiplier (v2/v3 models; not supported on Eleven v4).
• Speaker boost: Post-processing that enhances clarity and voice similarity.

{
  "text": "Customize my voice settings.",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "model_id": "eleven_v4",
  "voice_settings": {
    "stability": 0.5,
    "similarity_boost": 0.75
  }
}

stability: lower = more expressive, higher = more consistent. similarity_boost: higher = closer to the reference voice. See references/voice-settings.md for ranges and use-case presets.

Audio Tags

Eleven v4 reads bracketed, natural-language cues in text as stage directions:

• Place a tag before the words it affects: [shouts] I can't believe you said that to me.
• A tag's emotion carries forward, so add another tag only where the delivery should shift.
• Combine tags inside one set of brackets: [whispering, playful]. Plain-language tags work, such as [nervous, trying to sound confident] or [phone buzzing].
• Use one emotion per clause; contrasting tags on the same words blur the performance.
• Pair tags with punctuation: ellipses slow a line, dashes cut a speaker off, and [pause] / [long pause] add beats.
• If a delivery misses, swap the tag (try [let down] instead of [despair]) and regenerate rather than rewriting the line.

Language Selection

Use language_code with models that support language enforcement to guide pronunciation and text normalization. Unsupported language codes are ignored, and language_code is not supported on eleven_multilingual_v2.

{
  "text": "Bonjour, comment allez-vous?",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "model_id": "eleven_v4",
  "language_code": "fr"
}

Text Normalization

Controls how numbers, dates, and abbreviations are converted to spoken words. For example, "01/15/2026" becomes "January fifteenth, twenty twenty-six":

• "auto" (default): Model decides based on context
• "on": Always normalize (use when you want natural speech)
• "off": Speak literally (use when you want "zero one slash one five...")

{
  "text": "Call 1-800-555-0123 on 01/15/2026",
  "voice_id": "JBFqnCBsd6RMkjVDRZzb",
  "model_id": "eleven_v4",
  "apply_text_normalization": "on"
}

Request Stitching

When generating long audio in multiple requests, the audio can have pops, unnatural pauses, or tone shifts at the boundaries. Request stitching solves this by letting each request know what comes before/after it:

{"text": "This is the first part.", "voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_v4", "next_text": "And this continues the story."}

{"text": "And this continues the story.", "voice_id": "JBFqnCBsd6RMkjVDRZzb", "model_id": "eleven_v4", "previous_text": "This is the first part."}

Each response has a request-id header (saved by --dump-header above, e.g. grep -i '^request-id:' speech-headers.txt). Instead of previous_text, you can pass up to three earlier request IDs in previous_request_ids (and later ones in next_request_ids when regenerating a middle part). Keep the same voice, model, and voice settings for every part, split long text at sentence or paragraph boundaries (each request takes up to 10,000 characters), and concatenate the files in order, for example with ffmpeg -f concat.

Output Formats

Pass output_format in the body, and name the file to match the format.

Error Handling

Errors return JSON with a message:

• 400 "Voice … is not available": use your own voice or a premade voice (see "Voice IDs").
• 400 with an ElevenLabs message: invalid parameters (check model_id, voice settings the model does not support, or text length).
• 402: the organization is out of Letta credits. Tell the user; do not retry.
• 500: retry once, then report the failure.

References

• Voice Settings