ElevenLabs Integration

High-quality, low-carbon text-to-speech with GateFlow's sustainability optimization.

Available Models

TTS Models

• eleven_turbo_v2_5 - Fastest, most efficient voice synthesis
• eleven_flash_v2_5 - Balanced speed and quality
• eleven_multilingual_v2 - Supports 29+ languages

Voice Options

• Standard Voices: 30+ pre-built voices across genders and accents
• Custom Voices: Create and clone custom voices
• Emotional Voices: Expressive speech with emotional tone

Sustainability Features

ElevenLabs through GateFlow offers:

• Carbon-Optimized TTS: Up to 40% lower carbon footprint vs traditional TTS
• Efficient Audio Processing: Minimal compute for high-quality speech
• Time-Shifted Execution: Defer audio generation to low-carbon periods
• Batch Processing: Generate multiple audio files in single optimized requests

Example Usage

Basic Text-to-Speech

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gateflow.ai/v1",
    api_key="gw_prod_your_key_here"
)

# Generate speech with carbon optimization
response = client.audio.speech.create(
    model="eleven_turbo_v2_5",
    voice="rachel",  # Standard ElevenLabs voice
    input="This is a sustainably generated audio message.",
    routing_mode="sustain_optimized"  # Enable carbon-optimized routing
)

# Save the audio file
with open("output.mp3", "wb") as f:
    f.write(response.content)

print(f"Carbon footprint: {response.sustainability.carbon_gco2e} gCO₂e")
print(f"Carbon saved: {response.sustainability.carbon_saved_gco2e} gCO₂e")

Using Sustain Mode with ElevenLabs

# Let GateFlow choose the most sustainable ElevenLabs model
response = client.audio.speech.create(
    model="elevenlabs:auto",  # Auto-select most efficient ElevenLabs model
    voice="rachel",
    input="Generating this audio with minimal environmental impact.",
    routing_mode="sustain_optimized",
    minimum_quality_score=85
)

print(f"Selected model: {response.model}")
print(f"Audio duration: {response.duration_seconds}s")
print(f"Carbon intensity: {response.sustainability.carbon_per_second_gco2e} gCO₂e/s")

Time-Shifted Audio Generation

# Defer audio generation to low-carbon period
response = client.audio.speech.create(
    model="eleven_turbo_v2_5",
    voice="rachel",
    input="This podcast episode will be generated during low-carbon hours.",
    routing_mode="time_shift_optimized",
    max_delay_hours=12,  # Will generate within 12 hours when carbon intensity is lowest
    priority="normal"
)

print(f"Job ID: {response.deferred_job_id}")
print(f"Estimated carbon savings: {response.estimated_carbon_savings_gco2e} gCO₂e")

ElevenLabs-Specific Features

Voice Customization

# Use custom voice with specific settings
response = client.audio.speech.create(
    model="eleven_multilingual_v2",
    voice="custom_voice_id",  # Your custom voice ID
    input="Personalized voice with sustainability benefits.",
    voice_settings={
        "stability": 0.75,
        "similarity_boost": 0.85,
        "style": 0.3,
        "use_speaker_boost": True
    }
)

Emotional Speech

# Generate speech with emotional tone
response = client.audio.speech.create(
    model="eleven_turbo_v2_5",
    voice="rachel",
    input="I'm excited about our sustainability initiatives!",
    voice_settings={
        "stability": 0.6,
        "similarity_boost": 0.7,
        "style": 0.8,  # Higher style for more emotional expression
        "use_speaker_boost": True
    }
)

Batch Audio Generation

# Generate multiple audio files in one optimized request
batch_response = client.audio.speech.batch_create(
    model="eleven_turbo_v2_5",
    voice="rachel",
    inputs=[
        "First audio segment with sustainability focus.",
        "Second audio segment about carbon reduction.",
        "Third audio segment on renewable energy."
    ],
    routing_mode="sustain_optimized",
    batch_optimization="carbon_efficient"  # Optimize batch for minimal carbon impact
)

for i, audio_file in enumerate(batch_response.audio_files):
    with open(f"segment_{i}.mp3", "wb") as f:
        f.write(audio_file.content)
    print(f"Segment {i}: {audio_file.sustainability.carbon_gco2e} gCO₂e")

Sustainability Best Practices

Model Selection Guide

Use Case	Recommended Model
Fast narration	eleven_turbo_v2_5
Balanced quality	eleven_flash_v2_5
Multilingual content	eleven_multilingual_v2
Long-form audiobooks	eleven_turbo_v2_5 + batch

Optimization Tips

1. Use turbo model: eleven_turbo_v2_5 offers best carbon efficiency
2. Batch processing: Generate multiple files in single optimized request
3. Time-shifting: Defer non-urgent audio generation to low-carbon periods
4. Right-size quality: Use appropriate model for your quality needs
5. Reuse voices: Avoid creating multiple similar custom voices

Performance Characteristics

Audio Quality

• Bitrate: 192 kbps (high quality)
• Sample Rate: 44.1 kHz
• Format: MP3 (default), also supports WAV, OGG

Performance Metrics

• Generation Speed: 50-200 characters/second
• Latency: 200-800ms for first chunk, streaming thereafter
• Max Duration: 5,000 characters per request (~5 minutes audio)

Integration with Other GateFlow Features

Multi-Provider Audio Pipeline

# Combine STT → LLM → TTS with sustainability optimization
stt_response = client.audio.transcriptions.create(
    model="whisper-1",  # OpenAI STT
    file=open("input_audio.mp3", "rb"),
    routing_mode="sustain_optimized"
)

# Process transcript with LLM
llm_response = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": f"Summarize this sustainably: {stt_response.text}"}],
    routing_mode="sustain_optimized"
)

# Generate speech with ElevenLabs
tts_response = client.audio.speech.create(
    model="eleven_turbo_v2_5",
    voice="rachel",
    input=llm_response.choices[0].message.content,
    routing_mode="sustain_optimized"
)

# Total carbon footprint
total_carbon = (
    stt_response.sustainability.carbon_gco2e +
    llm_response.sustainability.carbon_gco2e +
    tts_response.sustainability.carbon_gco2e
)
print(f"Total pipeline carbon: {total_carbon} gCO₂e")

Semantic Caching for Audio

# Cache frequent TTS requests
response = client.audio.speech.create(
    model="eleven_turbo_v2_5",
    voice="rachel",
    input="Frequently requested audio message about sustainability.",
    cache_ttl_seconds=86400,  # Cache for 24 hours
    cache_key="welcome_message_v2"  # Custom cache key
)

Troubleshooting

"ElevenLabs API key not configured"

Solution: Add your ElevenLabs API key in the GateFlow Dashboard under Settings → Providers.

"Voice not found: custom_voice_id"

Solution: Ensure the voice ID exists in your ElevenLabs account and is properly configured in GateFlow.

"Audio generation timed out"

Solution:

1. Reduce input text length (max 5,000 characters)
2. Use time-shifted execution for long audio
3. Check ElevenLabs API status
4. Configure fallbacks to other TTS providers

"Carbon savings lower than expected"

Solution:

1. Verify Sustain Mode is properly configured
2. Check grid carbon intensity in your region
3. Try different ElevenLabs models
4. Enable time-shifted execution for better savings

Migration from Direct ElevenLabs API

Key Differences

Feature	Direct ElevenLabs API	GateFlow Integration
API Format	ElevenLabs-specific	OpenAI-compatible
Authentication	ElevenLabs API key	GateFlow API key
Model Names	eleven_multilingual_v2	eleven_multilingual_v2
Carbon Tracking	Manual	Automatic
Multi-provider	No	Yes
Fallbacks	Manual	Automatic
Sustainability	Basic	Advanced optimization

Migration Example

Before (Direct ElevenLabs API):

from elevenlabs import generate, play

audio = generate(
    text="Hello from ElevenLabs!",
    voice="Rachel",
    model="eleven_multilingual_v2"
)
play(audio)

After (GateFlow Integration):

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gateflow.ai/v1",
    api_key="gw_prod_your_gateflow_key"
)

response = client.audio.speech.create(
    model="eleven_multilingual_v2",
    voice="rachel",
    input="Hello from ElevenLabs via GateFlow with sustainability benefits!"
)

with open("output.mp3", "wb") as f:
    f.write(response.content)

Next Steps

• Explore Cohere Integration - Efficient chat and rerank models
• Configure Sustain Mode - Automatic carbon optimization
• Set Up Time-Shifted Execution - Defer audio generation to low-carbon periods
• View Audio Analytics - Track your ElevenLabs carbon savings