Phone Voice Integration

Turn your OpenClaw into a phone-callable assistant with ElevenLabs Agents + Twilio.

What you get:

📞 Call your bot from any phone
🔐 Caller ID authentication + voice PIN security
🛡️ Call screening (whitelist-based)
🧠 Full memory context (loads MEMORY.md, USER.md)
💰 Cost tracking per call
📝 Call transcripts with summaries
⏱️ Rate limiting
🌐 Permanent tunnel (Cloudflare) or temporary (ngrok)

Architecture

Phone → Twilio → ElevenLabs Agent → Your Bridge → Anthropic Claude → OpenClaw Tools
                                          ↓
                                    Memory Context
                                    (MEMORY.md, USER.md)

Flow:

Caller dials your Twilio number
Twilio routes to ElevenLabs Agent
Agent sends chat completions to your bridge (mimics OpenAI API)
Bridge translates to Anthropic, injects context from memory files
Claude response → ElevenLabs TTS → caller hears it

Prerequisites

OpenClaw installed and running
ElevenLabs account + API key
Twilio account + phone number
Anthropic API key
Cloudflare tunnel or ngrok (for exposing localhost)

Setup

1. Enable Chat Completions in OpenClaw

Not needed for this skill — the bridge bypasses OpenClaw and calls Claude directly. This gives you more control over memory injection and cost tracking.

2. Create the Bridge Server

The bridge is a FastAPI server that:

Accepts OpenAI-compatible
```
/v1/chat/completions
```
requests from ElevenLabs
Injects memory context (MEMORY.md, USER.md, live data)
Calls Anthropic Claude API
Streams responses back in OpenAI format
Logs costs and transcripts

Key files:

```
server.py
```
— FastAPI app with /v1/chat/completions endpoint
```
fred_prompt.py
```
— System prompt builder (loads memory files)
```
.env
```
— Secrets (API keys, tokens, whitelist)
```
contacts.json
```
— Caller whitelist for screening

3. Set Up Cloudflare Tunnel (Recommended)

Permanent, secure alternative to ngrok:

# Install cloudflared
brew install cloudflare/cloudflare/cloudflared
Login and configure
cloudflared tunnel login
cloudflared tunnel create <tunnel-name>
Run the tunnel

cloudflared tunnel --url http://localhost:8013 run <tunnel-name>

Add a CNAME in Cloudflare DNS:

voice.yourdomain.com → <tunnel-id>.cfargotunnel.com

Or use ngrok (temporary):

ngrok http 8013

4. Configure ElevenLabs Agent

Option A: Manual (UI)

Go to ElevenLabs dashboard → Conversational AI
Create new agent
Under LLM settings → Custom LLM

Set URL:

https://voice.yourdomain.com/v1/chat/completions

Add header:

Authorization: Bearer <YOUR_BRIDGE_TOKEN>

Option B: Programmatic (API)

# Step 1: Store your bridge auth token as a secret
curl -X POST https://api.elevenlabs.io/v1/convai/secrets \
  -H "xi-api-key: YOUR_ELEVENLABS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "new",
    "name": "bridge_auth_token",
    "value": "YOUR_BRIDGE_AUTH_TOKEN"
  }'
Response: {"secret_id": "abc123..."}
Step 2: Create the agent

curl -X POST https://api.elevenlabs.io/v1/convai/agents/create 

-H "xi-api-key: YOUR_ELEVENLABS_API_KEY" 

-H "Content-Type: application/json" 

-d '{
"conversation_config": {
"agent": {
"language": "en",
"prompt": {
"llm": "custom-llm",
"prompt": "You are a helpful voice assistant.",
"custom_llm": {
"url": "https://voice.yourdomain.com/v1/chat/completions",
"api_key": {"secret_id": "abc123..."}
}
}
}
}
}'

5. Connect Twilio Phone Number

In ElevenLabs agent settings:

Go to Phone section
Enter Twilio Account SID and Auth Token
Select your Twilio phone number
Save

Done! Your bot now answers that phone number.

Security Features

Caller ID Authentication

Recognizes whitelisted numbers automatically:

// contacts.json
{
  "+12505551234": {
    "name": "Alice",
    "role": "family"
  }
}

Voice PIN Challenge

For unknown callers or high-security actions:

VOICE_PIN = "banana"  # Set in .env

Caller must say the PIN to proceed.

Call Screening

Unknown numbers get a receptionist prompt:

"This is Fred's assistant. I can take a message or help with general questions."

Rate Limiting

Configurable per-hour limits:

RATE_LIMIT_PER_HOUR = 10

Prevents abuse and runaway costs.

Memory Injection

The bridge auto-loads context before each call:

Files read:

```
MEMORY.md
```
— Long-term facts about user, projects, preferences
```
USER.md
```
— User profile (name, location, timezone)
Recent call transcripts (cross-call memory)

Live data injection:

Current time/date
Weather (optional, via API)
Calendar events (optional, via gog CLI)

All injected into the system prompt before Claude sees the conversation.

Cost Tracking

Every call logs to

memory/voice-calls/costs.jsonl

{
  "call_sid": "CA123...",
  "timestamp": "2026-02-03T10:30:00",
  "caller": "+12505551234",
  "duration_sec": 45,
  "total_cost_usd": 0.12,
  "breakdown": {
    "twilio": 0.02,
    "elevenlabs": 0.08,
    "anthropic": 0.02
  }
}

Run analytics on the JSONL to track monthly spend.

Usage Example

Call your bot:

Dial your Twilio number
If you're whitelisted → casual conversation starts
If you're unknown → receptionist mode
Ask it to check your calendar, send a message, set a reminder, etc.

Outbound calling (optional):

curl -X POST https://voice.yourdomain.com/call/outbound \
  -H "Authorization: Bearer <BRIDGE_TOKEN>" \
  -d '{"to": "+12505551234", "message": "Reminder: dentist at 3pm"}'

Configuration Options

Environment variables (.env):

ANTHROPIC_API_KEY=sk-ant-...
ELEVENLABS_API_KEY=sk_...
ELEVENLABS_AGENT_ID=agent_...
TWILIO_ACCOUNT_SID=AC...
TWILIO_AUTH_TOKEN=...
TWILIO_NUMBER=+1...
LLM_BRIDGE_TOKEN=<random-secure-token>
VOICE_PIN=<your-secret-word>
CLAWD_DIR=/path/to/clawd

Whitelist (contacts.json):

{
  "+12505551234": {"name": "Alice", "role": "family"},
  "+12505555678": {"name": "Bob", "role": "friend"}
}

Advanced: Office Hours

Restrict calls to business hours:

# In server.py
OFFICE_HOURS = {
    "enabled": True,
    "timezone": "America/Vancouver",
    "weekdays": {"start": "09:00", "end": "17:00"},
    "weekends": False
}

Outside hours → voicemail prompt.

Debugging

Test the bridge directly:

curl -X POST https://voice.yourdomain.com/v1/chat/completions \
  -H "Authorization: Bearer <BRIDGE_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": false
  }'

Check logs:

tail -f ~/clawd/memory/voice-calls/bridge.log

Verify Twilio webhook:

Call your number
Check Twilio console → Call logs → Webhook status
Should see 200 responses from ElevenLabs

Cost Estimates

Per-minute breakdown:

Twilio: ~$0.01/min (inbound) + carrier fees
ElevenLabs TTS: ~$0.05/min (varies by voice quality)
Anthropic Claude: ~$0.01/min (depends on token usage)
Total: ~$0.07-0.10/min (~$4-6/hour of talk time)

Use rate limiting and call screening to control costs.

Comparison: This vs Basic Tutorial

ElevenLabs official tutorial:

✅ Basic integration
❌ No security
❌ No memory persistence
❌ No cost tracking
❌ Temporary ngrok URL

This skill (Phone Voice v2.0):

✅ All of the above
✅ Caller ID + PIN security
✅ Cross-call memory
✅ Cost tracking & analytics
✅ Permanent tunnel (Cloudflare)
✅ Rate limiting
✅ Call screening
✅ Transcript logging

License

MIT — use freely, credit appreciated.

Built by Fred (@FredMolty) — running on OpenClaw since 2026.

Phone Voice Integration

AI Skill Market Insights

Be Part of the 0+ Developer Community