Smart Proxy Patterns for AI Agents

Multi-agent systems have a boilerplate problem. Every agent-to-agent communication requires serialization, deserialization, error handling, retry logic, timeout management, and logging. Multiply this by every connection between every agent, and you're spending more time on plumbing than on intelligence.

Smart proxies solve this by sitting between agents and handling the communication infrastructure. The agents send simple messages. The proxy handles everything else: retries, transforms, routing, logging, circuit breaking, and protocol translation.

The result is agents that focus on their domain expertise while the proxy handles the messy reality of distributed communication.

Key Takeaways

• Smart proxies reduce inter-agent communication code by 70% by centralizing serialization, error handling, and retry logic
• Protocol bridging lets agents written in different languages communicate seamlessly through a shared proxy layer
• Response transformation proxies convert between agent output formats without modifying either agent
• Circuit breaker proxies prevent cascade failures by detecting unhealthy agents and routing around them
• Logging proxies create a complete audit trail of every agent interaction for debugging and compliance

Why Agents Need Proxies

The Direct Communication Problem

In a naive multi-agent architecture, every agent talks directly to every other agent:

Agent A → Agent B (JSON over HTTP)
Agent A → Agent C (Protobuf over gRPC)
Agent B → Agent C (JSON over WebSocket)
Agent B → Agent D (Markdown over stdin)

Each connection requires:

• Format negotiation
• Error handling specific to the transport
• Retry logic tuned to the destination
• Timeout configuration
• Authentication (if agents are on different machines)

For N agents, you have N(N-1)/2 possible connections, each with its own communication code. At 10 agents, that's 45 connections. At 20, it's 190.

The Proxy Solution

A smart proxy centralizes communication through a single hub:

Agent A → Proxy → Agent B
Agent A → Proxy → Agent C
Agent B → Proxy → Agent C
Agent B → Proxy → Agent D

Every agent knows one protocol: how to talk to the proxy. The proxy handles the translation to each destination agent's preferred format and transport.

Pattern 1: Request Router Proxy

The simplest smart proxy routes requests to the appropriate agent based on content:

interface AgentMessage {
  intent: string
  payload: any
  source: string
  timestamp: number
}

class RequestRouterProxy {
  private routes: Map<string, AgentHandler> = new Map()

  register(intent: string, handler: AgentHandler) {
    this.routes.set(intent, handler)
  }

  async route(message: AgentMessage): Promise<any> {
    const handler = this.routes.get(message.intent)

    if (!handler) {
      throw new Error(`No agent registered for intent: ${message.intent}`)
    }

    console.log(`Routing ${message.intent} from ${message.source} to ${handler.name}`)

    return handler.process(message.payload)
  }
}

// Usage
const proxy = new RequestRouterProxy()
proxy.register('analyze-code', codeAnalysisAgent)
proxy.register('generate-tests', testGenerationAgent)
proxy.register('review-pr', prReviewAgent)

The router proxy decouples agents from each other. The code analysis agent doesn't need to know that test generation exists. It sends its results to the proxy, and the proxy decides what happens next.

This is the same principle behind multi-agent coordination skills, but implemented at the infrastructure level rather than the skill level.

Pattern 2: Response Transformation Proxy

Different agents produce output in different formats. A code analysis agent might return structured JSON. A documentation agent might return markdown. A test agent might return executable code. The transformation proxy normalizes these outputs.

class TransformationProxy {
  private transformers: Map<string, OutputTransformer> = new Map()

  registerTransformer(agentId: string, transformer: OutputTransformer) {
    this.transformers.set(agentId, transformer)
  }

  async transformResponse(agentId: string, rawOutput: any): Promise<StandardOutput> {
    const transformer = this.transformers.get(agentId)

    if (!transformer) {
      // Default: wrap raw output in standard envelope
      return {
        agentId,
        timestamp: Date.now(),
        format: 'raw',
        data: rawOutput,
      }
    }

    return transformer.transform(rawOutput)
  }
}

// Transformer for a markdown-producing agent
const markdownTransformer: OutputTransformer = {
  transform(rawOutput: string): StandardOutput {
    return {
      agentId: 'doc-agent',
      timestamp: Date.now(),
      format: 'markdown',
      data: rawOutput,
      metadata: {
        wordCount: rawOutput.split(/\s+/).length,
        hasCodeBlocks: rawOutput.includes('```'),
        headings: rawOutput.match(/^#+\s+.+$/gm) || [],
      },
    }
  },
}

Transformation proxies are essential when building agent pipelines where one agent's output feeds into another agent's input. The proxy ensures format compatibility without either agent needing to know the other's format.

Pattern 3: Circuit Breaker Proxy

In production multi-agent systems, individual agents can fail, become slow, or produce garbage output. A circuit breaker proxy detects these failures and prevents them from cascading:

enum CircuitState {
  CLOSED = 'closed',     // Normal operation
  OPEN = 'open',         // Agent is unhealthy, reject requests
  HALF_OPEN = 'half-open' // Testing if agent has recovered
}

class CircuitBreakerProxy {
  private state: CircuitState = CircuitState.CLOSED
  private failureCount = 0
  private lastFailure = 0
  private readonly threshold = 5
  private readonly resetTimeout = 30000 // 30 seconds

  async execute(request: AgentMessage): Promise<any> {
    if (this.state === CircuitState.OPEN) {
      if (Date.now() - this.lastFailure > this.resetTimeout) {
        this.state = CircuitState.HALF_OPEN
      } else {
        throw new Error('Circuit is open: agent is unhealthy')
      }
    }

    try {
      const result = await this.agent.process(request)
      this.onSuccess()
      return result
    } catch (error) {
      this.onFailure()
      throw error
    }
  }

  private onSuccess() {
    this.failureCount = 0
    this.state = CircuitState.CLOSED
  }

  private onFailure() {
    this.failureCount++
    this.lastFailure = Date.now()
    if (this.failureCount >= this.threshold) {
      this.state = CircuitState.OPEN
    }
  }
}

Circuit breakers are critical for agent systems that include external API calls. If one agent relies on a third-party service that goes down, the circuit breaker prevents the entire agent pipeline from stalling on timeouts.

Pattern 4: Caching Proxy

AI agent calls are expensive. A caching proxy stores results and serves them from cache when the same request appears:

class CachingProxy {
  private cache: Map<string, CachedResult> = new Map()

  async process(request: AgentMessage): Promise<any> {
    const cacheKey = this.computeKey(request)
    const cached = this.cache.get(cacheKey)

    if (cached && !this.isExpired(cached)) {
      console.log(`Cache hit for ${request.intent}`)
      return cached.result
    }

    const result = await this.agent.process(request)
    this.cache.set(cacheKey, {
      result,
      timestamp: Date.now(),
      ttl: this.getTTL(request.intent),
    })

    return result
  }

  private computeKey(request: AgentMessage): string {
    return `${request.intent}:${JSON.stringify(request.payload)}`
  }

  private getTTL(intent: string): number {
    // Different intents have different cache lifetimes
    const ttls: Record<string, number> = {
      'analyze-code': 300000,    // 5 minutes
      'categorize-skill': 3600000, // 1 hour
      'generate-metadata': 86400000, // 24 hours
    }
    return ttls[intent] || 60000 // Default: 1 minute
  }
}

Caching proxies pair well with the build cache strategies used in AI project builds. The same principles of content-based keys and TTL management apply.

Pattern 5: Logging and Observability Proxy

The most universally useful proxy pattern: a transparent wrapper that logs every interaction:

class ObservabilityProxy {
  async process(request: AgentMessage): Promise<any> {
    const startTime = performance.now()
    const requestId = crypto.randomUUID()

    console.log(JSON.stringify({
      event: 'agent_request',
      requestId,
      agent: this.agentId,
      intent: request.intent,
      source: request.source,
      timestamp: new Date().toISOString(),
    }))

    try {
      const result = await this.agent.process(request)
      const duration = performance.now() - startTime

      console.log(JSON.stringify({
        event: 'agent_response',
        requestId,
        agent: this.agentId,
        duration: Math.round(duration),
        success: true,
      }))

      return result
    } catch (error) {
      const duration = performance.now() - startTime

      console.log(JSON.stringify({
        event: 'agent_error',
        requestId,
        agent: this.agentId,
        duration: Math.round(duration),
        error: error.message,
      }))

      throw error
    }
  }
}

This proxy creates a complete audit trail of every agent interaction. When debugging a multi-agent system, the log reveals exactly which agent failed, how long it took, and what it was trying to do.

Composing Proxies

The real power comes from composing multiple proxy patterns into a stack:

// Build a proxy stack: logging → caching → circuit breaker → agent
const agent = new CodeAnalysisAgent()
const circuitBreaker = new CircuitBreakerProxy(agent)
const cache = new CachingProxy(circuitBreaker)
const logger = new ObservabilityProxy(cache)

// All requests go through the full stack
const result = await logger.process(request)

Each layer adds its concern without the other layers (or the agent) knowing about it. This is the decorator pattern applied to agent communication.

FAQ

When should I use a proxy vs. direct agent communication?

Use proxies when you have 3+ agents that communicate with each other, when you need logging or retry logic, or when agents use different communication formats. For 2 agents with a simple request-response pattern, direct communication is simpler.

Do proxies add latency to agent communication?

Minimal latency for in-process proxies (microseconds). Network proxies add network round-trip time. Caching proxies can dramatically reduce total latency by avoiding expensive agent calls entirely.

How do I handle streaming responses through a proxy?

Use async iterators or event streams. The proxy forwards chunks as they arrive rather than waiting for the complete response. This is essential for proxying AI model responses that stream token by token.

Can I use existing API gateway tools as agent proxies?

Yes. Tools like Kong, Envoy, or even Nginx can serve as smart proxies for agents that communicate over HTTP. They provide routing, rate limiting, and circuit breaking out of the box. Custom proxies are needed for non-HTTP protocols or AI-specific transformations.

How do I test proxy behavior?

Mock the downstream agents and verify the proxy's behavior: correct routing, proper caching, circuit breaker state transitions, and accurate logging. Proxy tests should cover both happy paths and failure scenarios.

Sources

• Microsoft Design Patterns: Proxy -- Enterprise proxy and ambassador patterns
• Martin Fowler: Circuit Breaker -- Circuit breaker pattern definition
• Node.js Streams Documentation -- Streaming patterns for proxy implementations
• Anthropic Agent SDK -- Building AI agents with proxy support

Explore production-ready AI skills at aiskill.market/browse or submit your own skill to the marketplace.