Channel affinity integration guide

Stable identifiers help continuous conversations, coding assistants, and long-context tasks reuse channels and context caches.

AGTCloud uses the stable identifier in a request to give subsequent requests priority on a channel that previously succeeded. If that channel becomes unavailable, the platform automatically selects another available channel. This identifier is called an affinity key.

Channel affinity stabilizes routing; context caching reuses computation. A stable channel does not guarantee a cache hit on every request.

Choose an integration method

Prefer identifier fields already available in your protocol. If changing the request body is inconvenient, your SDK or gateway can pass a stable identifier in a request header.

Use x-conversation-id across protocols

x-conversation-id passes a stable identifier in a request header, so no affinity identifier field is needed in the body. This works with protocols such as OpenAI, Anthropic, and Gemini:

curl 'https://api.agtcloud.ai/v1/chat/completions' \
  -H 'Authorization: Bearer your-api-key' \
  -H 'Content-Type: application/json' \
  -H 'x-conversation-id: conversation_3cb9dd9a4819594254a779c90a2171' \
  --data '{
    "model": "MODEL_ID",
    "messages": [
      {
        "role": "user",
        "content": "Continue analyzing this issue."
      }
    ]
  }'

This header is used for platform channel affinity and is not treated as a model parameter by default. Keep the same value throughout a continuous conversation rather than generating one for every request.

Replace MODEL_ID in the examples with your model ID, and use a model API key for authentication.

Integration by model family

OpenAI

OpenAI Prompt Caching is enabled automatically and primarily relies on matching request prefixes. Responses and Chat Completions can use prompt_cache_key for cache routing. AGTCloud also uses it for channel affinity and forwards it to upstream providers that support the field.

curl 'https://api.agtcloud.ai/v1/responses' \
  -H 'Authorization: Bearer your-api-key' \
  -H 'Content-Type: application/json' \
  --data '{
    "model": "MODEL_ID",
    "input": "Continue the current task.",
    "prompt_cache_key": "project_knowledge_v3"
  }'

An affinity key can identify a conversation, project, knowledge-base version, or repository. To continue an existing Responses conversation, also set previous_response_id to the response ID returned previously. It is not needed on the first request.

Claude / Anthropic

Claude marks reusable prefixes with content-block cache_control and also supports automatic multi-turn caching. metadata.user_id can serve as a platform affinity key; clients such as Claude Code often already send this kind of stable identifier. It is not a cache breakpoint and does not replace cache_control.

curl 'https://api.agtcloud.ai/v1/messages' \
  -H 'x-api-key: your-api-key' \
  -H 'anthropic-version: 2023-06-01' \
  -H 'Content-Type: application/json' \
  --data '{
    "model": "MODEL_ID",
    "max_tokens": 1024,
    "metadata": {
      "user_id": "conversation_3cb9dd9a4819594254a779c90a2171"
    },
    "system": [
      {
        "type": "text",
        "text": "Put stable business rules and tool constraints here.",
        "cache_control": {
          "type": "ephemeral"
        }
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "Continue analyzing this repository."
      }
    ]
  }'

Claude builds cache prefixes in the order tools, system, then messages. Changes to tool definitions, system instructions, or cached messages affect that level and subsequent cache levels. Short or longer TTLs can be selected according to upstream capabilities.

Gemini

Use x-conversation-id for channel affinity, and choose a caching method for your scenario:

• Implicit caching: Enabled by default for Gemini 2.5 and newer models; suitable for regular conversations that reuse a stable prefix over a short period.
• Explicit caching: Create a cache resource through /v1beta/cachedContents, then reference it with cachedContent. This suits long documents, long videos, fixed knowledge bases, and repository analysis. Your client manages cache creation, expiry, and subsequent references.

DeepSeek

DeepSeek automatically enables disk-based context caching and reuses complete, stable prefixes. deepseek-v4-flash and deepseek-v4-pro use user_id to distinguish users for safety processing, per-user rate limiting, and K/V Cache isolation.

Requests with the same model and user_id preferentially reuse a channel that previously succeeded.

curl 'https://api.agtcloud.ai/v1/chat/completions' \
  -H 'Authorization: Bearer your-api-key' \
  -H 'Content-Type: application/json' \
  --data '{
    "model": "MODEL_ID",
    "user_id": "user_123",
    "messages": [
      {
        "role": "system",
        "content": "Put stable business rules, tool descriptions, or a knowledge-base summary here."
      },
      {
        "role": "user",
        "content": "Continue analyzing this issue."
      }
    ]
  }'

user_id must contain only ASCII characters, be no longer than 512 characters, and remain stable for the same user. If a user ID is unavailable, use a stable identifier generated by your application. If you cannot change the body, use the x-conversation-id method above.

Default concurrency limits per user_id are:

If user_id is omitted, the platform assigns an available identifier from its configured UID pool. The request can still be sent normally.

Inspect usage.prompt_cache_hit_tokens and usage.prompt_cache_miss_tokens in the response for the numbers of cached and uncached tokens.

Qwen

Qwen / DashScope supports implicit caching, explicit caching, and Session Cache. For models such as qwen3.6-plus, qwen3.6-flash, and qwen3-coder-plus, choose according to the calling protocol:

• Chat Completions: Implicit caching is automatic. To mark reusable content explicitly, add cache_control to content blocks.
• Multi-turn Responses: Continue the response with previous_response_id and enable Session Cache with the request header x-dashscope-session-cache: enable.

Both methods can also send x-conversation-id for channel affinity. Here is a Chat Completions example:

curl 'https://api.agtcloud.ai/v1/chat/completions' \
  -H 'Authorization: Bearer your-api-key' \
  -H 'Content-Type: application/json' \
  -H 'x-conversation-id: conversation_3cb9dd9a4819594254a779c90a2171' \
  --data '{
    "model": "MODEL_ID",
    "messages": [
      {
        "role": "system",
        "content": [
          {
            "type": "text",
            "text": "Put stable long context exceeding the cache threshold here.",
            "cache_control": { "type": "ephemeral" }
          }
        ]
      },
      {
        "role": "user",
        "content": "Answer the question using the content above."
      }
    ],
    "stream": false
  }'

GLM / Z.AI

Models including GLM-5, GLM-5.1, GLM-4.7, GLM-4.6, and GLM-4.5 provide automatic context caching for long-running tasks, coding, and agents. Use a stable x-conversation-id for the same project or agent task:

curl 'https://api.agtcloud.ai/v1/chat/completions' \
  -H 'Authorization: Bearer your-api-key' \
  -H 'Content-Type: application/json' \
  -H 'x-conversation-id: agent_task_3cb9dd9a4819594254a779c90a2171' \
  --data '{
    "model": "MODEL_ID",
    "messages": [
      {
        "role": "system",
        "content": "Put stable agent task goals, tool constraints, and project context here."
      },
      {
        "role": "user",
        "content": "Continue with the next step."
      }
    ],
    "stream": false
  }'

Inspect usage.prompt_tokens_details.cached_tokens in the response for cached token usage. When retaining thinking content, also preserve the order and content of reusable historical segments.

Usage recommendations

• Keep prefixes stable: Put fixed instructions, tool definitions, long documents, and history summaries first; place the current question, time, and temporary logs last. Avoid reordering tools, rewriting unchanged history, or inserting random values.
• Choose identifiers by scope: Keep a conversation ID fixed within a conversation and a user ID fixed for that user over time. Distinguish projects and knowledge-base versions with their respective identifiers. Do not mix identifiers across tenants or substitute a shared project identifier for one that must distinguish users.
• Avoid sensitive plaintext: Do not use phone numbers, email addresses, identity document numbers, or API keys as affinity keys. You can irreversibly transform business identifiers. Do not use a one-time Request ID.
• Respect validity windows: The platform configures the affinity mapping lifetime; aligning it with the upstream cache window is recommended, commonly 5 minutes to 1 hour. Do not add an extra TTL to the request. Keeping an identifier unchanged does not extend upstream cache expiry.

Troubleshooting

A cache miss alone does not prove that channel affinity failed. For failed requests, see Errors and troubleshooting.

Related references

• OpenAI Prompt Caching
• Claude Prompt Caching
• Gemini Context Caching
• DeepSeek Context Caching
• Qwen Context Cache
• GLM / Z.AI Context Caching