Skip to main content

API quickstart

The coordinator exposes OpenAI-compatible model discovery and chat-completions routes. The examples below use a local coordinator at http://127.0.0.1:8787.

Check the coordinator​

curl http://127.0.0.1:8787/v1/models

Use a model identifier returned by this endpoint. A detected GPU alone does not make a model available; the coordinator publishes only registered deployments.

Send a completion​

curl http://127.0.0.1:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: docs-example-1" \
  -d '{
    "model": "distributed-small",
    "messages": [
      {
        "role": "user",
        "content": "Explain the active route in one sentence."
      }
    ],
    "max_tokens": 80,
    "session_id": "docs-session",
    "deadline_ms": 30000
  }'

Replace distributed-small with a model reported by your coordinator.

Stream tokens​

Set stream to true to receive a server-sent event stream:

{
  "model": "distributed-small",
  "messages": [{"role": "user", "content": "Hello"}],
  "stream": true
}

Keep the same session_id for related turns when you want the scheduler to preserve session affinity and reuse compatible stage state.

Deadlines and cancellation​

deadline_ms is an end-to-end request deadline. A caller should still apply its own connection timeout and close the request when the result is no longer useful. Mycellios propagates cancellation through the coordinator and worker path.

Full contract​

The versioned HTTP and worker contract lives in docs/openapi.yaml. Check the specification for supported fields and error responses before relying on an extension outside the OpenAI-compatible core.

Last updated on