How the network works
Mycellios turns several heterogeneous machines into a small number of useful execution stages. It does not assume that every connected node belongs in every request.
From model to route
- Profile the model. Read tensor metadata, architecture, memory needs, and the certified execution adapter.
- Inspect capacity. Collect node memory, runtime, link, and availability evidence.
- Form cells. Group devices that can cooperate over a low-latency boundary.
- Place stages. Assign contiguous layers or compatible experts to the smallest useful route.
- Execute. Send activations between stages while each stage keeps its own KV cache.
- Observe and recover. Apply deadlines, cancellation, health checks, and a compatible alternate route when one exists.
Coordinator and data path
The coordinator is the control plane. It registers workers, advertises models, selects replicas, preserves session affinity, and streams the public response.
The stage runtimes form the model data path. They exchange activations over persistent connections. Keeping the coordinator out of the activation relay avoids one unnecessary central hop.
Why routes stay short
Decoder stages remain dependent across layers. Adding another WAN hop can cost more latency than the extra machine contributes in compute. Mycellios therefore prefers:
- a few thick stages instead of hundreds of thin ones;
- tensor or expert parallelism inside a host or fast cell;
- replicas and concurrent routes when extra nodes cannot shorten one request;
- excluding a slow node when it makes the selected route worse.
Prefill and decode differ
Prefill moves and computes a larger prompt window, so memory and bandwidth can dominate. Decode repeatedly advances a small token step, so round-trip time and the number of stage boundaries become more important. The planner may choose different priorities for the two phases.