Inference routing

A coding task can involve several model requests and tool calls. taoCode coordinates that loop locally while the chosen gateway serves inference.

Request lifecycle

Your prompt and project context
  → local agent session
  → TAO provider resolves the model reference
  → authenticated request to the selected gateway
  → model response and tool requests
  → local tool execution
  → tool results returned for the next model request
  → final response

Secure transport↗

1. Client
2. Encrypted connection
3. Gateway

TLS encrypts traffic between your client and the gateway. The gateway terminates that connection. Encryption through to an enclave requires a separate supported transport.

What leaves the machine

Inference requests can include your prompt, relevant file content, conversation context, and tool results. The gateway receives the request even though tool execution happens locally.

Read Privacy & decentralization before choosing a gateway for sensitive work.

What happens when a request fails

Authentication errors, unavailable models, rate limits, and tool failures can interrupt the loop. Check the reported error, gateway connection, and selected reference before rerunning the task. Review any files already changed by the interrupted session.

Selecting a different route

Choose another full model reference to use a different gateway or model. This requires credentials and access for that gateway; changing the reference does not transfer account credit between services.