Ferrite AI Gateway gives applications one model-call interface while keeping provider credentials, outbound authority, budgets, and usage evidence at a governed boundary.
Add AI Gateway to a stack
Deploy apps/ferrite-ai-gateway as a private sibling and grant the calling app
only the queue or API capability required to submit a model request. Provider
hosts belong in the gateway’s egress allowlist, not every consumer app.
Use packages/ai-budgets to attach account, tenant, application, model, and
purpose attribution to each request. Reject a call before provider egress when
the applicable budget cannot admit it.
Stream model output
The Gateway’s audited egress path supports chunk-by-chunk streaming, so tokens can reach the app without buffering the complete response. Bound request size, provider timeout, total response bytes, and retry behavior.
Protect credentials and prompts
- Store provider credentials as write-only Ferrite secrets.
- Permit only declared provider origins, methods, paths, and headers.
- Do not log bearer tokens or raw sensitive prompts.
- Attribute every charge before returning success.
- Keep retries idempotent and avoid retrying a partially consumed stream unless the provider contract supports it.
Observe usage
Application usage and gateway attribution remain separate evidence. Read app
requests, bytes, latency, and credits in the Dashboard or with the MCP usage
tool; inspect gateway model/provider totals through its own bounded read model.
See Secrets & Egress and Analytics & Usage.