Architecture Overview

CogniGate is two processes with an unusually sharp line between them. The Go gateway serves every request a client ever makes and holds all of its own configuration in memory. The Java analytics service stores usage records in PostgreSQL and does nothing else.

The line is drawn where it is because of what each side must survive. A gateway that cannot answer until a database answers is a gateway with a database's availability, and routing traffic is the job it cannot fail at. Metering is the opposite: a usage record that is lost is money that cannot be billed, so it needs somewhere durable to live. Neither requirement is served well by the same store, so they do not share one.

System topology

Loading diagram...
  • Go gateway (:8080) — the whole data plane and the whole admin plane. Authentication, model catalogues, capability aliases, fallback chains, circuit breakers, quotas, budgets, rate and size limits, response caching, events, webhooks and the audit log all live here, in process.
  • Java analytics (:8081) — one endpoint the gateway writes to and two it reads from, and a scheduled job that prices a window of usage. It has no configuration API and no other consumer.
  • PostgreSQL (:5432) — usage records. Nothing else is stored here, because nothing else is sent here.
  • LLM providers — whatever a tenant has credentials for. CogniGate reaches them over their own APIs; it does not run models.

Request lifecycle

  1. Authentication. A client sends POST /v1/chat/completions with a data key. The gateway resolves the key to a tenant from its own store — no network call, no cache to be stale.
  2. Resolution. The requested model may be a real model id or a capability alias (fast, best, whatever the tenant defined). The alias resolves against the tenant's live catalogue, which the gateway refreshes from each configured provider.
  3. Routing and fallback. The resolved model has an ordered chain behind it. The gateway tries the first provider; on a failure that a retry could fix it moves to the next, up to a bounded depth, and reports how far it got in X-CogniGate-Fallback-Depth. A provider that keeps failing has its breaker opened and is skipped until it recovers.
  4. Execution. The request is forwarded, and the response is streamed back chunk by chunk if it was a streaming request. Token counts come from the provider's own usage block.
  5. Metering. The usage record is dispatched asynchronously. The client's response has already been completed by then; a slow or absent analytics service costs latency to nobody.

What the gateway keeps, and where

Every configuration resource — tenants, keys, providers, aliases, routing rules, quotas, webhooks, events, the audit log, cached responses and debug captures — lives in the gateway process. This is stated plainly rather than buried, because it has consequences an operator has to plan around:

  • A restart loses them. The reference deployment is a single gateway and the bootstrap admin key recreates the first tenant; a deployment that needs configuration to outlive a restart supplies it at startup.
  • Replicas do not share them. Two gateways are two independent configurations, two caches and two capture buffers. The gateway is horizontally scalable in the sense that any replica can serve any request it has been configured for — not in the sense that configuring one configures the rest.

Usage is the exception, and the exception is the whole reason the Java service exists. Records are shipped to it, written to PostgreSQL, and read back through GET /v1/usage and GET /v1/usage/breakdown.

Metering is decoupled on purpose

The gateway never waits on the analytics service, and it never drops a record because the service was down. A delivery that fails with a 5xx or a connection error is buffered and retried; a delivery refused with a 4xx is dropped after one attempt, because a malformed record retried forever would wedge everything queued behind it.

Records are idempotent on request_id. The gateway retries a delivery whose response it never saw, so the same record can legitimately arrive twice; the second arrival is recognised and answered 200 instead of billing twice.

This is why there is no depends_on: analytics in the reference deployment. The gateway builds its analytics client but does not dial it at startup, so an outage in the analytics service is an outage in exactly one service (GW-11).