Configuration

The gateway takes its configuration from three places, applied in this order: built-in defaults, then a YAML file if one is passed, then the environment. Later wins. Nothing is required — a gateway started with no file and no variables runs on the defaults — except the admin bootstrap key, without which nothing can reach the admin plane.

cognigate --config cognigate.config.yml

cognigate.config.yml in the repository lists every key with its default value and a note on what it is for. It is a valid configuration file, not a sketch of one: pass it as-is and the gateway behaves exactly as it would with no file at all.

Environment variables

Secrets belong here, not in the file. Every variable is read under two spellings — CG_<NAME> and <NAME> — so a deployment can namespace them or not.

The analytics service is a separate process with its own configuration. It reads standard Spring properties:

SPRING_DATASOURCE_URL=jdbc:postgresql://postgres-db:5432/cognigate
SPRING_DATASOURCE_USERNAME=cognigate_user
SPRING_DATASOURCE_PASSWORD=<your password>

# The shared bearer the gateway presents. No default: absent, the service
# refuses to start rather than publish an open metering API.
ANALYTICS_TOKEN=<generate one>

The options that matter most

The bootstrap key

admin:
    bootstrap_key: "" # set CG_ADMIN_BOOTSTRAP_KEY instead

The one credential that exists before any tenant does. There is no default, because a built-in value would be public knowledge. A value shorter than 16 characters is refused rather than honoured — a placeholder copied out of an example file should not become a deployment's admin password.

Use it once, to mint a real admin key through /admin/v1/admin-keys, then take it out of the environment.

What a tenant is charged

billing:
    mode: passthrough # passthrough | markup | absorb
    markup_pct: 0
    cost_visibility: hidden # hidden | hint | exact

Every usage row records two figures: what the request cost at the provider's rate, and what its tenant owes for it. This decides the second one, for the whole gateway.

passthrough, the default, charges the provider rate itself, so the charge tracks what the provider charges without a second price table to maintain and the operator keeps nothing. markup adds markup_pct on top, and is refused at startup if the percentage is missing. absorb charges nothing and leaves the whole bill with the operator, with usage still attributed per tenant and per key.

A model with no published rate costs zero, so it is charged zero in every mode. Quota caps are measured in cost rather than charge, so a spend limit still works under absorb.

cost_visibility decides how much of the first figure a tenant sees. hidden, the default, publishes none of it: the data plane carries no cost_usd key, and a cost quota slot is withheld with it since cap minus remaining is the consumption. hint publishes the cost rounded to one significant figure instead, which leaves a tenant able to see roughly what its traffic cost while making the exact charge beside it useless for working out the margin. exact publishes what the gateway computed. The admin plane, the stored row, the charge itself and quota enforcement are exact under all three. Usage & cost has the detail.

Quota enforcement

quotas:
    default_soft_threshold_pct: 80
    enforcement: "on" # on | observe

observe is how to size quotas before they start refusing traffic: caps still emit events and set X-CogniGate-Quota-State, but a request over the cap is served. Switch to on once the numbers look right.

Response caching

cache:
    enabled: false
    default_ttl: 5m
    max_ttl: 24h
    max_entry_bytes: 262144 # the largest response worth storing
    max_bytes: 67108864 # the cache as a whole

Off by default. An identical prompt returning a stored answer is correct only where the caller expects it, so this is a decision, not a default. A cached response is marked with X-CogniGate-Cache, and a tenant can be given its own TTL and its own POST /admin/v1/tenants/{tenant}/cache/flush.

max_entry_bytes decides what may be stored; max_bytes decides how much is kept. Without the second, a cache that refuses nothing individually still grows without limit collectively.

Limits

limits:
    max_request_bytes: 2097152 # 2 MiB
    max_response_bytes: 8388608 # 8 MiB
    request_timeout: 120s
    upstream_connect_timeout: 10s
    stream_idle_timeout: 60s
    max_concurrent_per_key: 32

rate_limit:
    requests_per_second: 50
    burst_capacity: 100
    requests_per_minute: 3600

These are the deployment's ceilings. A tenant may be given lower ones through PATCH /admin/v1/tenants/{tenant}; it cannot be given higher.

stream_idle_timeout is the one worth understanding: a stream that goes quiet for longer than this is ended with upstream_stream_stalled rather than left holding a connection open indefinitely.

The three rate figures are two ceilings, not one. requests_per_second and burst_capacity are a token bucket, which shapes traffic second by second; requests_per_minute is a plain count against a wall-clock minute, which is how an allowance is usually quoted to a customer. The default clears what the bucket admits in a minute, so out of the box it never refuses a request the bucket would have allowed. Lower it to make it the binding one, or set it to zero to switch it off. Both are measured before either is charged, so a request one refuses does not spend the other's allowance, and the Retry-After a client gets is the longer of the two waits.

Rate is per tenant and concurrency is per key, deliberately. A tenant that could lift its own rate ceiling by minting another key would not have one; a concurrency cap exists to stop one integration starving another inside the same tenant, which is a distinction that only means anything per key.

For a requests-per-day allowance, set a quota rather than a rate limit. quota.day.requests is the per-day counterpart, and it is a quota because a day is not a rate: it resets at a fixed hour, reads back with a remaining figure, can be scoped to a single key and can be run in observe mode first. Set a quota has the shape of one.

Fallback and circuit breaking

routing:
    max_fallback_depth: 5
    breaker:
        error_threshold: 5
        window: 30s
        open_duration: 60s

max_fallback_depth bounds how many providers one request may try. The breaker counts failures per provider: error_threshold of them inside window opens it, and it stays open for open_duration before a probe is allowed through.

System instructions

chat:
    system_instruction: ""

The one thing the gateway adds to a completion request. The text goes in front of every prompt as a system message, ahead of whatever the caller sent, and empty (the default) means nothing is added and the body reaches the provider as it arrived. CG_CHAT_SYSTEM_INSTRUCTION sets it from the environment, which is usually where it belongs: it is the value most likely to differ between two deployments of the same image.

A tenant can be given its own text as well:

curl -X PATCH http://localhost:8080/admin/v1/tenants/ten_7f3a \
  -H "Authorization: Bearer $COGNIGATE_ADMIN_KEY" \
  -H "Content-Type: application/json" \
  -d '{"system_instruction": "Never mention the weather."}'

Both are then sent as one system message, the deployment's text first, separated by a blank line. Neither is merged into a system message the caller wrote: that one still arrives after both, in the caller's own words.

That is what the gateway sends. What an upstream does with two system messages is the upstream's own behaviour, and it varies: Anthropic's OpenAI-compatible endpoint documents that it hoists every system and developer message into one initial system message joined by newlines, in order, so the instruction still leads. Google's OpenAI-compatibility page states no rule either way.

Read the billing line before switching this on. The injected text is prompt text like any other, so the provider counts its tokens and they land in cost_usd and in whatever the tenant is charged. A long instruction on high-volume traffic is a recurring cost, not a one-off.

Debug capture

debug:
    max_capture_ttl: 72h
    default_capture_ttl: 24h
    default_sample_rate: 0.01
    max_capture_bytes_per_tenant: 33554432 # 32 MiB
    capture_sweep_interval: 1m

Capture stores prompt and completion content, which is the one place in the product where request content is served back out. max_capture_ttl may be lowered but never raised above 72 hours — a value above the ceiling is rejected at startup, not quietly clamped.

TLS

gateway:
    tls_cert_file: ""
    tls_key_file: ""

Both planes are served by one listener, so this single pair turns the whole process to HTTPS and cannot leave one plane plaintext by accident. Set both or neither; half a pair fails at startup rather than silently serving plaintext.

The recommended production shape is still plaintext behind a private network or an operator-owned reverse proxy. This is the escape hatch for a deployment with neither.