Monitoring & observability

The dashboard's Monitor section shows what's flowing through the gateway in real time and after the fact: a Live feed, searchable request Logs, a Reliability view, and Reports. This page covers each surface and the headers that unlock per-session attribution.

Live Monitor

The Live Monitor streams completed (and in-flight) requests over Server-Sent Events:

GET /api/events

Each event carries the provider, resolved model, HTTP status, token counts (prompt / completion / cache read / cache write / reasoning), latency, cost and charge, the gateway key label, the session id, and — on failure — an error_message. Events come in two phases: started (in-flight) and completed (finished), correlated by a stable request_id.

The stream is auth-gated when a dashboard password is set or multi-tenant mode is on (the token may arrive as Authorization: Bearer or a ?token= query param so browsers' EventSource can use it). In the wide-open local default it's open like the rest of the dashboard.

Request Logs

GET /api/logs                       # filtered, paginated list (metadata)
GET /api/logs/facets                # distinct provider/model/source values (filter dropdowns)
GET /api/logs/{request_id}          # full detail incl. attempt timeline
GET /api/logs/{request_id}/export   # detail as downloadable JSON
DELETE /api/logs/{request_id}       # remove a single row + its bodies

The list supports filtering by provider, model, source, session, api_key_id, account_id, a status shorthand (success / error / client / server), a since/until window, and a free-text q over the error message and model names. Each row records the source wire format, attribution (key, account, session), the requested vs resolved model, the outcome (status, error_kind, error_message, tokens), economics (cost, charge, latency, TTFB), and the per-attempt fallback timeline.

The Logs filter

The dashboard's Logs page builds its filter bar from your own recent traffic rather than free-text guessing. Provider and Source are dropdowns, and Model is a typeahead — all three populated from GET /api/logs/facets (the distinct values across the last 5,000 log rows). API key and Account are selects populated from your keys/accounts. Session id — an exact-match field you rarely need — lives under a collapsed Advanced disclosure instead of the main filter row. Every filter applies immediately (on change for selects, on blur/Enter for the text inputs) — there's no separate Apply step — and each active filter renders as a removable chip (✕) above the results, so what's currently filtering the view is always visible at a glance.

Body capture & the privacy posture

Capturing full request/response bodies is OFF by default — merido is privacy-first, so by default only metadata and (redacted) headers are stored. You can change this:

• Set MERIDO_LOG_BODIES=true at startup, or
• Toggle it at runtime from the Logs page (persisted via PUT /api/settings, no restart needed).

Two important details:

• Capture applies to future requests only. Turning it on does not retroactively fill in bodies for requests already logged.
• Failed requests are an exception. Even with blanket body logging off, bodies of requests that fail (status ≥ 400) are retained so you can inspect what went wrong post-hoc.

Whenever bodies are stored, they're scrubbed first: secrets are masked and PII (emails, phone numbers) is replaced before the body is compressed and written. Logs are pruned by age (MERIDO_LOG_RETENTION_DAYS, default 30) and total size (MERIDO_LOG_RETENTION_MAX_BYTES, default 2 GiB).

Bodies can age out sooner than the rest of the log. A stored body is the customer's actual prompt and response text; the log row around it is only who called, which model, what it cost, how long it took. Set MERIDO_LOG_BODY_RETENTION_DAYS to delete the bodies on a shorter window while keeping the metadata for the full retention period — the rows stay queryable for cost and reliability analysis with the text removed. Default 0 keeps the previous behaviour (bodies live exactly as long as their rows); a value above MERIDO_LOG_RETENTION_DAYS has no effect, since the row is deleted first.

Stored bodies are scrubbed before they are written: credentials are masked, then PII is replaced. The scrub is deliberately precise rather than aggressive — it masks credentials whose shape it recognises and nothing else:

Masked in stored bodies	Left alone
sk-…, sk-ant-… (OpenAI / Anthropic)	Model ids (claude-opus-4-20250514)
AKIA… + AWS secret keys	Request / message / session ids
ghp_… and bare 40-hex GitHub tokens	Content hashes, sha256:… digests
AIza… (Google), xox… (Slack)	base64 payloads, wallet addresses
sk_live_… / rk_live_… (Stripe)	Filesystem paths and URLs
PEM private-key blocks, JWTs	Ordinary prose
All PII (emails, phone numbers)

Earlier versions also ran a generic "long high-entropy string" guess. It flagged far more ordinary content than credentials — model ids, ids, hashes and paths all landed in the database as [REDACTED_SECRET], and unevenly (gpt-4o-2024-11-20 is just under the length bar and survived while the Claude ids did not), so a real redaction was indistinguishable from a mangled one. That guess is now off for stored bodies.

The trade-off is explicit: a credential in a format merido has no detector for now reaches the log in the clear. If you paste a raw MERIDO_MASTER_KEY or a bespoke internal token into a prompt, expect to find it in the stored body. Bodies are off by default, and MERIDO_LOG_BODY_RETENTION_DAYS limits how long they live. Text sent back to the client — an upstream provider's error body, a mid-stream error frame — is still scrubbed at full strength, high-entropy guess included.

Attribution headers

To group and attribute traffic by session, agent, or cost center, send these request headers (the first match wins for each):

• Session — x-merido-session (also accepts x-claude-code-session-id, x-session-id).
• Agent — x-merido-agent (also x-claude-code-agent-id, agent-id, x-agent-id).
• Cost center — x-merido-cost-center, for chargeback grouping in Reports (falls back to the gateway key's default cost center).

The request's source (its wire format — OpenAI / Anthropic / Responses / Gemini) is recorded automatically and is filterable in Logs and Reports.

Agent attribution without headers

Autonomous agent clients send none of the headers above, and their config formats have no place to put one — so attribution that depends on them stays empty for exactly the workload that runs unattended and spends the most.

merido does not guess. It is the component that writes the client's config, so it gives each agent its own model id — <model>#<agent> — and splits the suffix back off before routing. The request routes exactly as <model> would, and <agent> is stored as agent_label. The identity is declared by construction, so it cannot be misattributed.

• Filter by agent in Logs → Advanced → Agent.
• Cap an agent's spend with a budget scoped to agent. The agent scope is the innermost one — narrower than the gateway key, because one key routinely serves several agents.
• Turn correlation comes from the standard W3C traceparent header, stored as turn_id and shown in the Logs Turn column. Click a turn to collapse the table to that one tool-call loop — every request the agent made to answer once. It is not a session id: successive turns of the same conversation carry different trace-ids. Filter it directly under Advanced → Turn id.

Recovered labels, for agents merido did not provision

An agent added by hand after the config was written — or any client calling a plain model id — declares nothing. Rather than leave the Logs row blank, merido reads a display label out of the request itself when the client's shape is recognised: OpenClaw, for instance, injects its own workspace path into the system prompt, so …/uc-data/<workspace>/<agent>/… yields <workspace> / <agent>.

This is display metadata and nothing more:

• It names the row in Logs (and is filterable there).
• It is never a budget key. Only a declared <model>#<agent> suffix reaches the budget scope chain, so turning this on cannot make an agent-scoped budget start matching traffic it never matched before, and one client's label can never spend another's allowance.
• An unrecognised client stays unattributed, exactly as before — under-reporting is the safe direction, and nothing is ever folded into another agent's bucket.

Why merido does not infer the agent from the prompt

The obvious alternative is to infer identity: successive turns of one agent share a long, stable system prompt, so cluster requests by shared prefix. merido implemented that, then measured it against 62 captured production requests. It merged two different agents into one identity — the failure that would let one agent spend another's budget.

The cause is structural. These clients deliberately front-load the shared, cacheable part of the prompt to win prompt-cache hits and keep per-agent content below the cache boundary, so any prefix sample lands in the region engineered to be identical. On real traffic, two requests from the same agent shared as little as 512 B of prefix while two from different agents shared up to 1600 B — no threshold separates them, capped or uncapped.

Reliability

GET /api/usage/reliability?window=24h

Reports per-model error rate and truncation rate over a window (24h / 7d / 30d / all), so you can spot a flaky provider or model at a glance.

Reports & trends

GET /api/usage/timeseries?window=24h&group_by=provider   # spend / tokens over time
GET /api/reports?group_by=cost_center&format=json|csv     # showback / invoices
GET /api/reports/turns                                     # cost per agent TURN
GET /api/sessions/{session_id}/cost                        # one session's burn rate

Reports group spend and tokens by cost_center, model, source, day, api_key, user, virtual_model, platform, channel, and export as JSON or CSV. The charge and margin columns are meaningful in multi-tenant mode (where a virtual-model markup can make charge differ from cost); in single-user local mode charge equals cost.

Agent-fleet axes

Three axes exist for running a fleet of autonomous agents:

group_by	Buckets by	Answers
agent	Effective agent — the declared header, then merido's #agent model suffix, then the identity recovered from the request shape	What does each agent cost?
agent_workspace	The workspace/project an agent belongs to	What does a team/project cost?
account	The upstream account that served the request	Which subscription is this draining?

account is a capacity axis, not a chargeback one. Subscription accounts carry quotas, so "which account absorbed this spend" is a different question from "who pays for it" — that one is user.

Cross-tabs. Pass breakdown= to add a second dimension, e.g. ?group_by=agent&breakdown=account for which agent is draining which subscription. Available between agent, agent_workspace, account, virtual_model, platform and channel; pairing one with a rollup-backed axis (cost_center, day, api_key, user) returns 422 rather than silently falling back to the one-dimensional report.

Two columns worth reading first

• Errors — failed requests per bucket. Blank (not 0) on rollup-backed axes: the daily rollup stores successful requests only, so a failure never reaches it, and "unknown" is the honest answer. Available on the agent-fleet axes above.
• Cache hit — cache_read_tokens / prompt_tokens. For agent traffic this is usually the largest cost lever: an agent re-sends a large system prompt on every turn, and that context is either cached or paid for in full each time. A low ratio on one agent points at that agent's prompt.

Cost per turn

GET /api/reports/turns reports what one answer costs. Agent clients emit one W3C traceparent trace per turn, so every model call in a tool-call loop shares a turn_id — merido groups on it and returns, per agent: turns, requests per turn, and the cost distribution (mean, p50, p95, max).

The distribution is the point. Agent spend is heavy-tailed: one runaway loop can outweigh a day of ordinary turns, and a mean hides it while p95 does not. Requests that carry no traceparent are excluded rather than counted as one-request turns; turns_covered tells you how much of the window was measurable at all.

Prometheus metrics

GET /metrics    # Prometheus text-exposition (core + OpenTelemetry GenAI metrics)

/metrics is unauthenticated in the wide-open local default and auth-gated on a hardened deploy. Alongside the core families — merido_requests_total, merido_requests_failed_total, merido_prompt_tokens_total / merido_completion_tokens_total, merido_cost_usd_total / merido_charge_usd_total, merido_upstream_errors_total{class=…}, and the merido_request_latency_ms histogram — the gateway exposes telemetry self-observability counters so you can tell when it is shedding its own telemetry under load (a healthy "fail safe, observe everything" posture means these stay at zero):

Counter	Non-zero means
merido_usage_events_dropped_total	The background usage writer's buffer overflowed and usage/spend rows were dropped — your cost/usage totals are undercounting.
merido_request_logs_dropped_total	The background request-log writer's buffer overflowed and request-log rows were dropped — some requests won't appear in Logs.
merido_live_events_lagged_total	One or more Live-Monitor (SSE) subscribers fell behind and missed events — the live feed isn't complete (it does not affect stored usage/logs).
merido_unpriced_model_requests_total	A token-consuming request ran on a model the price table can't resolve, so its upstream cost was booked as $0. Add a price override (or sync prices) to restore accurate cost tracking.

All four reflect best-effort, non-blocking behavior: merido never adds request latency to record telemetry, so under sustained overload it drops rows rather than stalling traffic. A steadily rising drop/lag counter is the signal to raise capacity or scale out (the cloud profile fans writes across instances); a rising unpriced counter is the signal to fix pricing. The matching merido_unpriced_model_requests_total also surfaces in the dashboard Advisor as a recommendation that names the specific unpriced models to add a price for.

Two gauges show how the live stream is sized, so a rising merido_live_events_lagged_total can be acted on:

Gauge	Meaning
merido_live_event_subscribers	Live-event (SSE) subscribers connected right now.
merido_live_event_buffer_capacity	Configured broadcast buffer size (events retained for a briefly-lagging subscriber). Raise it with MERIDO_LIVE_EVENT_BUFFER (default 256) for high-RPS deployments with many live viewers.

Database connection pool

Three gauges expose the database pool. They exist because a drained pool looks exactly like an idle server from the outside: every caller is queued waiting for a connection, which consumes no CPU, so the machine reports ~0% CPU and near-zero traffic while answering nothing.

Gauge	Meaning
merido_db_pool_connections_in_use	Connections checked out right now (a query is holding them).
merido_db_pool_connections_idle	Established connections sitting idle, ready to hand out.
merido_db_pool_max_connections	The pool ceiling (MERIDO_DB_MAX_CONNECTIONS, default 10).

The alert to set: merido_db_pool_connections_in_use reaching merido_db_pool_max_connections and staying there. Brief touches are normal under load; a sustained pin means requests are queuing for connections and the pool is the bottleneck. Either the ceiling is too low for your concurrency, or something is holding connections longer than it should.

When the pool is drained, callers now fail after MERIDO_DB_ACQUIRE_TIMEOUT_MS (default 3000) with a loud error rather than blocking on the sqlx default of 30 seconds. The default is deliberately under the 5s health-check timeout of a typical proxy, so a drained pool surfaces as attributable errors instead of a silent outage.

Webhook deliveries get their own pair of counters, covering the outbox/dispatcher pipeline described in Webhooks → Metrics:

Counter	Non-zero/rising means
merido_webhook_deliveries_total{status}	Delivery attempts by outcome (delivered / failed / dead) — watch failed/dead per endpoint to catch an unhealthy receiver before it gets auto-disabled.
merido_webhook_endpoints_auto_disabled_total	An endpoint failed continuously for 5 days and was automatically disabled.

Related

• Usage & the Advisor — the cost/savings analytics layer on top of usage.
• Audit log & SIEM export — the control-plane change log (separate from request logs).
• Webhooks — push the events behind these counters to your own endpoints instead of polling /metrics.
• Configuration — MERIDO_LOG_BODIES and the log-retention knobs.