Souq

What it is

Every week, for each Souq-enabled key with enough traced traffic, Souq combines two things: hard numbers from your gateway logs (calls, spend, latency, models, cache and savings activity) and a redacted sample of the key's captured conversations, which an LLM reads to describe recurring patterns. The result is one report per key per week, and past weeks stay browsable from the week selector at the top of the Report tab.

What's in a report

• At a glance— what's working, what's hindering, one quick win, and what's ahead.
• Activity — calls, spend, traces, error rate, latency and tokens, plus calls by day and hour, your top models, and which kinds of tools are calling the key.
• What you use the key for — the recurring kinds of work (coding, summarization, extraction, support and so on) with a breakdown of how often each shows up.
• How you prompt — techniques like role-setting, few-shot examples and explicit constraints, plus how requests are structured: how often they use a system prompt, define tools, ask for JSON, run multi-turn, or repeat an identical request.
• Wins and friction— what's working well, and the patterns that cost you retries or quality, with short paraphrased examples.
• Suggestions — concrete things to try, many with a copy-paste snippet you can drop into a system prompt.
• Savings & efficiency — what Auto Router, prompt caching and response caching already saved this key, and — when the traffic warrants it — tips to turn on response caching, Switchboard fallbacks or the Auto Router, with an estimated dollar impact where one can be computed.
• On the horizon — a couple of ideas for where to take this usage next.

Enabling it

Souq is off by default and configured per key, from the Preferences tab on the Souq page. It requires tracing to already be enabled on that key — there's no captured content to learn from otherwise; the dialog explains this in place and links to the Tracing page when it's not on yet. Disabling tracing on a key automatically disables Souq for it too — the two settings can never drift out of sync. Nothing needs to change in your application code.

When you get a report

A report only makes sense when there's enough to learn from, so Souq needs at least 30 traced conversations in 7 days on a key before it writes one. Until a key reaches that bar, the Report tab shows a progress meter instead. Once it does, Souq picks the key up automatically on its next daily check and writes the first report; after that it writes a new one about once a week. A key that goes quiet simply doesn't get a new report that week — past reports remain available.

How it works

For each key that's due, Souq first computes the numbers from your gateway logs, then draws an evenly spread sample of up to 50 conversations from the past 7 days — spread across the whole window rather than just the most recent calls, so one busy afternoon doesn't crowd out the rest of the week. That sample is redacted (see below), tagged for work type, technique and friction, and then written up into the report. Because the numbers come from your logs and only the patterns come from the model, the charts are exact counts, not estimates.

Privacy & redaction

Every sampled conversation is run through Mizan's full PII redaction pass before it reaches the analysis step — regardless of whether Aman is turned on for that key. Reports describe patterns rather than quoting your prompts, and no trace content is stored in the report: it holds only the aggregate numbers and the written analysis. Reports are scoped strictly to the workspace that generated them (your personal workspace, or your organization) — Souq is never a cross-tenant marketplace and nothing it produces is ever visible outside your own workspace.

Who pays for it

The analysis itself is billed to Mizan, not to your wallet. Enabling Souq never draws down your balance or counts against a key's spend limit — only the traced calls you already made are billed, exactly as before.