On-call & monitoring
Triage the alert in the thread it landed in.
#monitoring-prodprivate channel3 tool calls
An alert in a private monitoring channel, a document id in its body, and a person asking what happened. It pulls the production logs for that id through a read-only service account and answers with the failing call, the timeline and where the fault sits — in the thread, where the on-call engineer already is.
Sam7:48 AM
@attest_tag check what happened here — the doc id is in the alert above
attest_tagApp7:49 AM
- read_thread · the alert and its 3 replies
- gcp_query_logs · doc_id=0f12… · last 24h · 41 lines
- gcp_query_logs · call_extractor_api · 5 attempts
The extraction failed because the extractor API returned HTTP 500 on every attempt, about 90 seconds apart, until the workflow’s activity ran out of retries.
- Workflow
DocExtractorWorkflow→extract_document_activity, doc typeothers__… call_extractor_apiloggedextractor_api_server_error, status 500, on attempts 1–5 (19:53:41 → 19:59:06 UTC)- Response body: the generic gateway error page — the fault is on the service, not the document.
The call goes to the tier-2 extractor service on Cloud Run. Worth checking that service’s own logs for the same window — say the word and I’ll pull them.
google/gemini-flash6.1k in412 out$0.0034Configure
How the turn went
Two queries, one thread, and what it did not find said plainly.
- 01
The mention lands in the alert's thread
It rebuilds the thread from Slack: the alert, the replies, and the document id in the alert’s body. Nobody has to paste anything, and a follow-up in the same thread needs no mention.
- 02
It queries the logs, twice
The first query is the id across the last day; the second narrows to the one client call the first showed failing. Both go through the proxy with the service account attached at the edge, and both are rows in the audit trail with their status and timing.
- 03
It answers in the thread
The failing call, the timeline, the response body — and what it did not find, said plainly. Ask for the downstream service’s logs and it is the same tool again.
What it needs
The setup, in the console.
What it will not do
Enforced outside the model.
Nothing about the project can be changed through the GCP presets. A query is a read, and the presets carry no write at all — whatever the prompt, or a log line, says.
Tool output — log lines and alert bodies included — is quoted material, never an instruction. A line that reads like a command is reported, not obeyed.
It reads with its own token, so before touching any conversation that is not the current one and not public, it checks the asker's own membership. It cannot be used as a periscope into a channel you were not invited to.
Try it
Say it in a channel it is in.
The sentence as it was typed. It works in any channel the bot has been invited to that carries the connection above — no command syntax, no form, and follow-ups in the thread need no mention.
@attest_tag check what happened here — the doc id is in the alert above
gcp_query_logsread_thread
More in On-call & monitoring
The alert lands. So does the answer.
- The state of prod, before anyone pages.#monitoring-prod · what's erroring in prod right now?
- A week of alerts, one fix.#monitoring-prod · look at all the alerts for this customer in the last 7 days and suggest what fix should be applied
- A report out of a notification channel.#stripe-alerts · find every message that says a customer unsubscribed and make a report of who unsubscribed and when
Bring the channel this would live in.
Sign up, connect the service above in one channel, and type this sentence yourself — or run the same binary on your own infrastructure.