Engineering
Your own internal API becomes a tool.
#ml-internalprivate channel3 tool calls
An observability stack with no preset is an ordinary custom connection: a host, a credential the proxy injects, a writes policy. Asked what one document's extraction cost, it calls the API, totals the spans in the sandbox, and reports the figure with the trace it came from — correcting its own first grouping in the open.
Mei12:33 PM
@attest_tag what did the extraction of this document cost, end to end?
attest_tagApp12:34 PM
- http_request · GET langfuse.internal/api/public/traces?tags=doc_id:b5be… · 1 trace
- http_request · GET …/observations?traceId=b381… · 17 observations
- run_js · sum cost over 5 generations
Total cost of this extraction: $0.0668 (about 6.7 cents).
- Trace
ai_assist.v2, 12:31:45 UTC, doc type bill_of_lading, status 200 - 5 generation calls, all on the same model, all started within 50 ms of each other — parallel, not sequential batches: four table batches and one key-value pass
- Input 74,097 tokens, output 3,184; cost per call in the table below
My first split — “four batches, then a KV call” — was wrong; re-pulling all 17 observations showed the five running in parallel. The table is from the re-pull.
google/gemini-flash12.3k in560 out$0.0063Configure
How the turn went
Three fields make a connection; the credential joins at the edge.
- 01
A custom connection is three fields
The host, a credential — basic auth here — and a writes policy. Notes on the connection tell the model how the API is shaped; a domain entry lets it read the vendor’s docs without any credential.
- 02
The credential joins at the edge
The model emits a URL and a method. The proxy matches the host, injects the credential one hop before the request leaves, and scrubs the credential values back out of the response before the model or the thread sees it.
- 03
Totals in the sandbox, and a correction in the open
Seventeen observations went into
run_js; five generations and a sum came out. When the first grouping was wrong, the re-pull and the correction went in the same thread.
What it needs
The setup, in the console.
What it will not do
Enforced outside the model.
A host not on the channel's list is refused before the request leaves — the proxy decides, not the model.
Anything other than GET, HEAD or OPTIONS waits for a Confirm unless the connection's policy says auto.
Secrets that come home in a response — keys in a config endpoint, say — are redacted before the model or the thread sees them.
Try it
Say it in a channel it is in.
The sentence as it was typed. It works in any channel the bot has been invited to that carries the connection above — no command syntax, no form, and follow-ups in the thread need no mention.
@attest_tag what did the extraction of this document cost, end to end?
http_requestrun_js
More in Engineering
Ask the repository a question. Then hand it the fix.
- What changed since Monday, with the commits linked.#payments · what changed in the payments client since Monday?
- “Fix it and raise a PR” — and a draft PR arrives.#payments · fix the flaky retry-budget test so it stops depending on wall-clock time, and raise a PR
- The decision, who made it, and a link to where.#payments · what did we decide about retries this week, in three bullets? and who first suggested capping total retry time instead of the count?
Bring the channel this would live in.
Sign up, connect the service above in one channel, and type this sentence yourself — or run the same binary on your own infrastructure.