Telemetry surface

Every metric, span, attribute and log field the SDK may emit. This is a contract, not a sample: backend/app/observability/surface.py holds the same set as code, and backend/tests/unit/test_telemetry_surface.py fails if the two disagree or if the test suite emits anything not listed here.

Rules that govern the whole surface

1. Metric labels never carry identities. No agent name, user sub, session id, action id or target in any label value. Every label has a closed value set, which is what bounds cardinality — verified at 1,000 agents × 100,000 sessions by backend/tests/load/cardinality.py.
2. Traces and logs are pseudonymous by default. Opaque UUIDs pass through; agent names and the act chain are keyed HMACs, absent entirely unless the disclosure opt-in is on.
3. No secrets, ever. No token, jti, credential, body, params_hash, binding, query string or stack trace.
4. Attributes are authz.*. Domain-first per OpenTelemetry's naming guidance. Metric names keep their published agentic_auth_* prefix; the asymmetry is deliberate.
5. Nothing is read back. Telemetry is write-only output. No authorization path branches on a span, a log field or inbound trace context.

Metrics — enforcement point

Served at GET /__agentic/metrics on each enforcement point.

Name	Type	Labels	Meaning
agentic_auth_pep_calls_total	counter	outcome	Proxied calls by what the PEP did
agentic_auth_pep_blocks_total	counter	cause	Blocked calls by cause (see below)
agentic_auth_pep_stage_duration_seconds	histogram	stage	Where per-hop time went
agentic_auth_pep_overhead_seconds	histogram	—	Guardrail-added time: total − upstream
agentic_auth_pep_up	gauge	—	1 while serving; liveness is absent() over this
agentic_auth_pep_info	gauge	mode, version	1; mode answers "is it enforcing"

outcome: allowed · blocked · observed_would_block

observed_would_block was forwarded but refused on the merits. It is never folded into allowed — that would make observe mode read as a working guardrail — nor into blocked, which would overstate what the PEP did.

stage: decide · exchange · mint · upstream

cause, and which ones page

Value	Meaning	Pages?
policy	A policy deny — the guardrail worked	no
pdp_unreachable	/v1/authorize unreachable or unusable	yes
exchange_unreachable	/v1/token/exchange unreachable or unusable	yes
mint_unreachable	/v1/token/transaction unreachable or unusable	yes
audit_unavailable	The decision had no committed record, so the call was refused	yes
unauthorized	Missing, malformed or refused bearer token	no — the caller's
unresolvable_call	No upstream mapping, or no target/data class	no — configuration
body_too_large	A privileged body over the bind cap	no — the caller's
unbindable	A privileged body that is not a JSON object	no — the caller's
other	Unclassified; present so an unmapped path is visible rather than uncounted	investigate

Only the four marked yes page. A critical that fires for the caller's mistakes teaches people to ignore criticals.

Metrics — telemetry self-observation

Name	Type	Labels	Meaning
agentic_auth_trace_context_rejected_total	counter	cause	Inbound trace context not used
agentic_auth_telemetry_dropped_total	counter	signal	Signals lost — telemetry loss reporting itself
agentic_auth_telemetry_identity_disclosure	gauge	—	1 when full identities are exported

cause: malformed · absent · baggage_dropped — absent is normal for an agent the integrator does not control; malformed rising is a caller bug or an attempt.

signal: span · log

Metrics — earlier phases

Every name and label from the Phase 3 and Phase 4 contracts survives unchanged, verified by backend/tests/unit/test_metric_compatibility.py: agentic_auth_decisions, _denials, _audit_write_failures, _chain_query_duration_seconds, _chain_queries, _chain_writes, _taint_blocks, _taint_marks, _transaction_mints, _enrollments, _revocations, _revoked_hop_refusals, _exchange_pop_failures, the _drift_* family, and the verifier's _admissions, _transaction_rejections, _identity_view_age_seconds.

Adding a label to any of these is a breaking change. It keeps sum by (...) working while silently breaking every exact-match query.

Spans

Name	Emitted by	Once per
authz.decide	control plane	POST /v1/authorize
authz.exchange	control plane	POST /v1/token/exchange
authz.transaction.mint	control plane	POST /v1/token/transaction
pep.forward	enforcement point	proxied request
verify.admit	verifier library	admit(), when a sink is supplied

All set record_exception=False.

Attributes

Key	Type	Values
authz.action_id	string	UUID — the handle that resolves the delegation chain
authz.session_id	string	UUID
authz.hop	int	≥ 1
authz.outcome	string	allow · deny
authz.deny_reason	string	A DenyReason; present iff outcome=deny
authz.recoverable	bool	Step-up-consentable
authz.enforced	bool	false in observe mode
authz.target_kind	string	tool · agent
authz.data_class	string	From the manifest
authz.privileged	bool
authz.taint	string	trusted_internal · untrusted_external
authz.policy_version	int
authz.guarantee_level	string	full · connector · network · consent_only — how strong the enforcement actually was
authz.result	string	Exchange / mint outcome
authz.pop_failure_cause	string	missing · invalid · wrong_agent · not_enrolled
authz.pep.mode	string	enforcing · observing
authz.pep.outcome	string	As pep_calls_total
authz.pep.block_cause	string	As pep_blocks_total
authz.agent_ref	string	32-hex pseudonym of the acting agent — default
authz.act_chain_ref	string	32-hex pseudonym of the act chain — default
authz.taint_source_ref	string	32-hex pseudonym of the taint source — default
authz.agent	string	Plaintext — only with disclosure on
authz.act_chain	string	Plaintext — only with disclosure on
authz.taint_source	string	Plaintext — only with disclosure on
error.type	string	Exception class name — never the message, never the trace
http.request.method, http.response.status_code		HTTP semantic conventions

authz.deny_reason present iff authz.outcome = deny, mirroring the ck_audit_decisions_reason_iff_deny database constraint, so a span and its audit row can never disagree about whether a reason exists.

A pseudonym and its plaintext counterpart are never emitted together.

Never an attribute, at any setting: the full target URI, caller, subject_jti, any jti, params_hash, binding, bodies, query strings.

Structured log fields

Field	Meaning
authz.component	control_plane · pep · verifier · cli
authz.session_id	UUID, when in a session
authz.action_id	UUID, when a decision exists
authz.hop	When applicable
error.type	Exception class name

trace_id and span_id are attached from the active span automatically.

Severity: an ordinary policy deny is INFO; a guardrail failure needing a human is WARNING or above. A deny is never an error.

Messages are static so records group — identifiers go in fields, never interpolated into the text.

Pseudonymisation

pseudonym(kind, value) = hex(HMAC-SHA256(telemetry_key, len(kind):kind:value))[:32]

Stable within a deployment, so telemetry can be grouped by agent. Not reversible without the key, which the telemetry backend never holds — a plain digest of an agent name would be recovered by brute force in seconds, since the name space is small and enumerable.

Rotating AGENTIC_AUTH_TELEMETRY_KEY re-pseudonymises everything and breaks historical grouping. It is deliberately a different setting from issuer_private_key, which must never rotate casually.

Multi-process deployments

Set PROMETHEUS_MULTIPROC_DIR to an empty writable directory and the control plane builds its /metrics registry with multiprocess.MultiProcessCollector. Traces and logs need no equivalent — each worker exports independently and the backend reassembles by trace id.