Repository navigation
spike(framework): prove whether native OTel adds value beyond the measurement journal #297
Description
Question
Does native OpenTelemetry data add information or interoperability that the shipped AIDD measurement journal cannot provide?
Current baseline
AIDD already reads the files AI tools write, enriches them with task and step attribution, and stores a published JSON Lines record locally. aidd-telemetry:01-cost is its first reader. Measurement requires project opt-in, a person may refuse it locally, and the record contract excludes prompts, code, and diffs.
A hosted destination may be chosen by the person with separate consent. No destination, transport, or OpenTelemetry route is mandatory.
Evidence required
- Compare, on the same Claude Code session, native OTel data with the local-file reading and AIDD journal.
- State for each source: available fields, missing fields, attribution to task and step, correlation reliability, privacy boundary, and risk of duplicate accounting.
- Do not generalize the Claude result to another tool without equivalent evidence.
- Name an actual destination or consumer that cannot use the published record contract, if interoperability is claimed as the value.
Decision outcomes
- No material gain: retain the local journal and do not add an OTel receiver or exporter.
- Better source data: propose only the bounded native-source integration, including precedence and deduplication with local readings.
- Destination interoperability is necessary: propose only the output mapping, consent, delivery proof, and retained totals. JSON Lines remains the local source of truth.
Out of scope
- Migrating local storage to OTel.
- Selecting or embedding a hosted destination.
- Collecting any field outside the published record contract.
- Declaring an external total a billing amount without its provenance.
Acceptance criteria
- One same-session comparison documents coverage, correlation, and duplicate handling for native OTel and the local journal.
- The outcome states whether native OTel changes a user-relevant decision, with evidence.
- Any external-output proposal names its consumer and proves consent, reception, and total preservation.
- The result explicitly retains or rejects each candidate integration; no transport is adopted by implication.
Activity
[-]epic: LLM observability — OpenTelemetry tracing via hooks + execution logs + optimization skill[/-][+]Plugin LLM observability — OpenTelemetry tracing via hooks + execution logs + optimization skill[/+][-]Plugin LLM observability — OpenTelemetry tracing via hooks + execution logs + optimization skill[/-][+]feat: N2 — skill/agent execution observability (OpenTelemetry via hooks)[/+][-]feat: N2 — skill/agent execution observability (OpenTelemetry via hooks)[/-][+]feat: usage metrics — what tokens, gestures and skills are actually used[/+][-]feat: usage metrics — what tokens, gestures and skills are actually used[/-][+]feat(framework): usage metrics — what tokens, gestures and skills are used[/+]blafourcade commentedon Jul 10, 2026
Blocked by #421. The "à brainstormer avant de spécifier" section of this issue moved there: the observed unit, the consumer, and the exploiting skills. This issue gets rewritten against that outcome, or closed if the need does not survive the questions.
docs(framework): amend #297 — a hosted destination is allowed, and ne…
blafourcade commentedon Aug 31, 2026
Decision of record amended
The Standard clause above — "OpenTelemetry (spans, metrics, logs). Sink is an OTel collector. No SaaS." — is superseded by a written record:
aidd_docs/memory/internal/decisions/measurement-may-reach-a-hosted-destination.md, decided 2026-08-31 by @blafourcade.Measured against what ships, all three of its statements are contradicted: the sink is a directory of JSON Lines, the OTLP receiver produced no record on the machine that built it and has been deleted, and a hosted destination is being built.
The Privacy clause is reaffirmed unchanged, and exceeded — a person can now refuse in their own environment, and that refusal wins over a repository's committed switch. A hosted destination changes where measurement may go, never what it may contain.
The record also states the boundary the amendment rests on: the framework exposes, the destination analyses. It is not ours to fix which destination — the person names it, what travels is the published record contract, anyone can run their own, and nothing about measuring or reading requires an account.
blafourcade commentedon Sep 8, 2026
Clarification de cible : collecte native OTel et export externe
Revue du backlog du 8 septembre 2026. Cette note complète la décision du 31 août, sans rétablir OpenTelemetry comme transport obligatoire ni annoncer une migration décidée.
État et décision existante
La décision du 31 août retient la lecture des fichiers locaux des outils, enrichie par le journal AIDD, et un stockage local JSON Lines. Elle autorise une destination externe choisie par la personne, avec un consentement distinct pour l'envoi. OTel est une forme possible pour une destination, pas le format interne imposé au framework.
Le socle local existe. L'envoi externe reste un chantier : #662, « upload out of band, never at the session's expense », porte la transmission en arrière-plan, l'authentification, les reprises après erreur et l'idempotence. Cette issue doit être relue contre la décision du 31 août, puisque sa formulation hérite d'une ancienne voie d'export supprimée.
Deux questions distinctes
Valeur attendue et limites
Convertir les données JSON existantes en OTel ne crée aucune information supplémentaire et ne répare aucune attribution manquante. JSON Lines décrit ici un stockage ; OTel fournit un modèle de télémétrie et un écosystème, et OTLP un protocole de transport. Ils peuvent coexister.
Si une seule destination accepte déjà le contrat JSON publié, la valeur immédiate d'une conversion est limitée. Si plusieurs plateformes d'observabilité doivent recevoir ces données, l'interopérabilité devient une justification concrète.
La valeur propre d'AIDD est la relation entre consommation, tâche, issue et étapes du travail. Une source native OTel peut contribuer à cette attribution, mais son rapprochement avec le contexte AIDD doit être démontré. Si fichiers et export natif couvrent les mêmes consommations, une règle de priorité ou de dédoublonnage doit éviter de les compter deux fois. Les montants éventuellement exposés doivent garder leur provenance, sans être assimilés à une facture.
Orientation proposée, à valider par des preuves
La proposition de fermer #297 comme entièrement réalisée est retirée : l'existence du socle local ne prouve pas que toute la cible est satisfaite. L'issue reste ouverte pendant cette clarification. Cette note ne décide pas encore d'implémenter un récepteur OTLP ni de migrer le stockage.
Sources : Claude Code Monitoring, consulté le 8 septembre 2026 ; résilience du Collector OpenTelemetry.
[-]feat(framework): usage metrics — what tokens, gestures and skills are used[/-][+]spike(framework): prove whether native OTel adds value beyond the measurement journal[/+]blafourcade commentedon Sep 11, 2026
#297 — same-session comparison, Claude Code native OTel vs the AIDD journal
One
claude -psession, 2026-09-09, invoking a skill that writes into a task folder. Native export enabled withCLAUDE_CODE_ENABLE_TELEMETRY=1,OTEL_*_EXPORTER=otlp,http/jsonto a throwaway localreceiver. The AIDD hooks ran unchanged in the same session, so both sources describe the
same work.
Token counters: identical, to the unit
claude_code.token.usage)kind: request)Native OTel adds no token information. Ingesting both without a precedence rule would
double every counter.
What each source has that the other does not
cost_usd, 0.2578 for the sessionamount unknownskill_activatedfires, butskill.name=custom_skill,skill.source=projectSettingsstep_startnamesprobetask_declared,file_written, with the pathproject_iduser.email,user.id,user.account_uuid, sent by defaultaidd telemetry identitypromptattribute exists, value<REDACTED>by defaultCorrelation needs no heuristic: OTel's
session.idis the journal'svendor_id, alreadyrecorded as
vendor_field: "session.id".Reading
duplicate-accounting risk.
amount. That amount arrives without a price table, and its provenance is the vendor's own.
as
custom_skill; no task path reaches it at all.unless opted in, but the account email ships by default, where AIDD requires an opt-in.
The skill name is a flag away, and that changes the reading
Second run, same repo,
OTEL_LOG_TOOL_DETAILS=1added:skill_activatedcame back withskill.name=probe, its real name,invocation_trigger=claude-proactive. The promptstayed
<REDACTED>— that is a separate flag (OTEL_LOG_USER_PROMPTS).The documentation states the same rule for plugin skills:
skill.nameis the placeholder"custom_skill"for user-defined and third-party plugin skills unlessOTEL_LOG_TOOL_DETAILS=1, which also releasesplugin.nameandmarketplace.name. Measuredhere on a
projectSettingsskill; a plugin skill was not run, so that half rests on thedocumentation, not on a capture.
So native OTel can name the step on Claude Code. What it still never carries is the task
folder, the file a step wrote, the project identity and the backlog link — the journal's own
lines. The remaining value of the plugin on this tool is the link between a spend and the
unit of work it delivered, not the step name.
Not generalisable to the other four tools: this is one Claude Code session.
Other tools, from their documentation, not measured here
[otel]section inconfig.toml; openai/codex#12913 reports thatcodex execemits no metricsHow to reproduce
Any OTLP/HTTP-JSON endpoint that appends each payload it receives is enough. Point Claude Code at it with the variables above, run one session that invokes a project skill writing into
aidd_docs/tasks/, then compare theclaude_code.token.usagepoints withaidd telemetry reportfor the same period.blafourcade commentedon Oct 15, 2026
Context after #993: release 1 counts tokens from Claude Code's own session files, with no OTel. Every axis adds up to the same total, and CI checks the counts against a fake API. Native OTel would only be worth adding for something these files do not hold.