[ATLAS][Telemetry] Build local token and cost ingestion using Tokscale core patterns #229

Open
opened 2026-04-22 04:57:31 +00:00 by Rockachopa · 0 comments
Owner

Parent

Timmy_Foundation/hermes-agent#984

Source to Mine

  • junhoyeo/tokscale

What

Build a local-first telemetry ingestion layer that turns Hermes/agent session artifacts into normalized token and cost records without depending on a hosted dashboard.

Mine Tokscale core for:

  • unified usage schema
  • parser-per-client adapter model
  • cached pricing service with overrides and fallbacks

Tasks

  • Define a normalized schema for runs / sessions / messages with token and cost fields
  • Implement artifact parsers for Hermes output plus at least one second client/tool format
  • Add local pricing cache with override file and fallback behavior
  • Write rollups by model / provider / repo / agent / task
  • Document adapter contract for adding new parsers
  • Add fixture-backed ingestion tests

Acceptance Criteria

  • A normalized local schema exists for input/output/cache tokens and cost metadata
  • At least 2 ingestion adapters work end-to-end (Hermes + one other format)
  • Pricing can resolve from a local cache/override path without a mandatory hosted dependency
  • Rollups are emitted to a local report, DB, or artifact that can be consumed by the operator stack
  • Adapter interface is documented clearly enough for adding a third parser without guesswork
  • Fixture-backed tests prove ingestion and rollup correctness
## Parent Timmy_Foundation/hermes-agent#984 ## Source to Mine - `junhoyeo/tokscale` ## What Build a local-first telemetry ingestion layer that turns Hermes/agent session artifacts into normalized token and cost records without depending on a hosted dashboard. Mine Tokscale core for: - unified usage schema - parser-per-client adapter model - cached pricing service with overrides and fallbacks ## Tasks - [ ] Define a normalized schema for runs / sessions / messages with token and cost fields - [ ] Implement artifact parsers for Hermes output plus at least one second client/tool format - [ ] Add local pricing cache with override file and fallback behavior - [ ] Write rollups by model / provider / repo / agent / task - [ ] Document adapter contract for adding new parsers - [ ] Add fixture-backed ingestion tests ## Acceptance Criteria - [ ] A normalized local schema exists for input/output/cache tokens and cost metadata - [ ] At least 2 ingestion adapters work end-to-end (Hermes + one other format) - [ ] Pricing can resolve from a local cache/override path without a mandatory hosted dependency - [ ] Rollups are emitted to a local report, DB, or artifact that can be consumed by the operator stack - [ ] Adapter interface is documented clearly enough for adding a third parser without guesswork - [ ] Fixture-backed tests prove ingestion and rollup correctness
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/compounding-intelligence#229