Files
hermes-agent/web/src/pages/AnalyticsPage.tsx
Teknium af22421e87 feat(dashboard): page-scoped plugin slots for built-in pages (#15658)
* fix(terminal): three-layer defense against watch_patterns notification spam

Background processes that stack notify_on_complete=True with watch_patterns
can flood the user with duplicate, delayed notifications — matches deliver
asynchronously via the completion queue and continue arriving minutes after
the process has exited. The docstring warning against this (PR #12113) has
proven insufficient; agents still misuse the combination.

Three layered defenses, each sufficient on its own:

1. Mutual exclusion (terminal_tool.py): When both flags are set on a
   background process, drop watch_patterns with a warning. notify_on_complete
   wins because 'let me know when it's done' is the more useful signal and
   fires exactly once. Extracted as _resolve_notification_flag_conflict() so
   the rule is testable in isolation.

2. Suppress-after-exit (process_registry.py): _check_watch_patterns() now
   bails the moment session.exited is True. Post-exit chunks (buffered reads
   draining after the process is gone) no longer produce notifications. This
   is the fix flagged as future work in session 20260418_020302_79881c.

3. Global circuit breaker (process_registry.py): Per-session rate limits don't
   catch the sibling-flood case — N concurrent processes can each stay under
   8/10s and still collectively spam. New WATCH_GLOBAL_MAX_PER_WINDOW=15 cap
   trips a 30-second cooldown across ALL sessions, emits a single
   watch_overflow_tripped event, silently counts dropped events, and emits a
   watch_overflow_released summary when the cooldown ends.

Also updates the tool schema + docstring to document the new behavior.

Tests: 8 new tests covering all three fixes (suppress-after-exit x2,
mutual-exclusion resolver x4, global breaker trip/cooldown/release x2).
All 60 tests across test_watch_patterns.py, test_notify_on_complete.py,
test_terminal_tool.py pass.

Real-world trigger: self-inflicted in session 20260425_051924 — three
concurrent hermes-sweeper review subprocesses each set watch_patterns=
['failed validation', 'errored'] AND notify_on_complete=True, then iterated
over multiple items, producing enough matches per process to defeat the
per-session cap while staying under the global cap that didn't yet exist.

* fix(terminal): aggressive 1-per-15s watch_patterns rate limit + strike-3 promotion

Per Teknium's direction, the watch_patterns rate limit is now much more
aggressive and self-healing.

## New rule — per session

- HARD cap: 1 watch-match notification per 15 seconds per process.
- Any match arriving inside the cooldown window is dropped and counts as
  ONE strike for that window (many drops in the same window still = 1 strike).
- After 3 consecutive strike windows, watch_patterns is permanently disabled
  for the session and the session is auto-promoted to notify_on_complete
  semantics — exactly one notification when the process actually exits.
- A cooldown window that expires with zero drops resets the consecutive
  strike counter — healthy cadence is forgiven.

## Schema + docstring rewritten

The tool schema description now gives the model explicit guidance:
- notify_on_complete is 'the right choice for almost every long-running task'
- watch_patterns is for RARE one-shot signals on LONG-LIVED processes
- Do NOT use watch_patterns with loops/batch jobs — error patterns fire every
  iteration and will hit the strike limit fast
- Mutual exclusion is stated on both parameter descriptions
- 1/15s cooldown and 3-strike promotion are stated in the watch_patterns
  description so the model sees the contract every turn

## Removed

- WATCH_MAX_PER_WINDOW (8/10s) and WATCH_OVERLOAD_KILL_SECONDS (45) — the
  new 1/15s limit subsumes both; keeping them would double-count.
- _watch_window_hits / _watch_window_start / _watch_overload_since fields
  on ProcessSession. Replaced by _watch_last_emit_at / _watch_cooldown_until
  / _watch_strike_candidate / _watch_consecutive_strikes.

## Kept

- Global circuit breaker across all sessions (15/10s → 30s cooldown) as a
  secondary safety net for concurrent siblings. Still valuable when 20
  short-lived processes each fire once — none individually violates the
  per-session limit.
- Suppress-after-exit guard.
- Mutual exclusion resolver at the tool entry point.

## Tests

- 6 new tests in TestPerSessionRateLimit covering: first match delivers,
  second in cooldown suppressed, multi-drop = single strike, 3 strikes
  disables + promotes, clean window resets counter, suppressed count
  carried to next emit.
- Global circuit breaker tests rewritten to use fresh sessions instead of
  hacking removed per-window fields.
- 50/50 watch_patterns + notify_on_complete tests pass.
- 60/60 including test_terminal_tool.py pass.

* feat(dashboard): page-scoped plugin slots for built-in pages

Dashboard plugins can now inject components into specific built-in
pages (Sessions, Analytics, Logs, Cron, Skills, Config, Env, Docs,
Chat) without overriding the whole route.

Previously, plugins could only:
  1. Add new tabs (tab.path)
  2. Replace whole built-in pages (tab.override)
  3. Inject into global shell slots (header-*, footer-*, pre-main, ...)

None of those let a plugin add a banner, card, or widget to an
existing page. The new <page>:top / <page>:bottom slots close that
gap, reusing the existing registerSlot() API.

Changes
- web/src/plugins/slots.ts: 18 new KNOWN_SLOT_NAMES entries
  (sessions:top, sessions:bottom, analytics:top, ..., chat:bottom),
  grouped under "Shell-wide" vs "Page-scoped" in the docblock
- web/src/pages/*: each built-in page now renders
    <PluginSlot name="<page>:top" />
  as the first child of its outer wrapper and
    <PluginSlot name="<page>:bottom" />
  as the last child -- zero visual cost when no plugin registers
- plugins/example-dashboard: registers a demo banner into
  sessions:top via registerSlot(), with matching slots entry in
  the manifest -- so freshly-setup users can see what page-scoped
  slots look like without writing any plugin code
- website/docs: new "Page-scoped slots" table in the plugin
  authoring guide, with a worked example
- tests/hermes_cli/test_web_server.py: round-trip test for
  colon-bearing slot names (sessions:top, analytics:bottom, ...)

Validation
- npm run build: clean (tsc -b + vite build, 2761 modules)
- scripts/run_tests.sh tests/hermes_cli/test_web_server.py::TestDashboardPluginManifestExtensions: 5/5 pass
2026-04-25 06:55:35 -07:00

418 lines
15 KiB
TypeScript

import { useCallback, useEffect, useLayoutEffect, useState } from "react";
import {
BarChart3,
Brain,
Cpu,
Hash,
RefreshCw,
TrendingUp,
} from "lucide-react";
import { api } from "@/lib/api";
import type { AnalyticsResponse, AnalyticsDailyEntry, AnalyticsModelEntry, AnalyticsSkillEntry } from "@/lib/api";
import { timeAgo } from "@/lib/utils";
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import { usePageHeader } from "@/contexts/usePageHeader";
import { useI18n } from "@/i18n";
import { PluginSlot } from "@/plugins";
const PERIODS = [
{ label: "7d", days: 7 },
{ label: "30d", days: 30 },
{ label: "90d", days: 90 },
] as const;
const CHART_HEIGHT_PX = 160;
function formatTokens(n: number): string {
if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)}M`;
if (n >= 1_000) return `${(n / 1_000).toFixed(1)}K`;
return String(n);
}
function formatDate(day: string): string {
try {
const d = new Date(day + "T00:00:00");
return d.toLocaleDateString(undefined, { month: "short", day: "numeric" });
} catch {
return day;
}
}
function SummaryCard({
icon: Icon,
label,
value,
sub,
}: {
icon: React.ComponentType<{ className?: string }>;
label: string;
value: string;
sub?: string;
}) {
return (
<Card>
<CardHeader className="flex flex-row items-center justify-between pb-2">
<CardTitle className="text-sm font-medium">{label}</CardTitle>
<Icon className="h-4 w-4 text-muted-foreground" />
</CardHeader>
<CardContent>
<div className="text-2xl font-bold">{value}</div>
{sub && <p className="text-xs text-muted-foreground mt-1">{sub}</p>}
</CardContent>
</Card>
);
}
function TokenBarChart({ daily }: { daily: AnalyticsDailyEntry[] }) {
const { t } = useI18n();
if (daily.length === 0) return null;
const maxTokens = Math.max(...daily.map((d) => d.input_tokens + d.output_tokens), 1);
return (
<Card>
<CardHeader>
<div className="flex items-center gap-2">
<BarChart3 className="h-5 w-5 text-muted-foreground" />
<CardTitle className="text-base">{t.analytics.dailyTokenUsage}</CardTitle>
</div>
<div className="flex items-center gap-4 text-xs text-muted-foreground">
<div className="flex items-center gap-1.5">
<div className="h-2.5 w-2.5 bg-[#ffe6cb]" />
{t.analytics.input}
</div>
<div className="flex items-center gap-1.5">
<div className="h-2.5 w-2.5 bg-emerald-500" />
{t.analytics.output}
</div>
</div>
</CardHeader>
<CardContent>
<div className="flex items-end gap-[2px]" style={{ height: CHART_HEIGHT_PX }}>
{daily.map((d) => {
const total = d.input_tokens + d.output_tokens;
const inputH = Math.round((d.input_tokens / maxTokens) * CHART_HEIGHT_PX);
const outputH = Math.round((d.output_tokens / maxTokens) * CHART_HEIGHT_PX);
return (
<div
key={d.day}
className="flex-1 min-w-0 group relative flex flex-col justify-end"
style={{ height: CHART_HEIGHT_PX }}
>
{/* Tooltip */}
<div className="absolute bottom-full left-1/2 -translate-x-1/2 mb-2 hidden group-hover:block z-10 pointer-events-none">
<div className="bg-card border border-border px-2.5 py-1.5 text-[10px] text-foreground shadow-lg whitespace-nowrap">
<div className="font-medium">{formatDate(d.day)}</div>
<div>{t.analytics.input}: {formatTokens(d.input_tokens)}</div>
<div>{t.analytics.output}: {formatTokens(d.output_tokens)}</div>
<div>{t.analytics.total}: {formatTokens(total)}</div>
</div>
</div>
{/* Input bar */}
<div
className="w-full bg-[#ffe6cb]/70"
style={{ height: Math.max(inputH, total > 0 ? 1 : 0) }}
/>
{/* Output bar */}
<div
className="w-full bg-emerald-500/70"
style={{ height: Math.max(outputH, d.output_tokens > 0 ? 1 : 0) }}
/>
</div>
);
})}
</div>
{/* X-axis labels */}
<div className="flex justify-between mt-2 text-[10px] text-muted-foreground">
<span>{daily.length > 0 ? formatDate(daily[0].day) : ""}</span>
{daily.length > 2 && (
<span>{formatDate(daily[Math.floor(daily.length / 2)].day)}</span>
)}
<span>{daily.length > 1 ? formatDate(daily[daily.length - 1].day) : ""}</span>
</div>
</CardContent>
</Card>
);
}
function DailyTable({ daily }: { daily: AnalyticsDailyEntry[] }) {
const { t } = useI18n();
if (daily.length === 0) return null;
const sorted = [...daily].reverse();
return (
<Card>
<CardHeader>
<div className="flex items-center gap-2">
<TrendingUp className="h-5 w-5 text-muted-foreground" />
<CardTitle className="text-base">{t.analytics.dailyBreakdown}</CardTitle>
</div>
</CardHeader>
<CardContent>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-border text-muted-foreground text-xs">
<th className="text-left py-2 pr-4 font-medium">{t.analytics.date}</th>
<th className="text-right py-2 px-4 font-medium">{t.sessions.title}</th>
<th className="text-right py-2 px-4 font-medium">{t.analytics.input}</th>
<th className="text-right py-2 pl-4 font-medium">{t.analytics.output}</th>
</tr>
</thead>
<tbody>
{sorted.map((d) => {
return (
<tr key={d.day} className="border-b border-border/50 hover:bg-secondary/20 transition-colors">
<td className="py-2 pr-4 font-medium">{formatDate(d.day)}</td>
<td className="text-right py-2 px-4 text-muted-foreground">{d.sessions}</td>
<td className="text-right py-2 px-4">
<span className="text-[#ffe6cb]">{formatTokens(d.input_tokens)}</span>
</td>
<td className="text-right py-2 pl-4">
<span className="text-emerald-400">{formatTokens(d.output_tokens)}</span>
</td>
</tr>
);
})}
</tbody>
</table>
</div>
</CardContent>
</Card>
);
}
function ModelTable({ models }: { models: AnalyticsModelEntry[] }) {
const { t } = useI18n();
if (models.length === 0) return null;
const sorted = [...models].sort(
(a, b) => b.input_tokens + b.output_tokens - (a.input_tokens + a.output_tokens),
);
return (
<Card>
<CardHeader>
<div className="flex items-center gap-2">
<Cpu className="h-5 w-5 text-muted-foreground" />
<CardTitle className="text-base">{t.analytics.perModelBreakdown}</CardTitle>
</div>
</CardHeader>
<CardContent>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-border text-muted-foreground text-xs">
<th className="text-left py-2 pr-4 font-medium">{t.analytics.model}</th>
<th className="text-right py-2 px-4 font-medium">{t.sessions.title}</th>
<th className="text-right py-2 pl-4 font-medium">{t.analytics.tokens}</th>
</tr>
</thead>
<tbody>
{sorted.map((m) => (
<tr key={m.model} className="border-b border-border/50 hover:bg-secondary/20 transition-colors">
<td className="py-2 pr-4">
<span className="font-mono-ui text-xs">{m.model}</span>
</td>
<td className="text-right py-2 px-4 text-muted-foreground">{m.sessions}</td>
<td className="text-right py-2 pl-4">
<span className="text-[#ffe6cb]">{formatTokens(m.input_tokens)}</span>
{" / "}
<span className="text-emerald-400">{formatTokens(m.output_tokens)}</span>
</td>
</tr>
))}
</tbody>
</table>
</div>
</CardContent>
</Card>
);
}
function SkillTable({ skills }: { skills: AnalyticsSkillEntry[] }) {
const { t } = useI18n();
if (skills.length === 0) return null;
return (
<Card>
<CardHeader>
<div className="flex items-center gap-2">
<Brain className="h-5 w-5 text-muted-foreground" />
<CardTitle className="text-base">{t.analytics.topSkills}</CardTitle>
</div>
</CardHeader>
<CardContent>
<div className="overflow-x-auto">
<table className="w-full text-sm">
<thead>
<tr className="border-b border-border text-muted-foreground text-xs">
<th className="text-left py-2 pr-4 font-medium">{t.analytics.skill}</th>
<th className="text-right py-2 px-4 font-medium">{t.analytics.loads}</th>
<th className="text-right py-2 px-4 font-medium">{t.analytics.edits}</th>
<th className="text-right py-2 px-4 font-medium">{t.analytics.total}</th>
<th className="text-right py-2 pl-4 font-medium">{t.analytics.lastUsed}</th>
</tr>
</thead>
<tbody>
{skills.map((skill) => (
<tr key={skill.skill} className="border-b border-border/50 hover:bg-secondary/20 transition-colors">
<td className="py-2 pr-4">
<span className="font-mono-ui text-xs">{skill.skill}</span>
</td>
<td className="text-right py-2 px-4 text-muted-foreground">{skill.view_count}</td>
<td className="text-right py-2 px-4 text-muted-foreground">{skill.manage_count}</td>
<td className="text-right py-2 px-4">{skill.total_count}</td>
<td className="text-right py-2 pl-4 text-muted-foreground">
{skill.last_used_at ? timeAgo(skill.last_used_at) : "—"}
</td>
</tr>
))}
</tbody>
</table>
</div>
</CardContent>
</Card>
);
}
export default function AnalyticsPage() {
const [days, setDays] = useState(30);
const [data, setData] = useState<AnalyticsResponse | null>(null);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const { t } = useI18n();
const { setAfterTitle, setEnd } = usePageHeader();
const load = useCallback(() => {
setLoading(true);
setError(null);
api
.getAnalytics(days)
.then(setData)
.catch((err) => setError(String(err)))
.finally(() => setLoading(false));
}, [days]);
useLayoutEffect(() => {
const periodLabel =
PERIODS.find((p) => p.days === days)?.label ?? `${days}d`;
setAfterTitle(
<span className="flex items-center gap-2">
{loading && (
<div className="h-4 w-4 shrink-0 animate-spin rounded-full border-2 border-primary border-t-transparent" />
)}
<Badge variant="secondary" className="text-[10px]">
{periodLabel}
</Badge>
</span>,
);
setEnd(
<div className="flex w-full min-w-0 flex-wrap items-center justify-end gap-2 sm:gap-2">
<div className="flex flex-wrap items-center gap-1.5">
{PERIODS.map((p) => (
<Button
key={p.label}
type="button"
variant={days === p.days ? "default" : "outline"}
size="sm"
className="h-7 min-w-0 text-xs"
onClick={() => setDays(p.days)}
>
{p.label}
</Button>
))}
</div>
<Button
type="button"
variant="outline"
size="sm"
onClick={load}
disabled={loading}
className="h-7 text-xs"
>
<RefreshCw className="mr-1 h-3 w-3" />
{t.common.refresh}
</Button>
</div>,
);
return () => {
setAfterTitle(null);
setEnd(null);
};
}, [days, loading, load, setAfterTitle, setEnd, t.common.refresh]);
useEffect(() => {
load();
}, [load]);
return (
<div className="flex flex-col gap-6">
<PluginSlot name="analytics:top" />
{loading && !data && (
<div className="flex items-center justify-center py-24">
<div className="h-6 w-6 animate-spin rounded-full border-2 border-primary border-t-transparent" />
</div>
)}
{error && (
<Card>
<CardContent className="py-6">
<p className="text-sm text-destructive text-center">{error}</p>
</CardContent>
</Card>
)}
{data && (
<>
{/* Summary cards */}
<div className="grid gap-4 sm:grid-cols-2 lg:grid-cols-3">
<SummaryCard
icon={Hash}
label={t.analytics.totalTokens}
value={formatTokens(data.totals.total_input + data.totals.total_output)}
sub={t.analytics.inOut.replace("{input}", formatTokens(data.totals.total_input)).replace("{output}", formatTokens(data.totals.total_output))}
/>
<SummaryCard
icon={BarChart3}
label={t.analytics.totalSessions}
value={String(data.totals.total_sessions)}
sub={`~${(data.totals.total_sessions / days).toFixed(1)}${t.analytics.perDayAvg}`}
/>
<SummaryCard
icon={TrendingUp}
label={t.analytics.apiCalls}
value={String(data.totals.total_api_calls ?? data.daily.reduce((sum, d) => sum + d.sessions, 0))}
sub={t.analytics.acrossModels.replace("{count}", String(data.by_model.length))}
/>
</div>
{/* Bar chart */}
<TokenBarChart daily={data.daily} />
{/* Tables */}
<DailyTable daily={data.daily} />
<ModelTable models={data.by_model} />
<SkillTable skills={data.skills.top_skills} />
</>
)}
{data && data.daily.length === 0 && data.by_model.length === 0 && data.skills.top_skills.length === 0 && (
<Card>
<CardContent className="py-12">
<div className="flex flex-col items-center text-muted-foreground">
<BarChart3 className="h-8 w-8 mb-3 opacity-40" />
<p className="text-sm font-medium">{t.analytics.noUsageData}</p>
<p className="text-xs mt-1 text-muted-foreground/60">{t.analytics.startSession}</p>
</div>
</CardContent>
</Card>
)}
<PluginSlot name="analytics:bottom" />
</div>
);
}