fix: escalate the symptom phrasing people actually use, and never clear a reported symptom (#21)
Quality gates / quality (pull_request) Successful in 2m38s

Deterministic escalation only fired on a narrow set of literal phrases, so real reports
like "there is blood when I wipe" or "I have tarry stools" reached the chat provider
without the fixed medical guidance. Expanding the suite found 18 of 18 checked clinical
phrasings missing.

- domain: cover bleeding reported by wiping/toilet paper/anatomy/mixed-in phrasing,
  black-tarry-dark-red stool, reported temperature and high temperature, abdominal-scoped
  severe pain in every natural word order, and past-tense/slang inability to pass gas.
  Near-miss wording that must stay quiet (paper cut, blood pressure, black beans, dark red
  wine, a normal temperature, knee or back pain, passed gas, no gas in the car) is pinned.
- domain: a symptom reported in any truthy shape (1, "yes", "true") now counts as reported.
  Reading only the literal boolean `true` silently cleared a symptom, which is the one
  direction this boundary may never fail in.
- hermes-agent-service: the gateway ledger sanitizer used the same strict check, so a
  browser-supplied ledger entry could have its symptom cleared before the escalation check.
- tests/escalation-regression.test.js: per-flag positive and negative language coverage,
  model output can neither set nor clear a symptom/urgency/discomfort/note, and the
  escalation copy is byte-identical for every flag and under a missing, throwing, or
  malicious provider. Registered in the default unit gate.
- scripts/record_release_demo.mjs: the recording now types the new bleeding and tarry
  phrasing, and seeds a fictional ledger whose note says "stool was tarry yesterday", then
  asserts in the browser that exactly one new Timmy bubble carries the app's own copy and
  that the turn never reaches Hermes. Three new captions are required release gates.

Boundary unchanged: no new inference, no diagnosis, no image claim. Escalation only ever
gets broader on user-reported symptoms.
This commit is contained in:
timmy committed 2026-10-15 08:15:00 +00:00
1 parent a033cb3ae9
commit b4be7049ea
9 files changed
+430 -9

No files matched your search

+3
View File
@@ -37,6 +37,9 @@ DEMO_REQUIRED_CAPTIONS = (
"Delete all local data clears the ledger in this browser; there is no server copy of it to keep",
"Hermes Agent connected — free-text, contextual, and server-side",
"Urgent language is intercepted deterministically before Hermes",
"Bleeding reported as wiping still escalates, in the app’s own fixed words",
"Tarry and black stool wording escalates too: one red flag, one unchanging message",
"A tarry note already saved in the journal escalates the same way, before any provider call",
"An out-of-range answer fails closed: nothing is applied and no provider detail reaches you",
"A filled-in field that is out of range still fails closed, even when the other field came back blank",
"Reachable is not loaded: no AI control, no consent, and no upload until the pinned model is resident",
+38
View File
@@ -385,6 +385,44 @@ await tap('#send-chat', 450);
await page.getByText(/Pause and get medical help/i).waitFor();
if (hermesCalls !== 1) throw new Error('Urgent chat must be intercepted before Hermes');
await caption('Urgent language is intercepted deterministically before Hermes', 1800);
// Issue #21 — the escalation has to fire for the phrasing people actually use, not only for the one
// word the previous scene typed. Each line below is asserted in the browser: exactly one new Timmy
// bubble, carrying the app's own fixed copy, with hermesCalls unmoved.
const escalationBubbles = () => page.locator('.bubble.timmy', { hasText: 'Pause and get medical help' }).count();
const expectEscalation = async (text, label) => {
const before = await escalationBubbles();
await page.locator('#chat-message').fill(text);
await tap('#send-chat', 450);
const bubble = page.locator('.bubble.timmy', { hasText: 'Pause and get medical help' }).last();
await bubble.waitFor();
if (await escalationBubbles() !== before + 1) throw new Error(`${label}: expected exactly one escalation bubble`);
if (!/^Pause and get medical help\. These reported symptoms can need prompt medical care\./.test(await bubble.innerText())) {
throw new Error(`${label}: the escalation copy must be the app's own fixed string`);
}
if (hermesCalls !== 1) throw new Error(`${label}: an escalated turn must not reach Hermes`);
};
await expectEscalation('There is blood when I wipe', 'bleeding reported as wiping');
await caption('Bleeding reported as wiping still escalates, in the app’s own fixed words', 2600);
await expectEscalation('I have tarry stools', 'tarry stool');
await caption('Tarry and black stool wording escalates too: one red flag, one unchanging message', 2600);
// The same detector runs over the confirmed ledger, so a symptom already written down as a note
// escalates even when the question itself is innocuous. The seeded entry is fictional and stays in
// this browser, exactly like the summary scene above.
await page.evaluate(() => localStorage.clear());
await seedSyntheticLedger([{ ...sampleDay('demo-tarry-note', 1, 4), note: 'stool was tarry yesterday' }]);
await tap('[data-view="timmy"]', 550);
const ledgerEscalations = await escalationBubbles();
await page.locator('#chat-message').fill('What do my recent logs show?');
await tap('#send-chat', 450);
await page.locator('.bubble.timmy', { hasText: 'Pause and get medical help' }).last().waitFor();
if (await escalationBubbles() !== ledgerEscalations + 1) throw new Error('a tarry note in the journal must escalate the next answer');
if (hermesCalls !== 1) throw new Error('a ledger red flag must be intercepted before Hermes');
await caption('A tarry note already saved in the journal escalates the same way, before any provider call', 2800);
await page.evaluate(() => localStorage.clear());
await page.reload({ waitUntil: 'networkidle' });
await tap('[data-view="timmy"]', 500);
await sleep(400);
await page.evaluate(() => {
document.querySelector('#demo-caption')?.remove();