stackchain/timmy-talking-turd
2
0 Fork 0
Code Issues 21 Pull Requests 7 Actions 4 Packages Projects Releases 41 Wiki Activity

Execute security, abuse, load, and medical-image leakage tests #40

New Issue
Closed
opened 2026-08-19 01:59:02 +00:00 by timmy · 1 comment
timmy commented 2026-08-19 01:59:02 +00:00
Owner
Copy link
Copy source

Parent epic: #7

Outcome

Attack the release candidate before users do.

Acceptance criteria

  • Covers oversized/polyglot inputs, replay, queue flood, malformed model output, log inspection, and deletion.
  • Load test records p50/p95/error/abstention without image bodies.
  • All P0 findings block RC approval.

Dependencies

  • #39 — Define production topology, secrets, backup, and rollback runbook
  • #16 — Harden image ingress: magic bytes, re-encode, limits, metadata stripping, and rate control

Required evidence

  • Test or executable receipt attached to the issue/PR
  • Privacy/safety boundary checked where applicable
  • Clean checkout and relevant full suite pass

Delivery

Open a focused branch and PR with Closes #ISSUE. Do not include real medical images in issue comments or Git history.

Parent epic: #7 ## Outcome Attack the release candidate before users do. ## Acceptance criteria - [ ] Covers oversized/polyglot inputs, replay, queue flood, malformed model output, log inspection, and deletion. - [ ] Load test records p50/p95/error/abstention without image bodies. - [ ] All P0 findings block RC approval. ## Dependencies - #39 — Define production topology, secrets, backup, and rollback runbook - #16 — Harden image ingress: magic bytes, re-encode, limits, metadata stripping, and rate control ## Required evidence - Test or executable receipt attached to the issue/PR - Privacy/safety boundary checked where applicable - Clean checkout and relevant full suite pass ## Delivery Open a focused branch and PR with `Closes #ISSUE`. Do not include real medical images in issue comments or Git history.
timmy added this to the M4 — Production Readiness & Release milestone 2026-08-19 01:59:02 +00:00
timmy added the type/securitypriority/P0area/privacyarea/release labels 2026-08-19 01:59:02 +00:00
rockachopa commented 2026-10-30 11:13:41 +00:00
Member
Copy link
Copy source

[EVIDENCE] agent=timmy

PR: #91 (branch timmy/40-security-rc-evidence at f0a9a73, base 65a0258). Not merged; RC approval stays with the release job.

  • scripts/security_attack_suite.mjs drives the real server with synthetic hostile bytes while a counting stub worker records every call that reaches the model boundary. Nine cases, JSON report, exit 1 when blocked. docs/RC-SECURITY-EVIDENCE.md owns the rule.
  • Run on this branch: VERDICT: BLOCKED. Two P0 findings are real here — an HTML polyglot dressed as image/jpeg and 64 random bytes both reach the worker (counted, not inferred) and only fail afterwards on schema (observed 503 with providerCalls: 1). That fix is #16 / PR #82. Queue bounding is #17 / PR #64, held as P1 (flood-concurrency). Consent, oversized-body, malformed-answer refusal, replay, and the log scan pass.
  • Load phase: 8 requests, p50 24 ms, p95 31 ms, 0 errors, 8 abstentions, no image body in the report.
  • Tests: 244/244 unit (npm test), 8/8 for tests/security-attack.acceptance.mjs, which re-derives every case status from the recorded observation instead of trusting the runner's summary. Demo re-recorded on this tree: exit 0, 456 s, decodes.

This comment is the receipt, not a green light: the boundary is not claimed safe while the two P0 cases fail.

[EVIDENCE] agent=timmy PR: https://forge.alexanderwhitestone.com/git/stackchain/timmy-talking-turd/pulls/91 (branch `timmy/40-security-rc-evidence` at f0a9a73, base `65a0258`). Not merged; RC approval stays with the release job. - `scripts/security_attack_suite.mjs` drives the real server with synthetic hostile bytes while a counting stub worker records every call that reaches the model boundary. Nine cases, JSON report, exit 1 when blocked. `docs/RC-SECURITY-EVIDENCE.md` owns the rule. - Run on this branch: `VERDICT: BLOCKED`. Two P0 findings are real here — an HTML polyglot dressed as `image/jpeg` and 64 random bytes both reach the worker (counted, not inferred) and only fail afterwards on schema (observed `503` with `providerCalls: 1`). That fix is #16 / PR #82. Queue bounding is #17 / PR #64, held as P1 (`flood-concurrency`). Consent, oversized-body, malformed-answer refusal, replay, and the log scan pass. - Load phase: 8 requests, p50 24 ms, p95 31 ms, 0 errors, 8 abstentions, no image body in the report. - Tests: 244/244 unit (`npm test`), 8/8 for `tests/security-attack.acceptance.mjs`, which re-derives every case status from the recorded observation instead of trusting the runner's summary. Demo re-recorded on this tree: exit 0, 456 s, decodes. This comment is the receipt, not a green light: the boundary is not claimed safe while the two P0 cases fail.
timmy closed this issue 2026-10-30 12:02:47 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
area/backend
Application API and storage
 area/data
Dataset and provenance
 area/frontend
PWA and interaction design
 area/model
Vision model and inference
 area/privacy
Privacy and consent
 area/release
Release engineering and distribution
 area/safety
Clinical safety boundary
 blocked
Blocked on an explicit dependency or decision
 human-gate
Requires explicit human approval
 priority/P0
Release-critical
 priority/P1
High priority
 priority/P2
Planned
 state:review
type/docs
Documentation and evidence
 type/epic
Roadmap epic
 type/feature
Product feature
 type/ops
Infrastructure and operations
 type/research
Research or evaluation
 type/security
Security hardening
 type/test
Verification and quality gate
No labels area/privacy area/release priority/P0 type/security
Milestone
No items
No Milestone M4 — Production Readiness & Release
Projects
Clear projects
No projects
Assignees
Clear assignees
timmy (Timmy · the Wizard Mind) zephyrus-sparkborn (Zephyrus Sparkborn)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: stackchain/timmy-talking-turd#40