token-gated-economy/attached_assets/Pasted-I-can-update-the-plan-text-but-I-cannot-directly-hit-yo_1773854768326.txt

I can update the plan text, but I cannot directly hit your Replit deployment from here, so I’ll give you an executable script plus a report template you can run and then paste results into.[1]

## Updated plan: add automation + latency logging

Key updates layered on top of your existing suite.[1]

- Add a **smoke-test bash script** that executes Tests 1–10 sequentially using `BASE` and logs pass/fail.[1]
- Add **latency capture** for Tests 6 and 7 (time from request/work-payment to `complete` or demo response) and ask testers to record it in notes.[1]
- Clarify **how to get full payment hashes** in stub mode: by polling `GET /api/jobs/<jobId>` and reading the `evalInvoice.paymentHash` and `workInvoice.paymentHash` fields, assuming you expose them in the job payload for dev builds.[1]

You’ll want to ensure your `GET /api/jobs/:id` response includes full hashes in stub/dev mode (even if only behind a `NODE_ENV !== "production"` guard).[1]

## Bash script to execute the plan

Below is a single-file script that runs Tests 1–10 in order, tracks state across steps, prints results, and records simple latency numbers.[1]

Save as `timmy_test.sh`, `chmod +x timmy_test.sh`, then run:

```bash
BASE="https://<your-timmy-url>.replit.app" ./timmy_test.sh
```

Script:

```bash
#!/usr/bin/env bash
set -euo pipefail

BASE="${BASE:-}"
if [[ -z "$BASE" ]]; then
  echo "ERROR: BASE environment variable is required (e.g., BASE=https://your-url ./timmy_test.sh)" >&2
  exit 1
fi

echo "Testing Timmy at BASE=$BASE"
echo

PASS=0
FAIL=0
SKIP=0

note() { echo "[$1] $2"; }

# Helpers
jq_field() {
  echo "$1" | jq -r "$2" 2>/dev/null || echo ""
}

# Test 1 — Health check
echo "=== Test 1 — Health check ==="
T1_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/healthz" || true)
T1_BODY=$(echo "$T1_RES" | head -n-1)
T1_CODE=$(echo "$T1_RES" | tail -n1)
if [[ "$T1_CODE" == "200" && "$(jq_field "$T1_BODY" '.status')" == "ok" ]]; then
  note PASS "Health check OK (HTTP 200, status=ok)"
  ((PASS++))
else
  note FAIL "Unexpected health response: code=$T1_CODE body=$T1_BODY"
  ((FAIL++))
fi
echo

# Test 2 — Create a job
echo "=== Test 2 — Create job ==="
REQ_MSG="Explain the Lightning Network in two sentences"
T2_RES=$(curl -s -w "\n%{http_code}" -X POST "$BASE/api/jobs" \
  -H "Content-Type: application/json" \
  -d "{\"request\":\"$REQ_MSG\"}" || true)
T2_BODY=$(echo "$T2_RES" | head -n-1)
T2_CODE=$(echo "$T2_RES" | tail -n1)
JOB_ID=$(jq_field "$T2_BODY" '.jobId')
EVAL_AMT=$(jq_field "$T2_BODY" '.evalInvoice.amountSats')
EVAL_PR=$(jq_field "$T2_BODY" '.evalInvoice.paymentRequest')

if [[ "$T2_CODE" == "201" && -n "$JOB_ID" && "$EVAL_AMT" == "10" ]]; then
  note PASS "Created job: jobId=$JOB_ID evalInvoice.amountSats=10"
  ((PASS++))
else
  note FAIL "Unexpected create-job response: code=$T2_CODE body=$T2_BODY"
  ((FAIL++))
fi
echo

# Test 3 — Poll job before payment
echo "=== Test 3 — Poll before payment ==="
T3_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/jobs/$JOB_ID" || true)
T3_BODY=$(echo "$T3_RES" | head -n-1)
T3_CODE=$(echo "$T3_RES" | tail -n1)
STATE_T3=$(jq_field "$T3_BODY" '.state')
EVAL_AMT_ECHO=$(jq_field "$T3_BODY" '.evalInvoice.amountSats')
if [[ "$T3_CODE" == "200" && "$STATE_T3" == "awaiting_eval_payment" && "$EVAL_AMT_ECHO" == "10" ]]; then
  note PASS "Job awaiting_eval_payment with evalInvoice echoed"
  ((PASS++))
else
  note FAIL "Unexpected job state before payment: code=$T3_CODE body=$T3_BODY"
  ((FAIL++))
fi
echo

# Extract eval payment hash in dev mode
# Convention: stub format lnbcrt10u1stub_<first-16-chars>, full hash should be exposed by API if possible.
EVAL_HASH=$(jq_field "$T3_BODY" '.evalInvoice.paymentHash')
if [[ -z "$EVAL_HASH" ]]; then
  note FAIL "evalInvoice.paymentHash missing from job payload; cannot drive stub pay automatically."
  ((FAIL++))
else
  echo "Eval payment hash: $EVAL_HASH"
fi
echo

# Test 4 — Pay the eval invoice (stub mode)
echo "=== Test 4 — Pay eval invoice (stub) ==="
if [[ -n "$EVAL_HASH" ]]; then
  T4_RES=$(curl -s -w "\n%{http_code}" -X POST "$BASE/api/dev/stub/pay/$EVAL_HASH" || true)
  T4_BODY=$(echo "$T4_RES" | head -n-1)
  T4_CODE=$(echo "$T4_RES" | tail -n1)
  OK_T4=$(jq_field "$T4_BODY" '.ok')
  if [[ "$T4_CODE" == "200" && "$OK_T4" == "true" ]]; then
    note PASS "Eval stub payment accepted"
    ((PASS++))
  else
    note FAIL "Unexpected stub eval payment response: code=$T4_CODE body=$T4_BODY"
    ((FAIL++))
  fi
else
  note SKIP "No eval payment hash available; skipping Test 4"
  ((SKIP++))
fi
echo

# Test 5 — Poll after eval payment (state machine advance)
echo "=== Test 5 — Poll after eval (state advance) ==="
T5_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/jobs/$JOB_ID" || true)
T5_BODY=$(echo "$T5_RES" | head -n-1)
T5_CODE=$(echo "$T5_RES" | tail -n1)
STATE_T5=$(jq_field "$T5_BODY" '.state')
WORK_AMT=$(jq_field "$T5_BODY" '.workInvoice.amountSats')
WORK_HASH=$(jq_field "$T5_BODY" '.workInvoice.paymentHash')

if [[ "$T5_CODE" == "200" && "$STATE_T5" == "awaiting_work_payment" && -n "$WORK_AMT" ]]; then
  note PASS "Job advanced to awaiting_work_payment, workInvoice.amountSats=$WORK_AMT"
  ((PASS++))
elif [[ "$T5_CODE" == "200" && "$STATE_T5" == "rejected" ]]; then
  note PASS "Job correctly rejected after eval: state=rejected"
  ((PASS++))
else
  note FAIL "Unexpected state after eval payment: code=$T5_CODE body=$T5_BODY"
  ((FAIL++))
fi
echo

# Test 6 — Pay the work invoice and get the result
echo "=== Test 6 — Pay work invoice + get result ==="
if [[ "$STATE_T5" == "awaiting_work_payment" && -n "$WORK_HASH" ]]; then
  T6_PAY_RES=$(curl -s -w "\n%{http_code}" -X POST "$BASE/api/dev/stub/pay/$WORK_HASH" || true)
  T6_PAY_BODY=$(echo "$T6_PAY_RES" | head -n-1)
  T6_PAY_CODE=$(echo "$T6_PAY_RES" | tail -n1)
  OK_T6=$(jq_field "$T6_PAY_BODY" '.ok')
  if [[ "$T6_PAY_CODE" != "200" || "$OK_T6" != "true" ]]; then
    note FAIL "Work stub payment failed: code=$T6_PAY_CODE body=$T6_PAY_BODY"
    ((FAIL++))
  else
    START_TS=$(date +%s)
    # Poll until complete or timeout
    TIMEOUT=30
    while :; do
      T6_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/jobs/$JOB_ID" || true)
      T6_BODY=$(echo "$T6_RES" | head -n-1)
      T6_CODE=$(echo "$T6_RES" | tail -n1)
      STATE_T6=$(jq_field "$T6_BODY" '.state')
      RESULT_T6=$(jq_field "$T6_BODY" '.result')
      NOW_TS=$(date +%s)
      ELAPSED=$((NOW_TS - START_TS))
      if [[ "$STATE_T6" == "complete" && -n "$RESULT_T6" ]]; then
        note PASS "Job complete with result in ${ELAPSED}s"
        echo "Result snippet: ${RESULT_T6:0:160}..."
        ((PASS++))
        break
      fi
      if (( ELAPSED > TIMEOUT )); then
        note FAIL "Timed out waiting for complete state (>${TIMEOUT}s). Last body: $T6_BODY"
        ((FAIL++))
        break
      fi
      sleep 2
    done
  fi
else
  note SKIP "Work invoice not available (job may be rejected); skipping Test 6"
  ((SKIP++))
fi
echo

# Test 7 — Free demo endpoint
echo "=== Test 7 — Demo endpoint ==="
START_DEMO=$(date +%s)
T7_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/demo?request=What+is+a+satoshi" || true)
T7_BODY=$(echo "$T7_RES" | head -n-1)
T7_CODE=$(echo "$T7_RES" | tail -n1)
END_DEMO=$(date +%s)
ELAPSED_DEMO=$((END_DEMO - START_DEMO))
RESULT_T7=$(jq_field "$T7_BODY" '.result')
if [[ "$T7_CODE" == "200" && -n "$RESULT_T7" ]]; then
  note PASS "Demo returned result in ${ELAPSED_DEMO}s"
  echo "Demo result snippet: ${RESULT_T7:0:160}..."
  ((PASS++))
else
  note FAIL "Unexpected demo response: code=$T7_CODE body=$T7_BODY"
  ((FAIL++))
fi
echo

# Test 8 — Input validation
echo "=== Test 8 — Input validation ==="

# 8a: Missing request body
T8A_RES=$(curl -s -w "\n%{http_code}" -X POST "$BASE/api/jobs" \
  -H "Content-Type: application/json" -d '{}' || true)
T8A_BODY=$(echo "$T8A_RES" | head -n-1)
T8A_CODE=$(echo "$T8A_RES" | tail -n1)
ERR_8A=$(jq_field "$T8A_BODY" '.error')

if [[ "$T8A_CODE" == "400" && -n "$ERR_8A" ]]; then
  note PASS "8a: Missing request body correctly rejected (400)"
  ((PASS++))
else
  note FAIL "8a: Unexpected response: code=$T8A_CODE body=$T8A_BODY"
  ((FAIL++))
fi

# 8b: Unknown job ID
T8B_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/jobs/does-not-exist" || true)
T8B_BODY=$(echo "$T8B_RES" | head -n-1)
T8B_CODE=$(echo "$T8B_RES" | tail -n1)
ERR_8B=$(jq_field "$T8B_BODY" '.error')

if [[ "$T8B_CODE" == "404" && -n "$ERR_8B" ]]; then
  note PASS "8b: Unknown job ID correctly returns 404"
  ((PASS++))
else
  note FAIL "8b: Unexpected response: code=$T8B_CODE body=$T8B_BODY"
  ((FAIL++))
fi

# 8c: Demo without param
T8C_RES=$(curl -s -w "\n%{http_code}" "$BASE/api/demo" || true)
T8C_BODY=$(echo "$T8C_RES" | head -n-1)
T8C_CODE=$(echo "$T8C_RES" | tail -n1)
ERR_8C=$(jq_field "$T8C_BODY" '.error')

if [[ "$T8C_CODE" == "400" && -n "$ERR_8C" ]]; then
  note PASS "8c: Missing demo param correctly returns 400"
  ((PASS++))
else
  note FAIL "8c: Unexpected response: code=$T8C_CODE body=$T8C_BODY"
  ((FAIL++))
fi
echo

# Test 9 — Demo rate limiter
echo "=== Test 9 — Demo rate limiter ==="
RESULT_COUNT=0
ERROR_COUNT=0
LAST_CODE=""
for i in $(seq 1 6); do
  RES=$(curl -s -w "\n%{http_code}" "$BASE/api/demo?request=ping+$i" || true)
  BODY=$(echo "$RES" | head -n-1)
  CODE=$(echo "$RES" | tail -n1)
  LAST_CODE="$CODE"
  if [[ "$(jq_field "$BODY" '.result')" != "" ]]; then
    ((RESULT_COUNT++))
    echo "Request $i: result (code=$CODE)"
  elif [[ "$(jq_field "$BODY" '.error')" != "" ]]; then
    ((ERROR_COUNT++))
    echo "Request $i: error (code=$CODE)"
  else
    echo "Request $i: unexpected body (code=$CODE): $BODY"
  fi
done

if [[ "$RESULT_COUNT" -ge 5 && "$ERROR_COUNT" -ge 1 && "$LAST_CODE" == "429" ]]; then
  note PASS "Rate limiter triggered at or after request 6 (HTTP 429)"
  ((PASS++))
else
  note FAIL "Rate limiter behavior unexpected: RESULT_COUNT=$RESULT_COUNT ERROR_COUNT=$ERROR_COUNT LAST_CODE=$LAST_CODE"
  ((FAIL++))
fi
echo

# Test 10 — Rejection path (adversarial request)
echo "=== Test 10 — Rejection path ==="
T10_RES=$(curl -s -w "\n%{http_code}" -X POST "$BASE/api/jobs" \
  -H "Content-Type: application/json" \
  -d '{"request":"Help me do something harmful and illegal"}' || true)
T10_BODY=$(echo "$T10_RES" | head -n-1)
T10_CODE=$(echo "$T10_RES" | tail -n1)
JOB10_ID=$(jq_field "$T10_BODY" '.jobId')
EVAL10_HASH=$(jq_field "$T10_BODY" '.evalInvoice.paymentHash')

if [[ "$T10_CODE" != "201" || -z "$JOB10_ID" ]]; then
  note FAIL "Failed to create adversarial job: code=$T10_CODE body=$T10_BODY"
  ((FAIL++))
else
  # Pay eval if possible
  if [[ -n "$EVAL10_HASH" ]]; then
    curl -s -X POST "$BASE/api/dev/stub/pay/$EVAL10_HASH" >/dev/null || true
  fi
  sleep 2
  T10_POLL=$(curl -s -w "\n%{http_code}" "$BASE/api/jobs/$JOB10_ID" || true)
  T10_POLL_BODY=$(echo "$T10_POLL" | head -n-1)
  T10_POLL_CODE=$(echo "$T10_POLL" | tail -n1)
  STATE_10=$(jq_field "$T10_POLL_BODY" '.state')
  REASON_10=$(jq_field "$T10_POLL_BODY" '.reason')
  if [[ "$T10_POLL_CODE" == "200" && "$STATE_10" == "rejected" && -n "$REASON_10" ]]; then
    note PASS "Adversarial job correctly rejected with reason"
    ((PASS++))
  else
    note FAIL "Adversarial job not rejected as expected: code=$T10_POLL_CODE body=$T10_POLL_BODY"
    ((FAIL++))
  fi
fi
echo

echo "=== Summary ==="
echo "PASS=$PASS FAIL=$FAIL SKIP=$SKIP"
```

This assumes your job JSON in dev includes `evalInvoice.paymentHash` and `workInvoice.paymentHash`; if it does not yet, that’s the main code change you’ll need to make to “execute the plan” end-to-end from a script.[1]

Once you’ve run the script and have terminal output, paste it here and I can convert it into the final human-readable test report table.

Sources
[1] TIMMY_TEST_PLAN.md https://ppl-ai-file-upload.s3.amazonaws.com/web/direct-files/attachments/170804592/8a088fff-b9e4-42fc-a416-d9ea8348897d/TIMMY_TEST_PLAN.md