Preserve healthy live sections when one upstream times out #1429

Closed
opened 2026-08-26 11:20:42 +00:00 by timmy · 0 comments
Owner

Problem

A timeout in one live-snapshot upstream currently fails the entire cold snapshot. For example, stalled notifications discard already-successful My Work context and activity, returning HTTP 503 and making mobile work unavailable.

User value

Keep healthy live sections actionable during a partial Gitea outage. A notifications-only stall should still return fresh context/activity with an honest degraded Updates state, while retry/backoff applies only to the failed section.

Acceptance criteria

  • Context, events, and notifications have section-isolated deadlines within the existing overall deadline.
  • A stalled notifications request returns HTTP 200 when context/events succeed, with notifications unavailable/degraded.
  • A stalled identity request degrades context/events but preserves independently completed notifications.
  • Only timed-out sections receive failure count and retry cooldown.
  • Shared refresh/cancellation behavior and one-time identity lookup remain intact.
  • Behavioral tests prove cold partial success, selective backoff, cancellation, and the full suite passes.

Non-duplication evidence

Current origin/main (065cc46) still wraps _build_live_snapshot in one aggregate timeout. Historical work isolated ordinary section exceptions and shared-refresh cancellation, but no issue/recent commit isolates section deadlines. Open PR #1282 concerns bucket aggregation latency and is unrelated.

## Problem A timeout in one live-snapshot upstream currently fails the entire cold snapshot. For example, stalled notifications discard already-successful My Work context and activity, returning HTTP 503 and making mobile work unavailable. ## User value Keep healthy live sections actionable during a partial Gitea outage. A notifications-only stall should still return fresh context/activity with an honest degraded Updates state, while retry/backoff applies only to the failed section. ## Acceptance criteria - Context, events, and notifications have section-isolated deadlines within the existing overall deadline. - A stalled notifications request returns HTTP 200 when context/events succeed, with notifications unavailable/degraded. - A stalled identity request degrades context/events but preserves independently completed notifications. - Only timed-out sections receive failure count and retry cooldown. - Shared refresh/cancellation behavior and one-time identity lookup remain intact. - Behavioral tests prove cold partial success, selective backoff, cancellation, and the full suite passes. ## Non-duplication evidence Current `origin/main` (`065cc46`) still wraps `_build_live_snapshot` in one aggregate timeout. Historical work isolated ordinary section exceptions and shared-refresh cancellation, but no issue/recent commit isolates section deadlines. Open PR #1282 concerns bucket aggregation latency and is unrelated.
timmy self-assigned this 2026-08-26 11:20:42 +00:00
timmy closed this issue 2026-08-26 11:55:24 +00:00
Sign in to join this conversation.
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: stackchain/stackchain-dashboard#1429
No description provided.