Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a4beca2d9e | ||
|
|
5f3f0176f1 | ||
|
|
b3c76dcd69 | ||
|
|
540584a417 | ||
|
|
4f6ba0302d | ||
|
|
d7d6170c3e |
No files matched your search
@@ -30,8 +30,14 @@ idempotency keys, so retrying after a timeout, reload, process restart, or hando
|
||||
another worker replays a confirmed result instead of posting duplicate content. Results
|
||||
are coordinated through a bounded SQLite ledger. Set `STACKCHAIN_STATE_DIR` to a
|
||||
persistent, writable service directory (or set `STACKCHAIN_IDEMPOTENCY_DB` to an explicit
|
||||
SQLite path); the local default is `.stackchain-state/idempotency.sqlite3`. Direct API callers should preserve the `Idempotency-Key`
|
||||
header with the unchanged route and payload until a `201` response is confirmed. Closing an assigned issue, native Comment,
|
||||
SQLite path); the local default is `.stackchain-state/idempotency.sqlite3`. Ledger reads and
|
||||
writes run outside the request event loop, and lock admission is bounded to 100 ms by
|
||||
default. Tune it with `STACKCHAIN_IDEMPOTENCY_LOCK_TIMEOUT_SECONDS`; keep the value below
|
||||
route deadlines. Reservation contention returns retryable HTTP 503 with `Retry-After: 1`.
|
||||
If contention occurs after the upstream mutation, the dashboard fails closed with
|
||||
`Retry-After: 5` and asks the caller to verify the result before retrying. Direct API callers
|
||||
should preserve the `Idempotency-Key` header with the unchanged route and payload until a
|
||||
`201` response is confirmed. Closing an assigned issue, native Comment,
|
||||
Approve, and Request changes reviews, and assigned-PR merge require repository
|
||||
write permission. Native Comment, Approve, and Request changes reviews support
|
||||
head-scoped draft comments anchored to changed lines; the dashboard validates each
|
||||
@@ -68,6 +74,16 @@ API process is running and does not contact Gitea. GET `/readyz` is the
|
||||
readiness check: it validates the configured Gitea credentials and returns
|
||||
HTTP 503 with an error when Gitea is unavailable or authentication fails.
|
||||
|
||||
The application-lifetime Gitea transport bounds aggregate upstream traffic and
|
||||
coalesces identical concurrent GETs without caching completed responses. By
|
||||
default, at most eight requests run at once, with one slot reserved for authored
|
||||
mutations so polling and detail-read bursts cannot starve comments, reviews, or
|
||||
other writes. Requests that cannot enter within 250 ms fail as retryable HTTP
|
||||
503 responses. Tune these limits with `GITEA_MAX_CONCURRENCY` (minimum `2`) and
|
||||
`GITEA_ADMISSION_TIMEOUT_SECONDS` (minimum `0.001`); keep the admission timeout
|
||||
below the route deadlines. Streaming diff reads share the same read capacity,
|
||||
while POST, PATCH, PUT, and DELETE requests are never coalesced.
|
||||
|
||||
## Offline mobile shell
|
||||
|
||||
At phone widths, a persistent bottom task dock keeps **Work**, **Find**, **New**,
|
||||
@@ -95,15 +111,21 @@ data** deletes the snapshot, and opting out deletes it automatically.
|
||||
|
||||
API responses and mutations are never cached by the service worker. New issue captures,
|
||||
issue comments, pull-request comments, and unread-update replies use bounded local
|
||||
outboxes when connectivity or a retryable server failure prevents delivery. Drafts
|
||||
shows queued and needs-attention messages with explicit send/discard controls; reconnect
|
||||
flushes messages sequentially with their original idempotency keys. Tabs coordinate each
|
||||
queue through the browser lock manager (with an expiring local lease fallback), so an
|
||||
installed PWA and a browser tab cannot submit the same operation concurrently. Queue
|
||||
changes are broadcast to other tabs and refresh Drafts without polling; an abandoned
|
||||
lease can be reclaimed after expiry. State-sensitive actions such as reviews, merges,
|
||||
closures, labels, milestones, and assignments are never queued. Service-worker upgrades
|
||||
are atomic and remove only older
|
||||
outboxes when connectivity or a retryable server failure prevents delivery. Issue
|
||||
captures are also mirrored into IndexedDB and registered with Background Sync, so a
|
||||
supporting installed browser can deliver them after every dashboard client has closed.
|
||||
The worker verifies the current Gitea login, shares an atomic delivery claim with the
|
||||
foreground path, and preserves the original idempotency key. Browsers without
|
||||
IndexedDB or Background Sync keep the foreground reconnect behavior. Drafts shows
|
||||
queued and needs-attention messages with explicit send/discard controls; reopening the
|
||||
dashboard reconciles worker completions and permanent failures into the visible outbox.
|
||||
Foreground delivery flushes messages sequentially with their original idempotency keys.
|
||||
Tabs coordinate each queue through the browser lock manager (with an expiring local
|
||||
lease fallback), so an installed PWA and a browser tab cannot submit the same operation
|
||||
concurrently. Queue changes are broadcast to other tabs and refresh Drafts without
|
||||
polling; an abandoned lease can be reclaimed after expiry. State-sensitive actions such
|
||||
as reviews, merges, closures, labels, milestones, and assignments are never queued.
|
||||
Service-worker upgrades are atomic and remove only older
|
||||
`stackchain-dashboard-*` caches, preserving unrelated caches on the same origin.
|
||||
|
||||
Run the test suite with:
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
function createIndexedDbTransaction(indexedDB, dbName = 'stackchain-background-outbox-v1') {
|
||||
let databasePromise;
|
||||
function database() {
|
||||
if (!databasePromise) databasePromise = new Promise((resolve, reject) => {
|
||||
const request = indexedDB.open(dbName, 1);
|
||||
request.onupgradeneeded = () => {
|
||||
if (!request.result.objectStoreNames.contains('issues')) {
|
||||
request.result.createObjectStore('issues', { keyPath: 'id' });
|
||||
}
|
||||
};
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
return databasePromise;
|
||||
}
|
||||
const requested = request => new Promise((resolve, reject) => {
|
||||
request.onsuccess = () => resolve(request.result);
|
||||
request.onerror = () => reject(request.error);
|
||||
});
|
||||
return async work => {
|
||||
const db = await database();
|
||||
return new Promise((resolve, reject) => {
|
||||
const transaction = db.transaction('issues', 'readwrite');
|
||||
const objectStore = transaction.objectStore('issues');
|
||||
let result;
|
||||
let failed = false;
|
||||
transaction.oncomplete = () => failed ? undefined : resolve(result);
|
||||
transaction.onerror = () => reject(transaction.error);
|
||||
transaction.onabort = () => reject(transaction.error || new Error('Issue outbox transaction aborted'));
|
||||
Promise.resolve(work({
|
||||
getAll: () => requested(objectStore.getAll()),
|
||||
put: value => requested(objectStore.put(value)),
|
||||
delete: id => requested(objectStore.delete(id)),
|
||||
})).then(value => { result = value; }).catch(error => {
|
||||
failed = true;
|
||||
try { transaction.abort(); } catch (_abortError) { reject(error); }
|
||||
reject(error);
|
||||
});
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
function createIssueSyncStore({ transaction, indexedDB = globalThis.indexedDB, now = () => Date.now(), claimMs = 30000 } = {}) {
|
||||
const transact = transaction || createIndexedDbTransaction(indexedDB);
|
||||
|
||||
async function reconcile(items) {
|
||||
return transact(async records => {
|
||||
const existing = await records.getAll();
|
||||
const incoming = new Map(items.map(item => [item.id, { ...item }]));
|
||||
for (const current of existing) {
|
||||
const replacement = incoming.get(current.id);
|
||||
if (!replacement) {
|
||||
if (current.status !== 'sending' || Number(current.claimUntil) <= Number(now())) {
|
||||
await records.delete(current.id);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if ((current.status === 'sending' && Number(current.claimUntil) > Number(now())) ||
|
||||
(current.status === 'attention' && replacement.status === 'attention') ||
|
||||
current.status === 'sent') {
|
||||
incoming.set(current.id, current);
|
||||
}
|
||||
}
|
||||
for (const item of incoming.values()) await records.put(item);
|
||||
});
|
||||
}
|
||||
|
||||
async function claimNext(ownerLogin) {
|
||||
return transact(async records => {
|
||||
const timestamp = Number(now());
|
||||
const items = await records.getAll();
|
||||
const item = items.find(candidate => candidate.ownerLogin === ownerLogin &&
|
||||
(candidate.status === 'queued' || candidate.status === 'sending') &&
|
||||
(candidate.status !== 'sending' || Number(candidate.claimUntil) <= timestamp));
|
||||
if (!item) return null;
|
||||
const claimed = { ...item, status: 'sending', claimUntil: timestamp + claimMs };
|
||||
await records.put(claimed);
|
||||
return claimed;
|
||||
});
|
||||
}
|
||||
|
||||
async function claim(id, ownerLogin) {
|
||||
return transact(async records => {
|
||||
const timestamp = Number(now());
|
||||
const item = (await records.getAll()).find(candidate => candidate.id === id);
|
||||
if (!item || item.ownerLogin !== ownerLogin ||
|
||||
!['queued', 'sending'].includes(item.status) ||
|
||||
(item.status === 'sending' && Number(item.claimUntil) > timestamp)) return null;
|
||||
const claimed = { ...item, status: 'sending', claimUntil: timestamp + claimMs };
|
||||
await records.put(claimed);
|
||||
return claimed;
|
||||
});
|
||||
}
|
||||
|
||||
async function upsert(item) {
|
||||
return transact(async records => {
|
||||
const current = (await records.getAll()).find(candidate => candidate.id === item.id);
|
||||
if (current && ((current.status === 'sending' && Number(current.claimUntil) > Number(now())) ||
|
||||
(current.status === 'attention' && item.status === 'attention') ||
|
||||
current.status === 'sent')) return current;
|
||||
await records.put({ ...item });
|
||||
return item;
|
||||
});
|
||||
}
|
||||
|
||||
async function update(id, transform) {
|
||||
return transact(async records => {
|
||||
const item = (await records.getAll()).find(candidate => candidate.id === id);
|
||||
if (item) await records.put(transform(item));
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
reconcile,
|
||||
upsert,
|
||||
claim,
|
||||
claimNext,
|
||||
complete: id => update(id, item => ({ ...item, status: 'sent', claimUntil: 0 })),
|
||||
release: id => update(id, item => ({ ...item, status: 'queued', claimUntil: 0 })),
|
||||
fail: (id, error) => update(id, item => ({ ...item, status: 'attention', claimUntil: 0, error })),
|
||||
snapshot: () => transact(records => records.getAll()),
|
||||
countBlocked: ownerLogin => transact(async records =>
|
||||
(await records.getAll()).filter(item => item.status !== 'sent' && item.ownerLogin !== ownerLogin).length),
|
||||
};
|
||||
}
|
||||
|
||||
function createBackgroundIssueSync({ store, fetchJson, base = '' }) {
|
||||
function issueRequest(item) {
|
||||
const repository = String(item.repository || '').split('/').map(encodeURIComponent).join('/');
|
||||
return {
|
||||
url: base + 'api/v1/repos/' + repository + '/issues',
|
||||
options: {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Accept: 'application/json',
|
||||
'Content-Type': 'application/json',
|
||||
'Idempotency-Key': item.operationId,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
title: item.title,
|
||||
body: item.body,
|
||||
label_ids: item.labelIds,
|
||||
...(item.milestoneId ? { milestone_id: item.milestoneId } : {}),
|
||||
...(item.dueDate ? { due_date: item.dueDate + 'T23:59:59Z' } : {}),
|
||||
}),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function deliver(item) {
|
||||
const request = issueRequest(item);
|
||||
try {
|
||||
const issue = await fetchJson(request.url, request.options);
|
||||
await store.complete(item.id);
|
||||
return { issue };
|
||||
} catch (error) {
|
||||
const status = Number(error?.status || 0);
|
||||
if (status >= 400 && status < 500) {
|
||||
await store.fail(item.id, String(error?.message || 'Issue needs attention').slice(0, 240));
|
||||
return { attention: true, error };
|
||||
}
|
||||
await store.release(item.id);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async function send(item, currentLogin) {
|
||||
if (!currentLogin || item.ownerLogin !== currentLogin) return { blocked: true };
|
||||
await store.upsert(item);
|
||||
const claimed = await store.claim(item.id, currentLogin);
|
||||
if (!claimed) return { busy: true };
|
||||
return deliver(claimed);
|
||||
}
|
||||
|
||||
async function flush() {
|
||||
const identity = await fetchJson(base + 'api/v1/background-identity', {
|
||||
headers: { Accept: 'application/json' }, cache: 'no-store',
|
||||
});
|
||||
const login = String(identity?.login || '').trim();
|
||||
const confirmed = [];
|
||||
let attention = 0;
|
||||
if (!login) return { confirmed, blocked: 0, attention };
|
||||
while (true) {
|
||||
const item = await store.claimNext(login);
|
||||
if (!item) break;
|
||||
const result = await deliver(item);
|
||||
if (result.issue) confirmed.push(result.issue);
|
||||
if (result.attention) attention += 1;
|
||||
}
|
||||
const blocked = store.countBlocked ? await store.countBlocked(login) : 0;
|
||||
return { confirmed, blocked, attention };
|
||||
}
|
||||
|
||||
return {
|
||||
flush, send,
|
||||
reconcile: items => store.reconcile(items),
|
||||
snapshot: () => store.snapshot(),
|
||||
};
|
||||
}
|
||||
|
||||
createBackgroundIssueSync.createIssueSyncStore = createIssueSyncStore;
|
||||
if (typeof module !== 'undefined' && module.exports) module.exports = createBackgroundIssueSync;
|
||||
if (typeof globalThis !== 'undefined') {
|
||||
globalThis.createBackgroundIssueSync = createBackgroundIssueSync;
|
||||
globalThis.createIssueSyncStore = createIssueSyncStore;
|
||||
}
|
||||
@@ -741,6 +741,7 @@ textarea { resize: vertical; min-height: 120px; }
|
||||
<script src="static/widgets.js"></script>
|
||||
<script src="static/drafts.js"></script>
|
||||
<script src="static/outbox-coordinator.js"></script>
|
||||
<script src="static/background-issue-sync.js"></script>
|
||||
<script src="static/issue-outbox.js"></script>
|
||||
<script src="static/authored-outbox.js"></script>
|
||||
<script src="static/offline-work.js"></script>
|
||||
@@ -894,11 +895,28 @@ textarea { resize: vertical; min-height: 120px; }
|
||||
loadMilestones: item => issueController.loadMilestones(item),
|
||||
});
|
||||
const issueCapture = createIssueCapture({ fetchJson: fetchReviewJson, storage: localStorage });
|
||||
let backgroundIssueSync = null;
|
||||
if ('indexedDB' in window) {
|
||||
const backgroundIssueStore = createIssueSyncStore();
|
||||
backgroundIssueSync = createBackgroundIssueSync({
|
||||
store: backgroundIssueStore, fetchJson: fetchReviewJson,
|
||||
});
|
||||
backgroundIssueSync.requestSync = async () => {
|
||||
if (!('serviceWorker' in navigator)) return;
|
||||
const registration = await navigator.serviceWorker.ready;
|
||||
if (registration.sync) await registration.sync.register('stackchain-issue-outbox-v1');
|
||||
};
|
||||
}
|
||||
const outboxCoordinator = createOutboxCoordinator({ storage: localStorage });
|
||||
const issueOutbox = createIssueOutbox({
|
||||
storage: localStorage, fetchJson: fetchReviewJson, coordinator: outboxCoordinator,
|
||||
backgroundSync: backgroundIssueSync,
|
||||
getOwnerLogin: () => confirmedOwnerLogin,
|
||||
});
|
||||
if (backgroundIssueSync) {
|
||||
backgroundIssueSync.snapshot().then(records => issueOutbox.reconcileBackground(records))
|
||||
.catch(() => { /* The foreground localStorage outbox remains available. */ });
|
||||
}
|
||||
const authoredOutbox = createAuthoredOutbox({
|
||||
storage: localStorage, fetchJson: fetchReviewJson, coordinator: outboxCoordinator,
|
||||
getOwnerLogin: () => confirmedOwnerLogin,
|
||||
|
||||
+45
-14
@@ -1,4 +1,4 @@
|
||||
function createIssueOutbox({ storage, fetchJson, coordinator, getOwnerLogin = () => '', createOperationId, now = () => Date.now(), maxItems = 20 }) {
|
||||
function createIssueOutbox({ storage, fetchJson, coordinator, backgroundSync, getOwnerLogin = () => '', createOperationId, now = () => Date.now(), maxItems = 20 }) {
|
||||
const storageKey = 'stackchain.issue-outbox.v1';
|
||||
const operationId = createOperationId || (() =>
|
||||
globalThis.crypto?.randomUUID?.() || String(Date.now()) + '-' + Math.random().toString(16).slice(2)
|
||||
@@ -16,6 +16,11 @@ function createIssueOutbox({ storage, fetchJson, coordinator, getOwnerLogin = ()
|
||||
function write(items) {
|
||||
storage?.setItem(storageKey, JSON.stringify({ version: 2, items }));
|
||||
coordinator?.notify('issue');
|
||||
if (backgroundSync?.reconcile) {
|
||||
Promise.resolve(backgroundSync.reconcile(items))
|
||||
.then(() => items.length ? backgroundSync.requestSync?.() : undefined)
|
||||
.catch(() => { /* Foreground reconnect remains the compatibility fallback. */ });
|
||||
}
|
||||
}
|
||||
|
||||
function enqueue(draft) {
|
||||
@@ -74,18 +79,30 @@ function createIssueOutbox({ storage, fetchJson, coordinator, getOwnerLogin = ()
|
||||
const repository = item.repository.split('/').map(encodeURIComponent).join('/');
|
||||
const request = (async () => {
|
||||
try {
|
||||
const issue = await fetchJson('api/v1/repos/' + repository + '/issues', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Accept: 'application/json', 'Content-Type': 'application/json',
|
||||
'Idempotency-Key': item.operationId,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
title: item.title, body: item.body, label_ids: item.labelIds,
|
||||
...(item.milestoneId ? { milestone_id: item.milestoneId } : {}),
|
||||
...(item.dueDate ? { due_date: item.dueDate + 'T23:59:59Z' } : {}),
|
||||
}),
|
||||
});
|
||||
let issue;
|
||||
if (backgroundSync?.send) {
|
||||
const delivery = await backgroundSync.send(item, currentLogin);
|
||||
if (delivery.attention) {
|
||||
const error = delivery.error || new Error('Issue needs attention');
|
||||
error.status = Number(error.status || 422);
|
||||
throw error;
|
||||
}
|
||||
issue = delivery.issue;
|
||||
} else {
|
||||
issue = await fetchJson('api/v1/repos/' + repository + '/issues', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
Accept: 'application/json', 'Content-Type': 'application/json',
|
||||
'Idempotency-Key': item.operationId,
|
||||
},
|
||||
body: JSON.stringify({
|
||||
title: item.title, body: item.body, label_ids: item.labelIds,
|
||||
...(item.milestoneId ? { milestone_id: item.milestoneId } : {}),
|
||||
...(item.dueDate ? { due_date: item.dueDate + 'T23:59:59Z' } : {}),
|
||||
}),
|
||||
});
|
||||
}
|
||||
if (!issue) return { blocked: true };
|
||||
discard(item.id);
|
||||
return { issue };
|
||||
} catch (error) {
|
||||
@@ -139,7 +156,21 @@ function createIssueOutbox({ storage, fetchJson, coordinator, getOwnerLogin = ()
|
||||
return retryItem(id, currentLogin);
|
||||
}
|
||||
|
||||
return { enqueue, update, discard, flush, retry, list: () => read().map(item => ({ ...item })) };
|
||||
function reconcileBackground(records) {
|
||||
const statuses = new Map((records || []).map(item => [item.id, item]));
|
||||
const items = read().flatMap(item => {
|
||||
const background = statuses.get(item.id);
|
||||
if (background?.status === 'sent') return [];
|
||||
if (background?.status === 'attention') return [{
|
||||
...item, status: 'attention', error: String(background.error || 'Issue needs attention').slice(0, 240),
|
||||
}];
|
||||
return [item];
|
||||
});
|
||||
write(items);
|
||||
return items;
|
||||
}
|
||||
|
||||
return { enqueue, update, discard, flush, retry, reconcileBackground, list: () => read().map(item => ({ ...item })) };
|
||||
}
|
||||
|
||||
if (typeof module !== 'undefined' && module.exports) module.exports = createIssueOutbox;
|
||||
@@ -1,6 +1,7 @@
|
||||
const CACHE = 'stackchain-dashboard-shell-v11';
|
||||
const OUTAGE_STATUSES = new Set([500, 502, 503, 504]);
|
||||
const BASE = new URL('./', self.location.href).pathname;
|
||||
importScripts(BASE + 'static/background-issue-sync.js');
|
||||
const CACHE = 'stackchain-dashboard-shell-v12';
|
||||
const OUTAGE_STATUSES = new Set([500, 502, 503, 504]);
|
||||
const SHELL = [
|
||||
BASE,
|
||||
BASE + 'manifest.webmanifest',
|
||||
@@ -25,8 +26,24 @@ const SHELL = [
|
||||
BASE + 'static/work-route.js',
|
||||
BASE + 'static/context-poller.js',
|
||||
BASE + 'static/mobile-task-dock.js',
|
||||
BASE + 'static/background-issue-sync.js',
|
||||
];
|
||||
|
||||
async function fetchJson(url, options) {
|
||||
const response = await fetch(new URL(url, self.location.origin), options);
|
||||
const payload = await response.json().catch(() => ({}));
|
||||
if (!response.ok) {
|
||||
const error = new Error(payload.error || payload.detail || 'Background issue delivery failed.');
|
||||
error.status = response.status;
|
||||
throw error;
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
const issueSync = self.__issueSync || createBackgroundIssueSync({
|
||||
store: createIssueSyncStore(), fetchJson, base: BASE,
|
||||
});
|
||||
|
||||
self.addEventListener('install', event => {
|
||||
event.waitUntil(caches.open(CACHE).then(cache => cache.addAll(SHELL)).then(() => self.skipWaiting()));
|
||||
});
|
||||
@@ -38,6 +55,10 @@ self.addEventListener('activate', event => {
|
||||
)).then(() => self.clients.claim()));
|
||||
});
|
||||
|
||||
self.addEventListener('sync', event => {
|
||||
if (event.tag === 'stackchain-issue-outbox-v1') event.waitUntil(issueSync.flush());
|
||||
});
|
||||
|
||||
self.addEventListener('fetch', event => {
|
||||
const request = event.request;
|
||||
if (request.method !== 'GET' || request.url.includes('/api/')) return;
|
||||
|
||||
+119
-4
@@ -2,6 +2,7 @@ import asyncio
|
||||
import os
|
||||
import re
|
||||
import shlex
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
@@ -12,7 +13,116 @@ GITEA_TOKEN = os.getenv("GITEA_TOKEN", "")
|
||||
REVIEW_DIFF_MAX_BYTES = 64 * 1024
|
||||
REVIEW_DIFF_MAX_LINES = 400
|
||||
AVAILABLE_ISSUE_PAGE_CONCURRENCY = 3
|
||||
_client: httpx.AsyncClient | None = None
|
||||
_client: "GiteaTransport | None" = None
|
||||
|
||||
|
||||
class GiteaOverloadedError(RuntimeError):
|
||||
"""Raised when bounded transport admission expires before capacity is available."""
|
||||
|
||||
|
||||
class GiteaTransport:
|
||||
"""Application-lifetime HTTP transport with single-flight concurrent reads."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
max_concurrency: int = 8,
|
||||
admission_timeout: float = 0.25,
|
||||
**kwargs,
|
||||
) -> None:
|
||||
max_concurrency = max(2, max_concurrency)
|
||||
admission_timeout = max(0.001, admission_timeout)
|
||||
self._http = httpx.AsyncClient(base_url=GITEA_URL, timeout=10, **kwargs)
|
||||
self.max_concurrency = max_concurrency
|
||||
self.admission_timeout = admission_timeout
|
||||
self._request_slots = asyncio.Semaphore(max(2, max_concurrency))
|
||||
self._read_slots = asyncio.Semaphore(max(1, max_concurrency - 1))
|
||||
self._reads: dict[tuple, asyncio.Task[httpx.Response]] = {}
|
||||
|
||||
@property
|
||||
def is_closed(self) -> bool:
|
||||
return self._http.is_closed
|
||||
|
||||
def _read_key(self, url: str, kwargs: dict) -> tuple:
|
||||
params = tuple(httpx.QueryParams(kwargs.get("params", {})).multi_items())
|
||||
headers = tuple(sorted(httpx.Headers(kwargs.get("headers", {})).multi_items()))
|
||||
return url, params, headers
|
||||
|
||||
async def _acquire(self, semaphore: asyncio.Semaphore) -> None:
|
||||
try:
|
||||
await asyncio.wait_for(semaphore.acquire(), timeout=self.admission_timeout)
|
||||
except TimeoutError as exc:
|
||||
raise GiteaOverloadedError("Gitea transport is saturated") from exc
|
||||
|
||||
async def _perform_get(self, url: str, kwargs: dict) -> httpx.Response:
|
||||
await self._acquire(self._read_slots)
|
||||
try:
|
||||
await self._acquire(self._request_slots)
|
||||
try:
|
||||
return await self._http.get(url, **kwargs)
|
||||
finally:
|
||||
self._request_slots.release()
|
||||
finally:
|
||||
self._read_slots.release()
|
||||
|
||||
async def get(self, url: str, **kwargs) -> httpx.Response:
|
||||
key = self._read_key(url, kwargs)
|
||||
task = self._reads.get(key)
|
||||
if task is None:
|
||||
task = asyncio.create_task(self._perform_get(url, kwargs))
|
||||
self._reads[key] = task
|
||||
task.add_done_callback(
|
||||
lambda completed, request_key=key: (
|
||||
self._reads.pop(request_key, None)
|
||||
if self._reads.get(request_key) is completed else None
|
||||
)
|
||||
)
|
||||
return await asyncio.shield(task)
|
||||
|
||||
async def _mutate(self, method: str, url: str, kwargs: dict) -> httpx.Response:
|
||||
await self._acquire(self._request_slots)
|
||||
try:
|
||||
return await self._http.request(method, url, **kwargs)
|
||||
finally:
|
||||
self._request_slots.release()
|
||||
|
||||
async def post(self, url: str, **kwargs) -> httpx.Response:
|
||||
return await self._mutate("POST", url, kwargs)
|
||||
|
||||
async def patch(self, url: str, **kwargs) -> httpx.Response:
|
||||
return await self._mutate("PATCH", url, kwargs)
|
||||
|
||||
async def put(self, url: str, **kwargs) -> httpx.Response:
|
||||
return await self._mutate("PUT", url, kwargs)
|
||||
|
||||
async def delete(self, url: str, **kwargs) -> httpx.Response:
|
||||
return await self._mutate("DELETE", url, kwargs)
|
||||
|
||||
@asynccontextmanager
|
||||
async def stream(self, method: str, url: str, **kwargs):
|
||||
await self._acquire(self._read_slots)
|
||||
try:
|
||||
await self._acquire(self._request_slots)
|
||||
try:
|
||||
async with self._http.stream(method, url, **kwargs) as response:
|
||||
yield response
|
||||
finally:
|
||||
self._request_slots.release()
|
||||
finally:
|
||||
self._read_slots.release()
|
||||
|
||||
def __getattr__(self, name: str):
|
||||
return getattr(self._http, name)
|
||||
|
||||
async def aclose(self) -> None:
|
||||
tasks = list(self._reads.values())
|
||||
for task in tasks:
|
||||
if not task.done():
|
||||
task.cancel()
|
||||
if tasks:
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
self._reads.clear()
|
||||
await self._http.aclose()
|
||||
|
||||
|
||||
class WorkItems(list[dict]):
|
||||
@@ -58,14 +168,19 @@ def _auth() -> dict[str, str]:
|
||||
return headers
|
||||
|
||||
|
||||
def start_client(**kwargs) -> httpx.AsyncClient:
|
||||
def start_client(**kwargs) -> GiteaTransport:
|
||||
"""Create the application-lifetime Gitea transport."""
|
||||
global _client
|
||||
_client = httpx.AsyncClient(base_url=GITEA_URL, timeout=10, **kwargs)
|
||||
kwargs.setdefault("max_concurrency", int(os.getenv("GITEA_MAX_CONCURRENCY", "8")))
|
||||
kwargs.setdefault(
|
||||
"admission_timeout",
|
||||
float(os.getenv("GITEA_ADMISSION_TIMEOUT_SECONDS", "0.25")),
|
||||
)
|
||||
_client = GiteaTransport(**kwargs)
|
||||
return _client
|
||||
|
||||
|
||||
def _get_client() -> httpx.AsyncClient:
|
||||
def _get_client() -> GiteaTransport:
|
||||
if _client is None or _client.is_closed:
|
||||
return start_client()
|
||||
return _client
|
||||
|
||||
+61
-43
@@ -12,6 +12,10 @@ class Reservation:
|
||||
response: Any = None
|
||||
|
||||
|
||||
class IdempotencyLedgerBusy(RuntimeError):
|
||||
"""Raised when a durable completion cannot acquire the SQLite write lock."""
|
||||
|
||||
|
||||
class IdempotencyLedger:
|
||||
"""Small durable ledger for replaying successful Gitea mutations."""
|
||||
|
||||
@@ -21,18 +25,22 @@ class IdempotencyLedger:
|
||||
*,
|
||||
ttl_seconds: float,
|
||||
max_entries: int,
|
||||
lock_timeout_seconds: float = 0.1,
|
||||
clock: Callable[[], float] = time.time,
|
||||
) -> None:
|
||||
self.path = Path(path)
|
||||
self.ttl_seconds = ttl_seconds
|
||||
self.max_entries = max_entries
|
||||
self.lock_timeout_seconds = lock_timeout_seconds
|
||||
self.clock = clock
|
||||
self.path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._initialize()
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
connection = sqlite3.connect(self.path, timeout=5)
|
||||
connection.execute("PRAGMA busy_timeout = 5000")
|
||||
connection = sqlite3.connect(self.path, timeout=self.lock_timeout_seconds)
|
||||
connection.execute(
|
||||
f"PRAGMA busy_timeout = {max(1, int(self.lock_timeout_seconds * 1000))}"
|
||||
)
|
||||
return connection
|
||||
|
||||
def _initialize(self) -> None:
|
||||
@@ -61,43 +69,48 @@ class IdempotencyLedger:
|
||||
def reserve(self, key: str, fingerprint: tuple[Any, ...]) -> Reservation:
|
||||
encoded = self._fingerprint(fingerprint)
|
||||
now = self.clock()
|
||||
with self._connect() as connection:
|
||||
connection.execute("BEGIN IMMEDIATE")
|
||||
connection.execute(
|
||||
"DELETE FROM idempotency_operations "
|
||||
"WHERE status = 'completed' AND completed_at <= ?",
|
||||
(now - self.ttl_seconds,),
|
||||
)
|
||||
row = connection.execute(
|
||||
"SELECT fingerprint, status, response_json FROM idempotency_operations "
|
||||
"WHERE key = ?",
|
||||
(key,),
|
||||
).fetchone()
|
||||
if row is not None:
|
||||
if row[0] != encoded:
|
||||
return Reservation("conflict")
|
||||
if row[1] == "completed":
|
||||
return Reservation("completed", json.loads(row[2]))
|
||||
return Reservation("pending")
|
||||
count = connection.execute(
|
||||
"SELECT COUNT(*) FROM idempotency_operations"
|
||||
).fetchone()[0]
|
||||
if count >= self.max_entries:
|
||||
completed = connection.execute(
|
||||
"SELECT key FROM idempotency_operations "
|
||||
"WHERE status = 'completed' ORDER BY completed_at, rowid LIMIT 1"
|
||||
).fetchone()
|
||||
if completed is None:
|
||||
return Reservation("busy")
|
||||
try:
|
||||
with self._connect() as connection:
|
||||
connection.execute("BEGIN IMMEDIATE")
|
||||
connection.execute(
|
||||
"DELETE FROM idempotency_operations WHERE key = ?",
|
||||
(completed[0],),
|
||||
"DELETE FROM idempotency_operations "
|
||||
"WHERE status = 'completed' AND completed_at <= ?",
|
||||
(now - self.ttl_seconds,),
|
||||
)
|
||||
connection.execute(
|
||||
"INSERT INTO idempotency_operations "
|
||||
"(key, fingerprint, status, created_at) VALUES (?, ?, 'pending', ?)",
|
||||
(key, encoded, now),
|
||||
)
|
||||
row = connection.execute(
|
||||
"SELECT fingerprint, status, response_json FROM idempotency_operations "
|
||||
"WHERE key = ?",
|
||||
(key,),
|
||||
).fetchone()
|
||||
if row is not None:
|
||||
if row[0] != encoded:
|
||||
return Reservation("conflict")
|
||||
if row[1] == "completed":
|
||||
return Reservation("completed", json.loads(row[2]))
|
||||
return Reservation("pending")
|
||||
count = connection.execute(
|
||||
"SELECT COUNT(*) FROM idempotency_operations"
|
||||
).fetchone()[0]
|
||||
if count >= self.max_entries:
|
||||
completed = connection.execute(
|
||||
"SELECT key FROM idempotency_operations "
|
||||
"WHERE status = 'completed' ORDER BY completed_at, rowid LIMIT 1"
|
||||
).fetchone()
|
||||
if completed is None:
|
||||
return Reservation("busy")
|
||||
connection.execute(
|
||||
"DELETE FROM idempotency_operations WHERE key = ?",
|
||||
(completed[0],),
|
||||
)
|
||||
connection.execute(
|
||||
"INSERT INTO idempotency_operations "
|
||||
"(key, fingerprint, status, created_at) VALUES (?, ?, 'pending', ?)",
|
||||
(key, encoded, now),
|
||||
)
|
||||
except sqlite3.OperationalError as exc:
|
||||
if "locked" in str(exc).lower() or "busy" in str(exc).lower():
|
||||
return Reservation("busy")
|
||||
raise
|
||||
return Reservation("reserved")
|
||||
|
||||
def clear(self) -> None:
|
||||
@@ -106,9 +119,14 @@ class IdempotencyLedger:
|
||||
|
||||
def complete(self, key: str, response: Any) -> None:
|
||||
encoded = json.dumps(response, separators=(",", ":"), sort_keys=True)
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"UPDATE idempotency_operations SET status = 'completed', "
|
||||
"response_json = ?, completed_at = ? WHERE key = ?",
|
||||
(encoded, self.clock(), key),
|
||||
)
|
||||
try:
|
||||
with self._connect() as connection:
|
||||
connection.execute(
|
||||
"UPDATE idempotency_operations SET status = 'completed', "
|
||||
"response_json = ?, completed_at = ? WHERE key = ?",
|
||||
(encoded, self.clock(), key),
|
||||
)
|
||||
except sqlite3.OperationalError as exc:
|
||||
if "locked" in str(exc).lower() or "busy" in str(exc).lower():
|
||||
raise IdempotencyLedgerBusy from exc
|
||||
raise
|
||||
+35
-5
@@ -26,7 +26,7 @@ from src.gitea_proxy import (
|
||||
pull_review_detail,
|
||||
repos,
|
||||
)
|
||||
from src.idempotency import IdempotencyLedger
|
||||
from src.idempotency import IdempotencyLedger, IdempotencyLedgerBusy
|
||||
from src.models import Issue, Milestone, PullRequest, Repo, User
|
||||
from src.suggestion_engine import compute
|
||||
from src.views import router as frontend_router
|
||||
@@ -66,6 +66,9 @@ ISSUE_CREATION_IDEMPOTENCY_TTL_SECONDS = 600.0
|
||||
ISSUE_CREATION_IDEMPOTENCY_MAX_ENTRIES = 256
|
||||
AUTHORED_ACTION_IDEMPOTENCY_TTL_SECONDS = 600.0
|
||||
AUTHORED_ACTION_IDEMPOTENCY_MAX_ENTRIES = 256
|
||||
AUTHORED_ACTION_LEDGER_LOCK_TIMEOUT_SECONDS = float(
|
||||
os.getenv("STACKCHAIN_IDEMPOTENCY_LOCK_TIMEOUT_SECONDS", "0.1")
|
||||
)
|
||||
NOTIFICATION_MUTATION_TIMEOUT_SECONDS = 5.0
|
||||
NOTIFICATION_PAGE_TIMEOUT_SECONDS = 5.0
|
||||
WORK_PAGE_TIMEOUT_SECONDS = 5.0
|
||||
@@ -105,6 +108,7 @@ _idempotency_ledger = IdempotencyLedger(
|
||||
AUTHORED_ACTION_IDEMPOTENCY_MAX_ENTRIES
|
||||
+ ISSUE_CREATION_IDEMPOTENCY_MAX_ENTRIES
|
||||
),
|
||||
lock_timeout_seconds=AUTHORED_ACTION_LEDGER_LOCK_TIMEOUT_SECONDS,
|
||||
)
|
||||
_available_issue_snapshot_task: asyncio.Task | None = None
|
||||
_available_issue_snapshot_value: list[dict] | None = None
|
||||
@@ -264,7 +268,9 @@ async def _run_idempotent_authored_action(
|
||||
if not idempotency_key:
|
||||
return await asyncio.wait_for(operation, timeout=timeout)
|
||||
|
||||
reservation = _idempotency_ledger.reserve(idempotency_key, fingerprint)
|
||||
reservation = await asyncio.to_thread(
|
||||
_idempotency_ledger.reserve, idempotency_key, fingerprint
|
||||
)
|
||||
if reservation.state == "conflict":
|
||||
operation.close()
|
||||
raise HTTPException(status_code=409, detail="Idempotency key already used")
|
||||
@@ -294,7 +300,7 @@ async def _run_idempotent_authored_action(
|
||||
else:
|
||||
async def persist_result():
|
||||
result = await operation
|
||||
_idempotency_ledger.complete(idempotency_key, result)
|
||||
await asyncio.to_thread(_idempotency_ledger.complete, idempotency_key, result)
|
||||
return result
|
||||
|
||||
task = asyncio.create_task(persist_result())
|
||||
@@ -310,7 +316,14 @@ async def _run_idempotent_authored_action(
|
||||
_authored_action_operations.pop(idempotency_key, None)
|
||||
|
||||
task.add_done_callback(discard_completed)
|
||||
return await asyncio.wait_for(asyncio.shield(task), timeout=timeout)
|
||||
try:
|
||||
return await asyncio.wait_for(asyncio.shield(task), timeout=timeout)
|
||||
except IdempotencyLedgerBusy:
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="This action may have completed; verify its result before retrying",
|
||||
headers={"Retry-After": "5"},
|
||||
)
|
||||
|
||||
|
||||
def _context_payload(user_data, repo_data, issues_data, prs_data) -> dict:
|
||||
@@ -413,7 +426,7 @@ app.include_router(frontend_router)
|
||||
@app.middleware("http")
|
||||
async def prevent_live_api_caching(request, call_next):
|
||||
response = await call_next(request)
|
||||
if request.url.path in {"/api/v1/context", "/api/v1/events", "/api/v1/live", "/api/v1/available-issues", "/api/v1/search", "/api/v1/work-route"} or request.url.path.startswith("/api/v1/work/") or (
|
||||
if request.url.path in {"/api/v1/context", "/api/v1/background-identity", "/api/v1/events", "/api/v1/live", "/api/v1/available-issues", "/api/v1/search", "/api/v1/work-route"} or request.url.path.startswith("/api/v1/work/") or (
|
||||
request.url.path.startswith("/api/v1/repos/")
|
||||
and request.url.path.endswith("/review")
|
||||
) or request.url.path.startswith("/api/v1/notifications") or (
|
||||
@@ -520,6 +533,23 @@ async def context() -> JSONResponse:
|
||||
return JSONResponse(_context_payload(user_data, repo_data, issues_data, prs_data))
|
||||
|
||||
|
||||
@app.get("/api/v1/background-identity")
|
||||
async def background_identity() -> JSONResponse:
|
||||
"""Return only the account key required to safely drain a browser outbox."""
|
||||
try:
|
||||
user = await current_user()
|
||||
login = user.get("login") if isinstance(user, dict) else None
|
||||
if not isinstance(login, str) or not login.strip():
|
||||
raise ValueError("Gitea user response did not include a login")
|
||||
except Exception:
|
||||
return JSONResponse(
|
||||
{"error": "Gitea identity is temporarily unavailable"},
|
||||
status_code=503,
|
||||
headers={"Retry-After": "1"},
|
||||
)
|
||||
return JSONResponse({"login": login})
|
||||
|
||||
|
||||
@app.get("/api/v1/search")
|
||||
async def global_search(
|
||||
q: str = Query(min_length=2, max_length=100),
|
||||
|
||||
@@ -1,4 +1,7 @@
|
||||
import asyncio
|
||||
import sqlite3
|
||||
import threading
|
||||
import time
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
@@ -228,3 +231,170 @@ async def test_orphaned_pending_comment_fails_closed_without_upstream_retry(
|
||||
"This action may still be completing; verify its result before retrying"
|
||||
)
|
||||
assert calls == 0
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_slow_ledger_reservation_does_not_block_health_requests(monkeypatch):
|
||||
async def assigned(_repository, _number):
|
||||
return True
|
||||
|
||||
async def comment(_repository, _number, body):
|
||||
return {"id": 248, "body": body}
|
||||
|
||||
started = threading.Event()
|
||||
original_reserve = main._idempotency_ledger.reserve
|
||||
|
||||
def slow_reserve(key, fingerprint):
|
||||
started.set()
|
||||
time.sleep(0.2)
|
||||
return original_reserve(key, fingerprint)
|
||||
|
||||
monkeypatch.setattr(main._idempotency_ledger, "reserve", slow_reserve)
|
||||
monkeypatch.setattr(main.gitea_proxy, "is_assigned_issue", assigned)
|
||||
monkeypatch.setattr(main.gitea_proxy, "comment_on_issue", comment)
|
||||
transport = httpx.ASGITransport(app=main.app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
started_at = asyncio.get_running_loop().time()
|
||||
authored = asyncio.create_task(client.post(
|
||||
"/api/v1/repos/stackchain/api/issues/248/comments",
|
||||
json={"body": "Keep the dashboard responsive"},
|
||||
headers={"Idempotency-Key": "contention-248"},
|
||||
))
|
||||
assert await asyncio.to_thread(started.wait, 1)
|
||||
health = await client.get("/healthz")
|
||||
health_elapsed = asyncio.get_running_loop().time() - started_at
|
||||
authored_response = await authored
|
||||
|
||||
assert health.status_code == 200
|
||||
assert health_elapsed < 0.15
|
||||
assert authored_response.status_code == 201
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_locked_ledger_returns_bounded_retryable_response(monkeypatch, tmp_path):
|
||||
upstream_calls = 0
|
||||
|
||||
async def assigned(_repository, _number):
|
||||
return True
|
||||
|
||||
async def comment(_repository, _number, body):
|
||||
nonlocal upstream_calls
|
||||
upstream_calls += 1
|
||||
return {"id": 248, "body": body}
|
||||
|
||||
database = tmp_path / "contended.sqlite3"
|
||||
ledger = IdempotencyLedger(
|
||||
database,
|
||||
ttl_seconds=600,
|
||||
max_entries=256,
|
||||
lock_timeout_seconds=0.05,
|
||||
)
|
||||
monkeypatch.setattr(main, "_idempotency_ledger", ledger)
|
||||
monkeypatch.setattr(main.gitea_proxy, "is_assigned_issue", assigned)
|
||||
monkeypatch.setattr(main.gitea_proxy, "comment_on_issue", comment)
|
||||
blocker = sqlite3.connect(database)
|
||||
blocker.execute("BEGIN IMMEDIATE")
|
||||
transport = httpx.ASGITransport(app=main.app)
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
started_at = asyncio.get_running_loop().time()
|
||||
response = await client.post(
|
||||
"/api/v1/repos/stackchain/api/issues/248/comments",
|
||||
json={"body": "Retry without freezing"},
|
||||
headers={"Idempotency-Key": "locked-ledger-248"},
|
||||
)
|
||||
elapsed = asyncio.get_running_loop().time() - started_at
|
||||
finally:
|
||||
blocker.rollback()
|
||||
blocker.close()
|
||||
|
||||
assert response.status_code == 503
|
||||
assert response.headers["Retry-After"] == "1"
|
||||
assert response.json()["detail"] == "Authored action queue is busy; please retry"
|
||||
assert elapsed < 0.5
|
||||
assert upstream_calls == 0
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_slow_ledger_completion_does_not_block_health_requests(monkeypatch):
|
||||
async def assigned(_repository, _number):
|
||||
return True
|
||||
|
||||
async def comment(_repository, _number, body):
|
||||
return {"id": 249, "body": body}
|
||||
|
||||
started = threading.Event()
|
||||
original_complete = main._idempotency_ledger.complete
|
||||
|
||||
def slow_complete(key, response):
|
||||
started.set()
|
||||
time.sleep(0.2)
|
||||
return original_complete(key, response)
|
||||
|
||||
monkeypatch.setattr(main._idempotency_ledger, "complete", slow_complete)
|
||||
monkeypatch.setattr(main.gitea_proxy, "is_assigned_issue", assigned)
|
||||
monkeypatch.setattr(main.gitea_proxy, "comment_on_issue", comment)
|
||||
transport = httpx.ASGITransport(app=main.app)
|
||||
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
started_at = asyncio.get_running_loop().time()
|
||||
authored = asyncio.create_task(client.post(
|
||||
"/api/v1/repos/stackchain/api/issues/248/comments",
|
||||
json={"body": "Persist without freezing"},
|
||||
headers={"Idempotency-Key": "slow-completion-248"},
|
||||
))
|
||||
assert await asyncio.to_thread(started.wait, 1)
|
||||
health = await client.get("/healthz")
|
||||
health_elapsed = asyncio.get_running_loop().time() - started_at
|
||||
authored_response = await authored
|
||||
|
||||
assert health.status_code == 200
|
||||
assert health_elapsed < 0.15
|
||||
assert authored_response.status_code == 201
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_locked_completion_fails_closed_with_retry_guidance(monkeypatch, tmp_path):
|
||||
database = tmp_path / "completion-contention.sqlite3"
|
||||
ledger = IdempotencyLedger(
|
||||
database, ttl_seconds=600, max_entries=256, lock_timeout_seconds=0.05
|
||||
)
|
||||
blocker = None
|
||||
|
||||
async def assigned(_repository, _number):
|
||||
return True
|
||||
|
||||
async def comment(_repository, _number, body):
|
||||
nonlocal blocker
|
||||
blocker = sqlite3.connect(database)
|
||||
blocker.execute("BEGIN IMMEDIATE")
|
||||
return {"id": 250, "body": body}
|
||||
|
||||
monkeypatch.setattr(main, "_idempotency_ledger", ledger)
|
||||
monkeypatch.setattr(main.gitea_proxy, "is_assigned_issue", assigned)
|
||||
monkeypatch.setattr(main.gitea_proxy, "comment_on_issue", comment)
|
||||
transport = httpx.ASGITransport(app=main.app)
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
response = await client.post(
|
||||
"/api/v1/repos/stackchain/api/issues/248/comments",
|
||||
json={"body": "The upstream mutation may already exist"},
|
||||
headers={"Idempotency-Key": "locked-completion-248"},
|
||||
)
|
||||
finally:
|
||||
if blocker is not None:
|
||||
blocker.rollback()
|
||||
blocker.close()
|
||||
|
||||
assert response.status_code == 503
|
||||
assert response.headers["Retry-After"] == "5"
|
||||
assert response.json()["detail"] == (
|
||||
"This action may have completed; verify its result before retrying"
|
||||
)
|
||||
assert ledger.reserve(
|
||||
"locked-completion-248",
|
||||
("issue-comment", "stackchain/api", 248, "The upstream mutation may already exist"),
|
||||
).state == "pending"
|
||||
@@ -0,0 +1,315 @@
|
||||
import json
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from src import main
|
||||
from src.views import dashboard
|
||||
|
||||
|
||||
SYNC = Path(__file__).parents[1] / "frontend" / "background-issue-sync.js"
|
||||
|
||||
|
||||
def run_node(script: str) -> dict:
|
||||
completed = subprocess.run(
|
||||
["node", "-e", script], capture_output=True, check=True, text=True
|
||||
)
|
||||
return json.loads(completed.stdout)
|
||||
|
||||
|
||||
def test_closed_app_sync_delivers_matching_issue_once_with_original_idempotency_key():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
const item = {{
|
||||
id:'capture-1', operationId:'capture-1', ownerLogin:'timmy', status:'queued',
|
||||
repository:'stackchain/api', title:'Offline report', body:'Full context',
|
||||
labelIds:[3], milestoneId:4, dueDate:'2026-08-09', queuedAt:100,
|
||||
}};
|
||||
const state = {{item, completed:[], released:[], failed:[], calls:[]}};
|
||||
const store = {{
|
||||
claimNext: async owner => state.item && state.item.ownerLogin === owner ? {{...state.item}} : null,
|
||||
complete: async id => {{ state.completed.push(id); state.item = null; }},
|
||||
release: async id => state.released.push(id),
|
||||
fail: async (id, message) => state.failed.push({{id,message}}),
|
||||
}};
|
||||
const fetchJson = async (url, options = {{}}) => {{
|
||||
state.calls.push({{url,options}});
|
||||
if (url === 'api/v1/background-identity') return {{login:'timmy'}};
|
||||
return {{repository:'stackchain/api',number:251,title:'Offline report'}};
|
||||
}};
|
||||
(async () => {{
|
||||
const sync = createBackgroundIssueSync({{store,fetchJson}});
|
||||
const result = await sync.flush();
|
||||
process.stdout.write(JSON.stringify({{state,result}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["result"]["confirmed"][0]["number"] == 251
|
||||
assert output["state"]["completed"] == ["capture-1"]
|
||||
assert output["state"]["released"] == []
|
||||
assert output["state"]["failed"] == []
|
||||
assert output["state"]["calls"][0]["url"] == "api/v1/background-identity"
|
||||
mutation = output["state"]["calls"][1]
|
||||
assert mutation["url"] == "api/v1/repos/stackchain/api/issues"
|
||||
assert mutation["options"]["headers"]["Idempotency-Key"] == "capture-1"
|
||||
assert json.loads(mutation["options"]["body"]) == {
|
||||
"title": "Offline report",
|
||||
"body": "Full context",
|
||||
"label_ids": [3],
|
||||
"milestone_id": 4,
|
||||
"due_date": "2026-08-09T23:59:59Z",
|
||||
}
|
||||
|
||||
|
||||
def test_closed_app_sync_leaves_another_accounts_issue_queued():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
const state = {{owners:[],mutations:0}};
|
||||
const store = {{
|
||||
claimNext: async owner => {{ state.owners.push(owner); return null; }},
|
||||
countBlocked: async owner => owner === 'alexander' ? 1 : 0,
|
||||
complete: async () => {{}},
|
||||
}};
|
||||
const fetchJson = async url => {{
|
||||
if (url === 'api/v1/background-identity') return {{login:'alexander'}};
|
||||
state.mutations += 1;
|
||||
}};
|
||||
(async () => {{
|
||||
const result = await createBackgroundIssueSync({{store,fetchJson}}).flush();
|
||||
process.stdout.write(JSON.stringify({{state,result}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["state"] == {"owners": ["alexander"], "mutations": 0}
|
||||
assert output["result"]["confirmed"] == []
|
||||
assert output["result"]["blocked"] == 1
|
||||
|
||||
|
||||
def test_transient_delivery_failure_releases_claim_and_requests_another_sync():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
let item = {{id:'capture-2',operationId:'capture-2',ownerLogin:'timmy',repository:'o/r',title:'Retry',body:'Later',labelIds:[]}};
|
||||
const state = {{released:[],completed:[]}};
|
||||
const store = {{
|
||||
claimNext: async () => item ? (item = null, {{id:'capture-2',operationId:'capture-2',ownerLogin:'timmy',repository:'o/r',title:'Retry',body:'Later',labelIds:[]}}) : null,
|
||||
release: async id => state.released.push(id),
|
||||
complete: async id => state.completed.push(id),
|
||||
}};
|
||||
const fetchJson = async url => {{
|
||||
if (url === 'api/v1/background-identity') return {{login:'timmy'}};
|
||||
const error = new Error('Gitea unavailable'); error.status = 503; throw error;
|
||||
}};
|
||||
(async () => {{
|
||||
let error = null;
|
||||
try {{ await createBackgroundIssueSync({{store,fetchJson}}).flush(); }}
|
||||
catch (caught) {{ error = caught.message; }}
|
||||
process.stdout.write(JSON.stringify({{state,error}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["state"]["released"] == ["capture-2"]
|
||||
assert output["state"]["completed"] == []
|
||||
assert output["error"] == "Gitea unavailable"
|
||||
|
||||
|
||||
def test_permanent_delivery_failure_marks_issue_for_foreground_attention():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
let claimed = false;
|
||||
const state = {{failed:[],released:[]}};
|
||||
const store = {{
|
||||
claimNext: async () => claimed ? null : (claimed = true, {{id:'capture-3',operationId:'capture-3',ownerLogin:'timmy',repository:'o/r',title:'Invalid',body:'',labelIds:[]}}),
|
||||
fail: async (id, message) => state.failed.push({{id,message}}),
|
||||
release: async id => state.released.push(id),
|
||||
countBlocked: async () => 0,
|
||||
}};
|
||||
const fetchJson = async url => {{
|
||||
if (url === 'api/v1/background-identity') return {{login:'timmy'}};
|
||||
const error = new Error('Title is invalid'); error.status = 422; throw error;
|
||||
}};
|
||||
(async () => {{
|
||||
const result = await createBackgroundIssueSync({{store,fetchJson}}).flush();
|
||||
process.stdout.write(JSON.stringify({{state,result}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["state"]["failed"] == [
|
||||
{"id": "capture-3", "message": "Title is invalid"}
|
||||
]
|
||||
assert output["state"]["released"] == []
|
||||
assert output["result"]["attention"] == 1
|
||||
|
||||
|
||||
def test_issue_store_atomically_grants_one_delivery_claim():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
const records = new Map();
|
||||
let tail = Promise.resolve();
|
||||
const transaction = work => {{
|
||||
const run = tail.then(() => work({{
|
||||
getAll: async () => [...records.values()].map(value => ({{...value}})),
|
||||
put: async value => records.set(value.id, {{...value}}),
|
||||
delete: async id => records.delete(id),
|
||||
}}));
|
||||
tail = run.catch(() => {{}});
|
||||
return run;
|
||||
}};
|
||||
(async () => {{
|
||||
const store = createBackgroundIssueSync.createIssueSyncStore({{transaction,now:()=>1000,claimMs:5000}});
|
||||
await store.reconcile([{{id:'same',operationId:'same',ownerLogin:'timmy',status:'queued',repository:'o/r',title:'Once',labelIds:[]}}]);
|
||||
const claims = await Promise.all([store.claimNext('timmy'),store.claimNext('timmy')]);
|
||||
process.stdout.write(JSON.stringify({{claims,records:[...records.values()]}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert sum(claim is not None for claim in output["claims"]) == 1
|
||||
assert output["records"][0]["status"] == "sending"
|
||||
assert output["records"][0]["claimUntil"] == 6000
|
||||
|
||||
|
||||
def test_foreground_and_worker_race_still_posts_one_issue():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
const records = new Map(); let tail = Promise.resolve();
|
||||
const transaction = work => {{
|
||||
const run = tail.then(() => work({{
|
||||
getAll:async()=>[...records.values()].map(value=>({{...value}})),
|
||||
put:async value=>records.set(value.id,{{...value}}), delete:async id=>records.delete(id),
|
||||
}}));
|
||||
tail=run.catch(()=>{{}}); return run;
|
||||
}};
|
||||
const item={{id:'race',operationId:'race',ownerLogin:'timmy',status:'queued',repository:'o/r',title:'Exactly once',body:'',labelIds:[]}};
|
||||
let mutations=0;
|
||||
const fetchJson=async url=>{{
|
||||
if(url==='api/v1/background-identity') return {{login:'timmy'}};
|
||||
mutations+=1; return {{number:77}};
|
||||
}};
|
||||
(async()=>{{
|
||||
const store=createBackgroundIssueSync.createIssueSyncStore({{transaction}});
|
||||
await store.reconcile([item]);
|
||||
const sync=createBackgroundIssueSync({{store,fetchJson}});
|
||||
const [foreground,worker]=await Promise.all([sync.send(item,'timmy'),sync.flush()]);
|
||||
process.stdout.write(JSON.stringify({{foreground,worker,mutations,remaining:[...records.values()]}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["mutations"] == 1
|
||||
assert len(output["remaining"]) == 1
|
||||
assert output["remaining"][0]["status"] == "sent"
|
||||
delivered = int(bool(output["foreground"].get("issue"))) + len(
|
||||
output["worker"]["confirmed"]
|
||||
)
|
||||
assert delivered == 1
|
||||
|
||||
|
||||
def test_foreground_send_upserts_before_claim_when_mirror_is_still_pending():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
let record = null; const state={{upserts:0,mutations:0}};
|
||||
const store={{
|
||||
upsert:async item=>{{state.upserts+=1;record={{...item}};}},
|
||||
claim:async(id,owner)=>record?.id===id&&record?.ownerLogin===owner?{{...record}}:null,
|
||||
complete:async()=>{{record=null;}}, release:async()=>{{}}, fail:async()=>{{}},
|
||||
}};
|
||||
const fetchJson=async()=>{{state.mutations+=1;return {{number:88}};}};
|
||||
(async()=>{{
|
||||
const sync=createBackgroundIssueSync({{store,fetchJson}});
|
||||
const result=await sync.send({{id:'early',operationId:'early',ownerLogin:'timmy',repository:'o/r',title:'Fast',body:'',labelIds:[]}},'timmy');
|
||||
process.stdout.write(JSON.stringify({{state,result,record}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["state"] == {"upserts": 1, "mutations": 1}
|
||||
assert output["result"]["issue"]["number"] == 88
|
||||
assert output["record"] is None
|
||||
|
||||
|
||||
def test_completed_delivery_leaves_non_replayable_tombstone_for_next_page():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
const records=new Map();let tail=Promise.resolve();
|
||||
const transaction=work=>{{const run=tail.then(()=>work({{
|
||||
getAll:async()=>[...records.values()].map(value=>({{...value}})),
|
||||
put:async value=>records.set(value.id,{{...value}}),delete:async id=>records.delete(id),
|
||||
}}));tail=run.catch(()=>{{}});return run;}};
|
||||
(async()=>{{
|
||||
const store=createBackgroundIssueSync.createIssueSyncStore({{transaction,now:()=>100}});
|
||||
await store.reconcile([{{id:'done',ownerLogin:'timmy',status:'queued'}}]);
|
||||
await store.claimNext('timmy');
|
||||
await store.complete('done');
|
||||
const replay=await store.claimNext('timmy');
|
||||
const snapshot=await store.snapshot();
|
||||
process.stdout.write(JSON.stringify({{replay,snapshot}}));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["replay"] is None
|
||||
assert output["snapshot"] == [
|
||||
{"id": "done", "ownerLogin": "timmy", "status": "sent", "claimUntil": 0}
|
||||
]
|
||||
|
||||
|
||||
def test_user_edit_resets_worker_attention_item_for_retry():
|
||||
script = f"""
|
||||
const createBackgroundIssueSync = require({json.dumps(str(SYNC))});
|
||||
const records=new Map([['edit',{{id:'edit',ownerLogin:'timmy',status:'attention',title:'Bad',error:'Invalid'}}]]);
|
||||
let tail=Promise.resolve();
|
||||
const transaction=work=>{{const run=tail.then(()=>work({{
|
||||
getAll:async()=>[...records.values()].map(value=>({{...value}})),
|
||||
put:async value=>records.set(value.id,{{...value}}),delete:async id=>records.delete(id),
|
||||
}}));tail=run.catch(()=>{{}});return run;}};
|
||||
(async()=>{{
|
||||
const store=createBackgroundIssueSync.createIssueSyncStore({{transaction,now:()=>100}});
|
||||
await store.reconcile([{{id:'edit',ownerLogin:'timmy',status:'queued',title:'Fixed'}}]);
|
||||
const claimed=await store.claimNext('timmy');
|
||||
process.stdout.write(JSON.stringify(claimed));
|
||||
}})();
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["title"] == "Fixed"
|
||||
assert output["status"] == "sending"
|
||||
assert "error" not in output
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_background_identity_is_lightweight_and_never_cacheable(monkeypatch):
|
||||
calls = 0
|
||||
|
||||
async def user():
|
||||
nonlocal calls
|
||||
calls += 1
|
||||
return {"id": 7, "login": "timmy", "email": "private@example.com"}
|
||||
|
||||
monkeypatch.setattr(main, "current_user", user)
|
||||
transport = httpx.ASGITransport(app=main.app)
|
||||
async with httpx.AsyncClient(transport=transport, base_url="http://test") as client:
|
||||
response = await client.get("/api/v1/background-identity")
|
||||
|
||||
assert response.status_code == 200
|
||||
assert response.json() == {"login": "timmy"}
|
||||
assert response.headers["cache-control"] == "no-store"
|
||||
assert calls == 1
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_dashboard_wires_indexeddb_outbox_and_background_sync_fallback():
|
||||
html = await dashboard()
|
||||
|
||||
assert '<script src="static/background-issue-sync.js"></script>' in html
|
||||
assert "const backgroundIssueStore = createIssueSyncStore();" in html
|
||||
assert "createBackgroundIssueSync({" in html
|
||||
assert "backgroundSync: backgroundIssueSync" in html
|
||||
assert "registration.sync.register('stackchain-issue-outbox-v1')" in html
|
||||
assert "backgroundIssueSync.snapshot().then(records => issueOutbox.reconcileBackground(records))" in html
|
||||
assert "if ('indexedDB' in window)" in html
|
||||
@@ -28,6 +28,198 @@ async def test_gitea_transport_is_reused_across_requests_and_closed():
|
||||
assert client.is_closed
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_gitea_transport_coalesces_identical_concurrent_gets():
|
||||
calls = 0
|
||||
release = asyncio.Event()
|
||||
|
||||
async def handler(request):
|
||||
nonlocal calls
|
||||
calls += 1
|
||||
await release.wait()
|
||||
return httpx.Response(200, json={"path": request.url.path})
|
||||
|
||||
gitea_proxy.start_client(
|
||||
transport=httpx.MockTransport(handler),
|
||||
max_concurrency=4,
|
||||
admission_timeout=0.1,
|
||||
)
|
||||
try:
|
||||
requests = [asyncio.create_task(gitea_proxy.fetch("user")) for _ in range(20)]
|
||||
await asyncio.sleep(0)
|
||||
release.set()
|
||||
results = await asyncio.gather(*requests)
|
||||
finally:
|
||||
await gitea_proxy.stop_client()
|
||||
|
||||
assert calls == 1
|
||||
assert results == [{"path": "/api/v1/user"}] * 20
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_cancelling_one_coalesced_waiter_keeps_the_shared_get_running():
|
||||
started = asyncio.Event()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def handler(request):
|
||||
started.set()
|
||||
await release.wait()
|
||||
return httpx.Response(200, json={"path": request.url.path})
|
||||
|
||||
gitea_proxy.start_client(transport=httpx.MockTransport(handler))
|
||||
owner = asyncio.create_task(gitea_proxy.fetch("user"))
|
||||
try:
|
||||
await started.wait()
|
||||
waiter = asyncio.create_task(gitea_proxy.fetch("user"))
|
||||
await asyncio.sleep(0)
|
||||
waiter.cancel()
|
||||
with pytest.raises(asyncio.CancelledError):
|
||||
await waiter
|
||||
release.set()
|
||||
assert await owner == {"path": "/api/v1/user"}
|
||||
finally:
|
||||
release.set()
|
||||
await asyncio.gather(owner, return_exceptions=True)
|
||||
await gitea_proxy.stop_client()
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_gitea_transport_rejects_reads_beyond_bounded_capacity():
|
||||
active = 0
|
||||
peak = 0
|
||||
saturated = asyncio.Event()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def handler(request):
|
||||
nonlocal active, peak
|
||||
active += 1
|
||||
peak = max(peak, active)
|
||||
if active == 2:
|
||||
saturated.set()
|
||||
try:
|
||||
await release.wait()
|
||||
return httpx.Response(200, json={"path": request.url.path})
|
||||
finally:
|
||||
active -= 1
|
||||
|
||||
gitea_proxy.start_client(
|
||||
transport=httpx.MockTransport(handler),
|
||||
max_concurrency=3,
|
||||
admission_timeout=0.02,
|
||||
)
|
||||
first = asyncio.create_task(gitea_proxy.fetch("first"))
|
||||
second = asyncio.create_task(gitea_proxy.fetch("second"))
|
||||
try:
|
||||
await asyncio.wait_for(saturated.wait(), timeout=0.2)
|
||||
with pytest.raises(gitea_proxy.GiteaOverloadedError):
|
||||
await gitea_proxy.fetch("third")
|
||||
finally:
|
||||
release.set()
|
||||
await asyncio.gather(first, second)
|
||||
await gitea_proxy.stop_client()
|
||||
|
||||
assert peak == 2
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_gitea_transport_reserves_one_slot_for_mutations_and_bounds_them():
|
||||
active = 0
|
||||
peak = 0
|
||||
reads_started = asyncio.Event()
|
||||
mutation_started = asyncio.Event()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def handler(request):
|
||||
nonlocal active, peak
|
||||
active += 1
|
||||
peak = max(peak, active)
|
||||
if active == 2:
|
||||
reads_started.set()
|
||||
if request.method == "POST":
|
||||
mutation_started.set()
|
||||
try:
|
||||
await release.wait()
|
||||
return httpx.Response(200, json={"method": request.method})
|
||||
finally:
|
||||
active -= 1
|
||||
|
||||
client = gitea_proxy.start_client(
|
||||
transport=httpx.MockTransport(handler),
|
||||
max_concurrency=3,
|
||||
admission_timeout=0.02,
|
||||
)
|
||||
reads = [
|
||||
asyncio.create_task(gitea_proxy.fetch("first")),
|
||||
asyncio.create_task(gitea_proxy.fetch("second")),
|
||||
]
|
||||
mutation = None
|
||||
try:
|
||||
await asyncio.wait_for(reads_started.wait(), timeout=0.2)
|
||||
mutation = asyncio.create_task(client.post("/api/v1/mutate", json={"value": 1}))
|
||||
await asyncio.wait_for(mutation_started.wait(), timeout=0.2)
|
||||
with pytest.raises(gitea_proxy.GiteaOverloadedError):
|
||||
await asyncio.wait_for(
|
||||
client.post("/api/v1/mutate", json={"value": 1}), timeout=0.1
|
||||
)
|
||||
finally:
|
||||
release.set()
|
||||
await asyncio.gather(*reads, *(task for task in [mutation] if task is not None))
|
||||
await gitea_proxy.stop_client()
|
||||
|
||||
assert peak == 3
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_gitea_transport_reads_bulkhead_configuration_from_environment(monkeypatch):
|
||||
monkeypatch.setenv("GITEA_MAX_CONCURRENCY", "12")
|
||||
monkeypatch.setenv("GITEA_ADMISSION_TIMEOUT_SECONDS", "0.75")
|
||||
|
||||
client = gitea_proxy.start_client(
|
||||
transport=httpx.MockTransport(lambda request: httpx.Response(200, json={}))
|
||||
)
|
||||
try:
|
||||
assert client.max_concurrency == 12
|
||||
assert client.admission_timeout == 0.75
|
||||
finally:
|
||||
await gitea_proxy.stop_client()
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_streaming_reads_share_the_same_bounded_read_capacity():
|
||||
active = 0
|
||||
saturated = asyncio.Event()
|
||||
release = asyncio.Event()
|
||||
|
||||
async def handler(request):
|
||||
nonlocal active
|
||||
active += 1
|
||||
if active == 2:
|
||||
saturated.set()
|
||||
try:
|
||||
await release.wait()
|
||||
return httpx.Response(200, content=b"diff")
|
||||
finally:
|
||||
active -= 1
|
||||
|
||||
gitea_proxy.start_client(
|
||||
transport=httpx.MockTransport(handler),
|
||||
max_concurrency=3,
|
||||
admission_timeout=0.02,
|
||||
)
|
||||
streams = [
|
||||
asyncio.create_task(gitea_proxy.fetch_text("first", 100)),
|
||||
asyncio.create_task(gitea_proxy.fetch_text("second", 100)),
|
||||
]
|
||||
try:
|
||||
await asyncio.wait_for(saturated.wait(), timeout=0.2)
|
||||
with pytest.raises(gitea_proxy.GiteaOverloadedError):
|
||||
await asyncio.wait_for(gitea_proxy.fetch("third"), timeout=0.1)
|
||||
finally:
|
||||
release.set()
|
||||
await asyncio.gather(*streams)
|
||||
await gitea_proxy.stop_client()
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_application_lifespan_opens_and_closes_gitea_transport(monkeypatch):
|
||||
calls = []
|
||||
|
||||
@@ -163,6 +163,100 @@ Promise.all([reconnect,sendNow]).then(results => process.stdout.write(JSON.strin
|
||||
assert sum(len(result["confirmed"]) for result in output["results"]) >= 1
|
||||
|
||||
|
||||
def test_issue_outbox_mirrors_queue_and_registers_background_sync():
|
||||
script = f"""
|
||||
const createIssueOutbox = require({json.dumps(str(OUTBOX))});
|
||||
const values = new Map();
|
||||
const storage = {{getItem:k=>values.get(k)||null,setItem:(k,v)=>values.set(k,v)}};
|
||||
const state = {{snapshots:[],syncs:0}};
|
||||
const backgroundSync = {{
|
||||
reconcile: async items => state.snapshots.push(items.map(item => ({{...item}}))),
|
||||
requestSync: async () => {{ state.syncs += 1; }},
|
||||
}};
|
||||
const outbox = createIssueOutbox({{
|
||||
storage, backgroundSync, getOwnerLogin:()=> 'timmy', createOperationId:()=> 'background-1',
|
||||
}});
|
||||
outbox.enqueue({{repository:'stackchain/api',title:'Close the app',body:'Still deliver'}});
|
||||
setTimeout(() => process.stdout.write(JSON.stringify(state)), 0);
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["snapshots"][0][0]["operationId"] == "background-1"
|
||||
assert output["snapshots"][0][0]["ownerLogin"] == "timmy"
|
||||
assert output["syncs"] == 1
|
||||
|
||||
|
||||
def test_foreground_delivery_uses_same_atomic_background_claim():
|
||||
script = f"""
|
||||
const createIssueOutbox = require({json.dumps(str(OUTBOX))});
|
||||
const values = new Map();
|
||||
const storage = {{getItem:k=>values.get(k)||null,setItem:(k,v)=>values.set(k,v)}};
|
||||
const state = {{backgroundCalls:0,directCalls:0}};
|
||||
const backgroundSync = {{
|
||||
reconcile: async () => {{}}, requestSync: async () => {{}},
|
||||
send: async (item, login) => {{ state.backgroundCalls += 1; return {{issue:{{number:9}}, claimed:item.id + ':' + login}}; }},
|
||||
}};
|
||||
const outbox = createIssueOutbox({{
|
||||
storage, backgroundSync, getOwnerLogin:()=> 'timmy', createOperationId:()=> 'shared-claim',
|
||||
fetchJson:async () => {{ state.directCalls += 1; return {{number:10}}; }},
|
||||
}});
|
||||
outbox.enqueue({{repository:'o/r',title:'One owner'}});
|
||||
outbox.flush('timmy').then(result => process.stdout.write(JSON.stringify({{state,result,remaining:outbox.list()}})));
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["state"] == {"backgroundCalls": 1, "directCalls": 0}
|
||||
assert output["result"]["confirmed"][0]["number"] == 9
|
||||
assert output["remaining"] == []
|
||||
|
||||
|
||||
def test_foreground_background_delivery_surfaces_permanent_failure_immediately():
|
||||
script = f"""
|
||||
const createIssueOutbox = require({json.dumps(str(OUTBOX))});
|
||||
const values=new Map();const storage={{getItem:k=>values.get(k)||null,setItem:(k,v)=>values.set(k,v)}};
|
||||
const error=new Error('Title is invalid');error.status=422;
|
||||
const backgroundSync={{reconcile:async()=>{{}},requestSync:async()=>{{}},send:async()=>({{attention:true,error}})}};
|
||||
const outbox=createIssueOutbox({{storage,backgroundSync,getOwnerLogin:()=>'timmy',createOperationId:()=>'invalid'}});
|
||||
outbox.enqueue({{repository:'o/r',title:'Bad'}});
|
||||
outbox.flush('timmy').then(result=>process.stdout.write(JSON.stringify({{result,items:outbox.list()}})));
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output["items"][0]["status"] == "attention"
|
||||
assert output["items"][0]["error"] == "Title is invalid"
|
||||
assert output["result"]["confirmed"] == []
|
||||
|
||||
|
||||
def test_page_reconciles_worker_success_and_attention_into_visible_outbox():
|
||||
script = f"""
|
||||
const createIssueOutbox = require({json.dumps(str(OUTBOX))});
|
||||
const values=new Map();
|
||||
const storage={{getItem:k=>values.get(k)||null,setItem:(k,v)=>values.set(k,v)}};
|
||||
values.set('stackchain.issue-outbox.v1',JSON.stringify({{version:2,items:[
|
||||
{{id:'sent',operationId:'sent',ownerLogin:'timmy',status:'queued',title:'Done'}},
|
||||
{{id:'bad',operationId:'bad',ownerLogin:'timmy',status:'queued',title:'Fix me'}},
|
||||
]}}));
|
||||
const outbox=createIssueOutbox({{storage}});
|
||||
outbox.reconcileBackground([
|
||||
{{id:'sent',status:'sent'}},
|
||||
{{id:'bad',status:'attention',error:'Title is invalid'}},
|
||||
]);
|
||||
process.stdout.write(JSON.stringify(outbox.list()));
|
||||
"""
|
||||
output = run_node(script)
|
||||
|
||||
assert output == [
|
||||
{
|
||||
"id": "bad",
|
||||
"operationId": "bad",
|
||||
"ownerLogin": "timmy",
|
||||
"status": "attention",
|
||||
"title": "Fix me",
|
||||
"error": "Title is invalid",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_mobile_dashboard_queues_offline_captures_and_exposes_outbox_actions():
|
||||
html = await dashboard()
|
||||
|
||||
@@ -13,7 +13,7 @@ def run_worker_scenario(scenario: str) -> dict:
|
||||
const fs = require('fs');
|
||||
const vm = require('vm');
|
||||
const listeners = {{}};
|
||||
const state = {{ added: [], deleted: [], claimed: false, skipped: false, fetches: [], puts: [], failFetch: false, fetchStatus: 200, cachedBody: null }};
|
||||
const state = {{ added: [], deleted: [], claimed: false, skipped: false, fetches: [], puts: [], backgroundFlushes: 0, failFetch: false, fetchStatus: 200, cachedBody: null }};
|
||||
const cache = {{
|
||||
addAll: async urls => {{ state.added = urls; }},
|
||||
match: async request => state.cachedBody === null ? null : new Response(state.cachedBody),
|
||||
@@ -27,7 +27,9 @@ const context = {{
|
||||
addEventListener: (name, handler) => {{ listeners[name] = handler; }},
|
||||
skipWaiting: async () => {{ state.skipped = true; }},
|
||||
clients: {{ claim: async () => {{ state.claimed = true; }} }},
|
||||
__issueSync: {{ flush: async () => {{ state.backgroundFlushes += 1; }} }},
|
||||
}},
|
||||
importScripts: () => {{}},
|
||||
caches: {{
|
||||
open: async () => cache,
|
||||
keys: async () => ['stackchain-dashboard-old', 'another-app-cache'],
|
||||
@@ -53,6 +55,11 @@ async function dispatch(name, request) {{
|
||||
if (pending) await pending;
|
||||
return response ? await response : null;
|
||||
}}
|
||||
async function dispatchSync(tag) {{
|
||||
let pending;
|
||||
listeners.sync({{ tag, waitUntil: promise => {{ pending = promise; }} }});
|
||||
if (pending) await pending;
|
||||
}}
|
||||
(async () => {{
|
||||
{scenario}
|
||||
}})().catch(error => {{ console.error(error); process.exit(1); }});
|
||||
@@ -63,10 +70,22 @@ async function dispatch(name, request) {{
|
||||
return json.loads(completed.stdout)
|
||||
|
||||
|
||||
def test_mobile_attention_queue_ships_in_a_new_shell_cache():
|
||||
def test_background_issue_sync_ships_in_a_new_shell_cache():
|
||||
source = WORKER.read_text()
|
||||
|
||||
assert "stackchain-dashboard-shell-v11" in source
|
||||
assert "stackchain-dashboard-shell-v12" in source
|
||||
|
||||
|
||||
def test_background_sync_event_flushes_closed_app_issue_outbox_only_for_its_tag():
|
||||
result = run_worker_scenario(
|
||||
"""
|
||||
await dispatchSync('stackchain-issue-outbox-v1');
|
||||
await dispatchSync('another-app-sync');
|
||||
process.stdout.write(JSON.stringify(state));
|
||||
"""
|
||||
)
|
||||
|
||||
assert result["backgroundFlushes"] == 1
|
||||
|
||||
|
||||
def test_install_precaches_complete_subpath_scoped_app_shell():
|
||||
@@ -103,6 +122,7 @@ def test_install_precaches_complete_subpath_scoped_app_shell():
|
||||
"/dashboard/static/work-route.js",
|
||||
"/dashboard/static/context-poller.js",
|
||||
"/dashboard/static/mobile-task-dock.js",
|
||||
"/dashboard/static/background-issue-sync.js",
|
||||
}
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user