Timmy_Foundation/the-nexus
15
1
Fork 2
Code Issues 80 Pull Requests 71 Actions Packages Projects Releases Wiki Activity
Files
3518f0f2a2b9f818190ec845fe497d2700a9d90e
the-nexus/bin/__pycache__/nexus_watchdog.cpython-312.pyc

218 lines
23 KiB
Plaintext
Raw Normal View History

feat: Evennia-Nexus Bridge v2 — Live Event Streaming (#804) (#807) Co-authored-by: Allegro <allegro@hermes.local> Co-committed-by: Allegro <allegro@hermes.local>
2026-04-04 01:39:38 +00:00
<EFBFBD>
<14><>i<EFBFBD>J<00><01><><00>dZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl
Z
ddl Z ddl m Z mZddlmZddlmZmZmZmZej,ej.dd<08> <09>ej0d
<EFBFBD>Zd Zd Zej8<00>d z dz ZdZdZej@jCdd<12>Z"ej@jCdd<14>Z#ej@jCdd<16>Z$dZ%dZ&e Gd<19>d<1A><00>Z'e Gd<1B>d<1C><00>Z(eefd+d<1D>Z)d,d<1E>Z*eef d-d<1F>Z+d,d <20>Z,d.d/d!<21>Z-d0d"<22>Z.d1d#<23>Z/d2d$<24>Z0d3d%<25>Z1eeeef d4d&<26>Z2d5d6d'<27>Z3d7d(<28>Z4d)<29>Z5e6d*k(re5<65>yy)8u
Nexus Watchdog — The Eye That Never Sleeps
Monitors the health of the Nexus consciousness loop and WebSocket
gateway, raising Gitea issues when components go dark.
The nexus was dead for hours after a syntax error crippled
nexus_think.py. Nobody knew. The gateway kept running, but the
consciousness loop — the only part that matters — was silent.
This watchdog ensures that never happens again.
HOW IT WORKS
============
1. Probes the WebSocket gateway (ws://localhost:8765)
→ Can Timmy hear the world?
2. Checks for a running nexus_think.py process
→ Is Timmy's mind awake?
3. Reads the heartbeat file (~/.nexus/heartbeat.json)
→ When did Timmy last think?
4. If any check fails, opens a Gitea issue (or updates an existing one)
with the exact failure mode, timestamp, and diagnostic info.
5. If all checks pass after a previous failure, closes the issue
with a recovery note.
USAGE
=====
# One-shot check (good for cron)
python bin/nexus_watchdog.py
# Continuous monitoring (every 60s)
python bin/nexus_watchdog.py --watch --interval 60
# Dry-run (print diagnostics, don't touch Gitea)
python bin/nexus_watchdog.py --dry-run
# Crontab entry (every 5 minutes)
*/5 * * * * cd /path/to/the-nexus && python bin/nexus_watchdog.py
HEARTBEAT PROTOCOL
==================
The consciousness loop (nexus_think.py) writes a heartbeat file
after each think cycle:
~/.nexus/heartbeat.json
{
"pid": 12345,
"timestamp": 1711843200.0,
"cycle": 42,
"model": "timmy:v0.1-q4",
"status": "thinking"
}
If the heartbeat is older than --stale-threshold seconds, the
mind is considered dead even if the process is still running
(e.g., hung on a blocking call).
ZERO DEPENDENCIES
=================
Pure stdlib. No pip installs. Same machine as the nexus.
<EFBFBD>)<01> annotationsN)<02> dataclass<73>field)<01>Path)<04>Any<6E>Dict<63>List<73>Optionalz)%(asctime)s %(levelname)-7s %(message)sz%Y-%m-%d %H:%M:%S)<03>level<65>format<61>datefmtznexus.watchdog<6F> localhosti="z.nexuszheartbeat.jsoni,<00><<00> GITEA_URLzhttp://143.198.27.163:3000<30> GITEA_TOKEN<45><00>
NEXUS_REPOzTimmy_Foundation/the-nexus<75>watchdogz
[watchdog]c<01>L<00>eZdZUdZded<ded<ded<ee<06><07>Zded <y
) <0B> CheckResultz Result of a single health check.<2E>str<74>name<6D>bool<6F>healthy<68>message)<01>default_factoryzDict[str, Any]<5D>detailsN)<08>__name__<5F>
__module__<EFBFBD> __qualname__<5F>__doc__<5F>__annotations__r<00>dictr<00><00><00>bin/nexus_watchdog.pyrrks#<00><00>*<2A>
<0A>I<EFBFBD> <11>M<EFBFBD> <10>L<EFBFBD>#<23>D<EFBFBD>9<>G<EFBFBD>^<5E>9r%rc<01>T<00>eZdZUdZded<ded<dZded<d <09>Zed d
<EFBFBD><04>Zdd <0B>Z y )<0F> HealthReportz(Aggregate health report from all checks.<2E>float<61> timestamp<6D>List[CheckResult]<5D>checksTr<00>overall_healthyc<01>F<00>td<01>|jD<00><00>|_y)Nc3<01>4K<00>|]}|j<00><01><00>y<00>w<01>N)r<00><02>.0<EFBFBD>cs r&<00> <genexpr>z-HealthReport.__post_init__.<locals>.<genexpr>|s<00><00><><00>"B<><11>1<EFBFBD>9<EFBFBD>9<EFBFBD>"B<><42><00>)<03>allr,r-)<01>selfs r&<00> __post_init__zHealthReport.__post_init__{s<00><00>"<22>"B<>d<EFBFBD>k<EFBFBD>k<EFBFBD>"B<>B<><04>r%c<01>Z<00>|jD<00>cgc]}|jr<01>|<01><02>c}Scc}wr0)r,r)r7r3s r&<00> failed_checkszHealthReport.failed_checks~s <00><00><1F>;<3B>;<3B>8<>a<EFBFBD>a<EFBFBD>i<EFBFBD>i<EFBFBD><01>8<>8<><38>8s<00>(<04>(c
<01>^<00>tjdtj|j<00><00>}|jrdnd}d|<01><00>d|<02><00>dddg}|j
D]A}|j rd nd
}|jd |j<00>d |<05>d |j<00>d <0A><07><00>C|jr<>|jd<06>|jd<0E>|jD]<5D>}|jd|j<00>d<10><03>|jd<11>|j|j<00>|jr0|jtj|jd<12><13><00>|jd<11><00><>|jd<06>|jd|<01>d<15><03>dj|<03>S)zFormat as a Gitea issue body.z%Y-%m-%d %H:%M:%S UTCu🟢 ALL SYSTEMS OPERATIONALu🔴 FAILURES DETECTEDu## Nexus Health Report — z **Status:** rz| Check | Status | Details |z|:------|:------:|:--------|<7C>✅<>❌z| z | z |z### Failure Diagnosticsz
**z:**z```<60><00><01>indentz%*Generated by `nexus_watchdog.py` at <20>*<2A>
)<0F>time<6D>strftime<6D>gmtimer*r-r,r<00>appendrrr:r<00>json<6F>dumps<70>join)r7<00>ts<74>status<75>linesr3<00>icons r&<00> to_markdownzHealthReport.to_markdown<77>ss<00><00> <11>]<5D>]<5D>2<>D<EFBFBD>K<EFBFBD>K<EFBFBD><04><0E><0E>4O<34> P<><02>37<33>3G<33>3G<33>/<2F>Me<4D><06>*<2A>"<22><14> .<2E><1A>6<EFBFBD>(<28> #<23> <0E> *<2A> *<2A> 
<EFBFBD><05><16><1B><1B> A<01>A<EFBFBD><1D>I<EFBFBD>I<EFBFBD>5<EFBFBD>5<EFBFBD>D<EFBFBD> <11>L<EFBFBD>L<EFBFBD>2<EFBFBD>a<EFBFBD>f<EFBFBD>f<EFBFBD>X<EFBFBD>S<EFBFBD><14><06>c<EFBFBD>!<21>)<29>)<29><1B>B<EFBFBD>?<3F> @<40> A<01> <10> <1D> <1D> <11>L<EFBFBD>L<EFBFBD><12> <1C> <11>L<EFBFBD>L<EFBFBD>2<> 3<><19>'<27>'<27> %<25><01><15> <0C> <0C>t<EFBFBD>A<EFBFBD>F<EFBFBD>F<EFBFBD>8<EFBFBD>3<EFBFBD>/<2F>0<><15> <0C> <0C>s<EFBFBD>$<24><15> <0C> <0C>Q<EFBFBD>Y<EFBFBD>Y<EFBFBD>'<27><14>9<EFBFBD>9<EFBFBD><19>L<EFBFBD>L<EFBFBD><14><1A><1A>A<EFBFBD>I<EFBFBD>I<EFBFBD>a<EFBFBD>!@<40>A<><15> <0C> <0C>s<EFBFBD>$<24>  %<25> <0E> <0C> <0C>R<EFBFBD><18> <0A> <0C> <0C><<3C>R<EFBFBD>D<EFBFBD><01>B<>C<><13>y<EFBFBD>y<EFBFBD><15><1F>r%N)<02>returnr+)rOr)
rrr r!r"r-r8<00>propertyr:rNr$r%r&r(r(ts:<00><00>2<><14><14> <1D><1D> <20>O<EFBFBD>T<EFBFBD> <20>C<01><0E>9<><0E>9<> r%r(c <01><><00> tjtjtj<00>}|jd<01>|j ||f<02>}|j <00>|dk(rt ddd|<00>d|<01><00><04><07>St ddd |<00>d|<01>d
|<03>d <0B>|||d <0C><03> <0A>S#t$r)}t ddd|<04><00>||t|<04>d<0F><03> <0A>cYd}~Sd}~wwxYw)z<>Check if the WebSocket gateway is accepting connections.
Uses a raw TCP socket probe (not a full WebSocket handshake) to avoid
depending on the websockets library. If TCP connects, the gateway
process is alive and listening.
<20>rzWebSocket GatewayTz Listening on <20>:<3A>rrrFzConnection refused on z (errno=<3D>))<03>host<73>port<72>errno<6E>rrrrzProbe failed: )rVrW<00>errorN) <09>socket<65>AF_INET<45> SOCK_STREAM<41>
settimeout<EFBFBD>
connect_ex<EFBFBD>closer<00> Exceptionr)rVrW<00>sock<63>result<6C>es r&<00>check_ws_gatewayre<00>s<><00><00>
<EFBFBD><15>}<7D>}<7D>V<EFBFBD>^<5E>^<5E>V<EFBFBD>-?<3F>-?<3F>@<40><04> <0C><0F><0F><01><1A><15><1F><1F>$<24><04><1C>.<2E><06> <0C>
<EFBFBD>
<EFBFBD> <0C> <11>Q<EFBFBD>;<3B><1E>(<28><1C>'<27><04>v<EFBFBD>Q<EFBFBD>t<EFBFBD>f<EFBFBD>5<><0E> <0E> <1F>(<28><1D>0<><14><06>a<EFBFBD><04>v<EFBFBD>X<EFBFBD>f<EFBFBD>X<EFBFBD>Q<EFBFBD>O<>!%<25>t<EFBFBD>f<EFBFBD>E<> <0E> <0E><> <15>
<EFBFBD><1A>$<24><19>$<24>Q<EFBFBD>C<EFBFBD>(<28>!<21>4<EFBFBD>#<23>a<EFBFBD>&<26>A<> 
<EFBFBD>
<EFBFBD><EFBFBD>
<EFBFBD>s$<00>A>B<00>B<00> C<03>'C <03>C<03> Cc
<01><><00> tjgd<01>ddd<03><04>}|jdk(r<>|jj <00>j d<06>D<00>cgc]#}|j <00>s<01>|j <00><00><02>%}}t tj<00><00>}|D<00>cgc]
}||k7s<01> |<01><02> }}|r$tdddd j|<02><00>d
<EFBFBD>d |i<01> <0C>Stdd dd|ji<01> <0C>Scc}wcc}w#t$rtddd<10><11>cYSt$r'}tdd d|<04><00>dt |<04>i<01> <0C>cYd}~Sd}~wwxYw)z<>Check if nexus_think.py is running as a process.
Uses `pgrep -f` to find processes matching the script name.
This catches both `python nexus_think.py` and `python -m nexus.nexus_think`.
)<03>pgrepz-f<> nexus_thinkTrR)<03>capture_output<75>text<78>timeoutrrBzConsciousness LoopzRunning (PID: <20>, rU<00>pidsrYFu6nexus_think.py is not running — Timmy's mind is dark<72>pgrep_returncodez+pgrep not available, skipping process checkrTzProcess check failed: rZN) <0A>
subprocess<EFBFBD>run<75>
returncode<EFBFBD>stdout<75>strip<69>splitr<00>os<6F>getpidrrI<00>FileNotFoundErrorra)rc<00>prm<00>own_pidrds r&<00>check_mind_processrz<00>sO<00><00> '
<EFBFBD><1B><1E><1E> *<2A><1F>d<EFBFBD>A<EFBFBD>
<EFBFBD><06>
<12> <1C> <1C><01> !<21>'-<2D>}<7D>}<7D>':<3A>':<3A>'<<3C>'B<>'B<>4<EFBFBD>'H<>V<>!<21>A<EFBFBD>G<EFBFBD>G<EFBFBD>I<EFBFBD>A<EFBFBD>G<EFBFBD>G<EFBFBD>I<EFBFBD>V<>D<EFBFBD>V<><19>"<22>)<29>)<29>+<2B>&<26>G<EFBFBD>#<23>4<>!<21>q<EFBFBD>G<EFBFBD>|<7C>A<EFBFBD>4<>D<EFBFBD>4<><13>"<22>-<2D> <20>,<2C>T<EFBFBD>Y<EFBFBD>Y<EFBFBD>t<EFBFBD>_<EFBFBD>,=<3D>Q<EFBFBD>?<3F>#<23>T<EFBFBD>N<EFBFBD> <12><12><1B>%<25><19>L<>'<27><16>):<3A>):<3A>;<3B> 
<EFBFBD>
<EFBFBD><EFBFBD>W<01><>5<><35> <1D>
<EFBFBD><1A>%<25><18>A<>
<EFBFBD>
<EFBFBD>
<15>
<EFBFBD><1A>%<25><19>,<2C>Q<EFBFBD>C<EFBFBD>0<><1C>c<EFBFBD>!<21>f<EFBFBD>%<25> 
<EFBFBD>
<EFBFBD><EFBFBD>
<EFBFBD>sT<00>AC><00>C4<04>-C4<04>?#C><00>"
C9<04>-C9<04>1'C><00>C><00>4
C><00>>E<03>E<03>E<03>;E<03>Ec<01><><00>|j<00>stddd|<00>d<04>dt|<00>i<01><06>S tj|j <00><00>}|jd
d <0B>}tj<00>|z
}|jd d <0A>}|jdd<0F>}|jdd<0F>}||kDr'tdddt|<05><00>d|<01>d|<06>d|<07>d|<08><00>
|<02><06>Stddd|<06>dt|<05><00>d|<07><00>|<02><06>S#tj tf$r1}tddd|<03><00>t|<00>t|<03>d<08><02><06>cYd }~Sd }~wwxYw)z<>Check if the heartbeat file exists and is recent.
The consciousness loop should write this file after each think
cycle. If it's missing or stale, the mind has stopped thinking
even if the process is technically alive.
<20> HeartbeatFzNo heartbeat file at u — mind has never reported<65>pathrYzHeartbeat file corrupt: )r}rZNr*r<00>cycle<6C>?<3F>model<65>unknownrKuStale heartbeat — last pulse zs ago (threshold: z s). Cycle #z, model=z , status=TuAlive — cycle #rlz s ago, model=) <0B>existsrrrG<00>loads<64> read_text<78>JSONDecodeError<6F>OSError<6F>getrC<00>int) r}<00>stale_threshold<6C>datardr*<00>ager~r<>rKs r&<00>check_heartbeatr<74><00>ss<00><00> <10>;<3B>;<3B>=<3D><1A><1C><19>+<2B>D<EFBFBD>6<EFBFBD>1M<31>N<><1B>S<EFBFBD><14>Y<EFBFBD>'<27> 
<EFBFBD>
<EFBFBD>
<EFBFBD><13>z<EFBFBD>z<EFBFBD>$<24>.<2E>.<2E>*<2A>+<2B><04><15><08><08><1B>a<EFBFBD>(<28>I<EFBFBD>
<0E>)<29>)<29>+<2B> <09>
!<21>C<EFBFBD> <10>H<EFBFBD>H<EFBFBD>W<EFBFBD>c<EFBFBD> "<22>E<EFBFBD> <10>H<EFBFBD>H<EFBFBD>W<EFBFBD>i<EFBFBD> (<28>E<EFBFBD> <11>X<EFBFBD>X<EFBFBD>h<EFBFBD> <09> *<2A>F<EFBFBD>
<EFBFBD>_<EFBFBD><1C><1A><1C><19>1<>#<23>c<EFBFBD>(<28><1A><<1F>.<2E>/<2F>0<1A><1F><17><08><15><07>y<EFBFBD><16><08>B<01><19> 
<EFBFBD>
<EFBFBD> <17> <18><14>#<23>E<EFBFBD>7<EFBFBD>"<22>S<EFBFBD><13>X<EFBFBD>J<EFBFBD>m<EFBFBD>E<EFBFBD>7<EFBFBD>K<><14>  <06><06><>5 <11> <20> <20>'<27> *<2A>
<EFBFBD><1A><1C><19>.<2E>q<EFBFBD>c<EFBFBD>2<> <20><14>Y<EFBFBD><13>Q<EFBFBD><16>8<> 
<EFBFBD>
<EFBFBD><EFBFBD>
<EFBFBD>s<00>#C?<00>?E <03>&E<03>>E <03>E c <01><00>tt<00>jjdz dz }|j<00>st ddd<05><06>S |j <00>}t |t|<00>d<07>t dddt|<01><00>d <09><03><06>S#t$rq}t dd
d |j<00>d |j<00><00>t|<00>|j|j|jxsd j<00>d<0E><04><0F>cYd}~Sd}~wwxYw)a Verify nexus_think.py can be parsed by Python.
This catches the exact failure mode that killed the nexus: a syntax
error introduced by a bad commit. Python's compile() is a fast,
zero-import check that catches SyntaxErrors before they hit runtime.
<20>nexusznexus_think.pyz Syntax HealthTz3nexus_think.py not found at expected path, skippingrT<00>execz!nexus_think.py compiles cleanly (z bytes)FzSyntaxError at line z: r)<04>file<6C>line<6E>offsetrjrYN)r<00>__file__<5F>parentr<74>rr<><00>compiler<00>len<65> SyntaxError<6F>lineno<6E>msgr<67>rjrs)<03> script_path<74>sourcerds r&<00>check_syntax_healthr<68>.s<><00><00><17>x<EFBFBD>.<2E>'<27>'<27>.<2E>.<2E><17>8<>;K<>K<>K<EFBFBD> <16> <1D> <1D> <1F><1A> <20><18>I<>
<EFBFBD>
<EFBFBD> 
<EFBFBD><1C>&<26>&<26>(<28><06><0F><06><03>K<EFBFBD>(<28>&<26>1<><1A> <20><18>7<><03>F<EFBFBD> <0B>}<7D>G<EFBFBD>L<>
<EFBFBD>
<EFBFBD><EFBFBD>
<17> 
<EFBFBD><1A> <20><19>*<2A>1<EFBFBD>8<EFBFBD>8<EFBFBD>*<2A>B<EFBFBD>q<EFBFBD>u<EFBFBD>u<EFBFBD>g<EFBFBD>><3E><1B>K<EFBFBD>(<28><19><08><08><1B>(<28>(<28><1A><16><16><1C>2<EFBFBD>,<2C>,<2C>.<2E> <0E>

<EFBFBD>
<EFBFBD><EFBFBD> 
<EFBFBD>s <00> AB
<00>
D<03>A&C?<03>9D<03>?Dc<01>b<00>ddl}ddl}tjd<03><00>d|<01><00>}|r#t j
|<02>j <00>nd}|jj|||<00><05>}tr|jddt<00><00><02>|jdd <09>|jd
d <09> |jj|d <0B> <0C>5}|j<00>j<00>}|j<00>rt j|<08>nicddd<02>S#1swYyxYw#|j j"$rJ} t$j'd | j(| j<00>j<00>dd<00>Yd} ~ yd} ~ wt*$r } t$j'd| <09>Yd} ~ yd} ~ wwxYw)z<Make a Gitea API request. Returns parsed JSON or empty dict.rN<>/z/api/v1)r<><00>method<6F> Authorizationztoken z Content-Typezapplication/json<6F>Accept<70>)rkz Gitea %d: %s<><73>zGitea request failed: %s)<16>urllib.request<73> urllib.errorr<00>rstriprGrH<00>encode<64>request<73>Requestr<00>
add_header<EFBFBD>urlopen<65>read<61>decodersr<>rZ<00> HTTPError<6F>logger<65>warning<6E>codera)
r<EFBFBD>r}r<><00>urllib<69>url<72>body<64>req<65>resp<73>rawrds
r&<00>_gitea_requestr<74>UsM<00><00><19><17> <16> <1D> <1D>c<EFBFBD> "<22> #<23>7<EFBFBD>4<EFBFBD>&<26>
1<EFBFBD>C<EFBFBD>(,<2C>4<EFBFBD>:<3A>:<3A>d<EFBFBD> <1B> "<22> "<22> $<24>$<24>D<EFBFBD>
<10>.<2E>.<2E>
<20>
<20><13>4<EFBFBD><06>
<20>
?<3F>C<EFBFBD><12> <0B><0E><0E><EFBFBD>&<26><1B> <0A>(><3E>?<3F><07>N<EFBFBD>N<EFBFBD>><3E>#5<>6<><07>N<EFBFBD>N<EFBFBD>8<EFBFBD>/<2F>0<> <14> <13>^<5E>^<5E> #<23> #<23>C<EFBFBD><12> #<23> 4<> :<3A><04><16>)<29>)<29>+<2B>$<24>$<24>&<26>C<EFBFBD>&)<29>i<EFBFBD>i<EFBFBD>k<EFBFBD>4<EFBFBD>:<3A>:<3A>c<EFBFBD>?<3F>r<EFBFBD> :<3A> :<3A> :<3A><> <12><<3C><<3C> !<21> !<21><14><0E><0E><0E>~<7E>q<EFBFBD>v<EFBFBD>v<EFBFBD>q<EFBFBD>v<EFBFBD>v<EFBFBD>x<EFBFBD><EFBFBD><EFBFBD>/@<40><14>#<23>/F<>G<><13><> <14><14><0E><0E><0E>1<>1<EFBFBD>5<><13><><14>sD<00>,D$<00> AD<03> D$<00>D!<07>D$<00>!D$<00>$F.<03>=AF<03> F.<03>F)<03>)F.c<01><><00>tddt<00>d<03><03>}|rt|t<00>sy|D]-}|j dd<06>}|j t <00>s<01>+|cSy)z-Find an existing open watchdog issue, if any.<2E>GET<45>/repos/z'/issues?state=open&type=issues&limit=20N<30>titler)r<><00>
GITEA_REPO<EFBFBD>
isinstance<EFBFBD>listr<74><00>
startswith<EFBFBD>WATCHDOG_TITLE_PREFIX)<03>issues<65>issuer<65>s r&<00>find_open_watchdog_issuer<65>nse<00><00> <1B> <0A>
<11>*<2A><1C>D<>E<><06>F<EFBFBD> <12><1A>F<EFBFBD>D<EFBFBD>1<><13><17><19><05><15> <09> <09>'<27>2<EFBFBD>&<26><05> <10> <1B> <1B>1<> 2<><18>L<EFBFBD><19> r%c<01><><00>|j}djd<02>|D<00><00>}t<00>d|<02><00>}tddt<00>d<06>||j <00>dgd<08><03> <09>S)
z*Create a Gitea issue for a health failure.rlc3<01>4K<00>|]}|j<00><01><00>y<00>wr0)rr1s r&r4z%create_alert_issue.<locals>.<genexpr><3E>s<00><00><><00>2<>a<EFBFBD>1<EFBFBD>6<EFBFBD>6<EFBFBD>2<>r5z Nexus health failure: <20>POSTr<54>z/issues<65>Timmy)r<>r<><00> assignees<65>r<>)r:rIr<>r<>r<>rN)<04>report<72>failed<65>
componentsr<EFBFBD>s r&<00>create_alert_issuer<65>~sh<00><00> <13> !<21> !<21>F<EFBFBD><15><19><19>2<>6<EFBFBD>2<>2<>J<EFBFBD>$<24>%<25>%<<3C>Z<EFBFBD>L<EFBFBD> I<>E<EFBFBD> <19><0E>
<11>*<2A><1C>W<EFBFBD>%<25><1A><1A>&<26>&<26>(<28>!<21><19>
<EFBFBD> <06>r%c<01>T<00>tddt<00>d|<00>d<04>d|j<00>i<01><06>S)z>Add a comment to an existing watchdog issue with new findings.r<>r<><00>/issues/<2F> /commentsr<73>r<><00>r<>r<>rN<00><02> issue_numberr<72>s r&<00>update_alert_issuer<65><00>s5<00><00> <19><0E>
<11>*<2A><1C>X<EFBFBD>l<EFBFBD>^<5E>9<EFBFBD>=<3D><14>f<EFBFBD>(<28>(<28>*<2A> +<2B> <06>r%c<01><><00>tddt<00>d|<00>d<04>dd|j<00>zdzi<01><08>td dt<00>d|<00><00>d
d i<01><08>y ) z/Close a watchdog issue when health is restored.r<>r<>r<>r<>r<>u## 🟢 Recovery Confirmed
u(
*Closing — all systems operational.*r<><00>PATCH<43>state<74>closedNr<4E>r<>s r&<00>close_alert_issuer<65><00>sh<00><00><12><0E>
<11>*<2A><1C>X<EFBFBD>l<EFBFBD>^<5E>9<EFBFBD>=<3D><14> ,<2C><14> <20> <20>"<22> #<23>:<3A> ;<3B> <0B><06><13><0F>
<11>*<2A><1C>X<EFBFBD>l<EFBFBD>^<5E>4<><15>x<EFBFBD> <20>r%c<01><><00>t||<01>t<00>t||<03>t<00>g}t t j
<00>|<04><01>S)z6Run all health checks and return the aggregate report.)r*r,)rerzr<>r<>r(rC)<05>ws_host<73>ws_port<72>heartbeat_pathr<68>r,s r&<00>run_health_checksr<73><00>s@<00><00> <19><17>'<27>*<2A><1A><1C><17><0E><0F>8<><1B><1D> <06>F<EFBFBD> <18>$<24>)<29>)<29>+<2B>f<EFBFBD> =<3D>=r%c<01><><00>|r*tjd|jrd<02>yd<03>ytstj d<05>yt <00>}|jr,|r)tjd|d<00>t |d|<00>yy|r)tjd|d<00>t|d|<00>yt|<00>}|r,|jd<07>rtjd |d<00>yyy)
z=Create, update, or close Gitea issues based on health status.u DRY RUN — would %s Gitea issuer`z create/updateNu,GITEA_TOKEN not set — cannot create issuesu%Health restored — closing issue #%d<>numberu&Still unhealthy — updating issue #%dzCreated alert issue #%d)
r<EFBFBD><00>infor-rr<>r<>r<>r<>r<>r<>)r<><00>dry_run<75>existingrcs r&<00>alert_on_failurer<65><00>s<><00><00><0E><0E> <0B> <0B>6<> &<26> 6<> 6<>W<EFBFBD> M<01><0E>=L<01> M<01><0E> <16><0E><0E><0E>E<>F<><0E>'<27>)<29>H<EFBFBD> <0A><1D><1D> <13> <12>K<EFBFBD>K<EFBFBD>?<3F><18>(<28>AS<41> T<> <1D>h<EFBFBD>x<EFBFBD>0<>&<26> 9<> <14> <14> <12>K<EFBFBD>K<EFBFBD>@<40>(<28>8<EFBFBD>BT<42> U<> <1E>x<EFBFBD><08>1<>6<EFBFBD> :<3A>'<27><06>/<2F>F<EFBFBD><15>&<26>*<2A>*<2A>X<EFBFBD>.<2E><16> <0B> <0B>5<>v<EFBFBD>h<EFBFBD>7G<37>H<>/<2F>vr%c<01>2<00>t|j|jt|j<00>|j
<00><01>}|j D]k}|jrtjntj}|jrdnd}tj|d||j|j<00><00>m|js#t!||j"<00><05>|jS|j"st!||j"<00><05>|jS)z4Run one health check cycle. Returns True if healthy.<2E>r<>r<>r<>r<>r<r=z %s %s: %s)r<>)r<>r<>r<>rr<>r<>r,r<00>logging<6E>INFO<46>ERRORr<52><00>logrrr-r<>r<>)<05>argsr<73><00>checkr rMs r&<00>run_oncer<65><00>s<><00><00> <1E><14> <0C> <0C><14> <0C> <0C><1B>D<EFBFBD>/<2F>/<2F>0<><1C>,<2C>,<2C> <06>F<EFBFBD><18><1D><1D>H<01><05> %<25> <0A> <0A><07> <0C> <0C>7<EFBFBD>=<3D>=<3D><05><1D> <0A> <0A>u<EFBFBD>5<EFBFBD><04><0E>
<EFBFBD>
<EFBFBD>5<EFBFBD>+<2B>t<EFBFBD>U<EFBFBD>Z<EFBFBD>Z<EFBFBD><15><1D><1D>G<>H<01>
<12> !<21> !<21><18><16><14><1C><1C>6<> <12> !<21> !<21>!<21><12>\<5C>\<5C><18><16><14><1C><1C>6<> <11> !<21> !<21>!r%c<01>P<00><07>tjd<01><02>}|jdtd<04><05>|jdtt
d<07><08>|jd t t<00>d
<EFBFBD><05>|jd ttd <0C><08>|jd dd<0F><10>|jdttd<12><08>|jddd<14><10>|jdddd<17><18>|j<00>}|jr<>tjd|j<00>d<1A><07>fd<1B>}tjtj |<02>tjtj"|<02><00>rBt%|<01>t'|j<00>D]}<03>snt)j*d<1C><00><00>r<01>Ayyt%|<01>}|j,r<>t/|j0|j2t5|j6<00>|j8<00><1D>}t;t=j>|j@|jB|jDD<00>cgc]1}|jF|jH|jJ|jLd<1E><04><02>3c}d<1F>d <20>!<21><00>tOjP|rd"nd<1C>ycc}w)#Nu5Nexus Watchdog — monitors consciousness loop health)<01> descriptionz --ws-hostz+WebSocket gateway host (default: localhost))<02>default<6C>helpz --ws-portz&WebSocket gateway port (default: 8765))<03>typer<65>r<>z--heartbeat-pathzPath to heartbeat filez--stale-thresholdz;Seconds before heartbeat is considered stale (default: 300)z--watch<63>
store_truez$Run continuously instead of one-shot)<02>actionr<6E>z
--intervalz2Seconds between checks in watch mode (default: 60)z --dry-runz/Print diagnostics without creating Gitea issuesz--json<6F> output_jsonz9Output results as JSON (for integration with other tools))r<><00>destr<74>z4Watchdog starting in continuous mode (interval: %ds)Tc<01>6<00><01>d<01>tjd|<00>y)NFz!Received signal %d, shutting down)r<>r<>)<03>signum<75>frame<6D>_runnings <20>r&<00>_handle_sigtermzmain.<locals>._handle_sigterms<00><><00><1C>H<EFBFBD> <12>K<EFBFBD>K<EFBFBD>;<3B>V<EFBFBD> Dr%<00>r<>rY)rr*r,r>r?r))<29>argparse<73>ArgumentParser<65> add_argument<6E>DEFAULT_WS_HOSTr<54><00>DEFAULT_WS_PORTr<00>DEFAULT_HEARTBEAT_PATH<54>DEFAULT_STALE_THRESHOLD<4C>DEFAULT_INTERVAL<41>
parse_args<EFBFBD>watchr<68>r<><00>interval<61>signal<61>SIGTERM<52>SIGINTr<54><00>rangerC<00>sleepr<70>r<>r<>r<>rr<>r<><00>printrGrHr-r*r,rrrr<00>sys<79>exit)<08>parserr<72>r<><00>_rr<>r3r<>s @r&<00>mainr<00>sg<00><><00> <15> $<24> $<24>K<><06>F<EFBFBD> <0B><17><17><13>_<EFBFBD> :<3A><18><06> <0B><17><17><13>#<23><EFBFBD> 5<><18><06> <0B><17><17><1A>C<EFBFBD>(><3E>$?<3F> %<25><18><06> <0B><17><17><1B>#<23>/F<> J<><18><06> <0B><17><17><11>,<2C> 3<><18><06> <0B><17><17><14>3<EFBFBD>(8<> A<><18><06> <0B><17><17><13>L<EFBFBD> ><3E><18><06> <0B><17><17><10><1C>M<EFBFBD> H<><18><06>
<12> <1C> <1C> <1E>D<EFBFBD> <0B>z<EFBFBD>z<EFBFBD><0E> <0B> <0B>J<>D<EFBFBD>M<EFBFBD>M<EFBFBD>Z<><17><08> E<01>
<0F> <0A> <0A>f<EFBFBD>n<EFBFBD>n<EFBFBD>o<EFBFBD>6<><0E> <0A> <0A>f<EFBFBD>m<EFBFBD>m<EFBFBD>_<EFBFBD>5<><16> <14>T<EFBFBD>N<EFBFBD><1A>4<EFBFBD>=<3D>=<3D>)<29> <1E><01><1F><19><14>
<EFBFBD>
<EFBFBD>1<EFBFBD> <0A> <1E><17><1B>4<EFBFBD>.<2E><07> <0F> <1B> <1B>&<26><1C> <0C> <0C><1C> <0C> <0C>#<23>D<EFBFBD>$7<>$7<>8<> $<24> 4<> 4<> <0E>F<EFBFBD> <12>$<24>*<2A>*<2A>!<21>1<>1<>#<23>-<2D>-<2D>$<24>]<5D>]<5D><12><1A><1F>V<EFBFBD>V<EFBFBD><01> <09> <09> !<21> <09> <09>a<EFBFBD>i<EFBFBD>i<EFBFBD>A<01><12><0E><18><19> <1A> <0C><08><08>g<EFBFBD><11>1<EFBFBD>%<25><>s<00>6J#<10>__main__)rVrrWr<>rOr)rOr)r}rr<>r<>rOrr0)r<>rr}rr<><00>Optional[dict]rOr)rOr)r<>r(rOr)r<>r<>r<>r(rOr)r<>r<>r<>r(rO<00>None)
r<EFBFBD>rr<>r<>r<>rr<>r<>rOr()F)r<>r(r<>rrOr)r<>zargparse.NamespacerOr)7r!<00>
__future__rr<>rGr<>rur r[rorrC<00> dataclassesrr<00>pathlibr<00>typingrrr r
<00> basicConfigr<67><00> getLoggerr<72>rr<00>homerrr<00>environr<6E>rrr<><00>WATCHDOG_LABELr<4C>rr(rerzr<>r<>r<>r<>r<>r<>r<>r<>r<>r<>rrr$r%r&<00><module>r s<><00><01>@<04>D#<23><0F> <0B><0E> <09> <0A> <0A><11>
<EFBFBD> <0B>(<28><18>,<2C>,<2C><13><07><13><13>
<11>,<2C>,<2C> 6<> <1F><02>
<1B><17> <1A> <1A>+<2B> ,<2C><06><1E><0F><16><0F>"<22><14><19><19><1B>x<EFBFBD>/<2F>2B<32>B<><16><1D><17><15><10> <0E>J<EFBFBD>J<EFBFBD>N<EFBFBD>N<EFBFBD>;<3B>(D<> E<> <09><10>j<EFBFBD>j<EFBFBD>n<EFBFBD>n<EFBFBD>]<5D>B<EFBFBD>/<2F> <0B> <0F>Z<EFBFBD>Z<EFBFBD>^<5E>^<5E>L<EFBFBD>*F<> G<>
<EFBFBD><1B><0E>$<24><15>
 <0B>:<3A>:<3A> <0B>:<3A> <0B>+ <20>+ <20> <0B>+ <20>`"1<>o<EFBFBD> 
<EFBFBD>F-
<EFBFBD>b(<28>2<>3<06>
<0E>3<06><18>3<06><11>3<06>l"
<EFBFBD>N<14>2 <10> <06>"<06><06>*#<23>"<22>1<>2<> ><3E> <10> ><3E> <10> ><3E><19> ><3E><19> ><3E>
<12> ><3E> I<01>6"<22>.M&<26>` <0C>z<EFBFBD><19><08>F<EFBFBD>r%
Reference in New Issue Copy Permalink