Compare commits
1 Commits
fix/678
...
step35/874
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
41ac45e49b |
@@ -1,9 +1,7 @@
|
||||
# GENOME.md — timmy-academy
|
||||
|
||||
Refreshed against live repo state on 2026-04-22.
|
||||
Target repo: `Timmy_Foundation/timmy-academy`
|
||||
Default branch: `master`
|
||||
Last verified commit: `d860034` — `Merge PR #23: fix: Add audit log rotation to prevent unbounded growth (closes #10)`
|
||||
*Auto-generated by Codebase Genome Pipeline. 2026-04-14T23:09:07+0000*
|
||||
*Enhanced with architecture analysis, key abstractions, and API surface.*
|
||||
|
||||
## Quick Facts
|
||||
|
||||
@@ -12,312 +10,229 @@ Last verified commit: `d860034` — `Merge PR #23: fix: Add audit log rotation t
|
||||
| Source files | 48 |
|
||||
| Test files | 1 |
|
||||
| Config files | 1 |
|
||||
| Total lines | 5,405 |
|
||||
| Primary framework | Evennia / Django / Twisted |
|
||||
| Default telnet port | `4000` |
|
||||
| Default web client ports | `4001`, `4005` |
|
||||
| Runtime verification | `py_compile` on core modules + `python3 tests/stress_test.py --help` |
|
||||
| Total lines | 5,353 |
|
||||
| Last commit | 395c9f7 Merge PR 'Add @who command' (#7) into master (2026-04-13) |
|
||||
| Branch | master |
|
||||
| Test coverage | 0% (35 untested modules) |
|
||||
|
||||
## Project Overview
|
||||
## What This Is
|
||||
|
||||
`timmy-academy` is Timmy Academy: an Evennia MUD world used for agent convening, operator training, and crisis-response practice. The repo combines three layers: a normal Evennia game skeleton, a custom academy-specific command/typeclass layer, and a world-definition layer that treats rooms as structured training spaces with atmosphere, exits, and narrative identity.
|
||||
Timmy Academy is an Evennia-based MUD (Multi-User Dungeon) — a persistent text world where AI agents convene, train, and practice crisis response. It runs on Bezalel VPS (167.99.126.228) with telnet on port 4000 and web client on port 4001.
|
||||
|
||||
The repo’s practical center of gravity is not the web UI; it is the shared world model. Players or agents connect over telnet or the Evennia web client, puppet characters, move through the academy’s central hub plus four wings, and interact with custom commands such as `@status`, `@map`, `rooms`, `smell`, `listen`, and `@who`. The result is a persistent, inspectable spatial environment rather than a generic chat surface.
|
||||
|
||||
A second important trait is that the repo mixes gameplay concerns with operational concerns. `server/conf/settings.py` enables detailed audit logging. `typeclasses/audited_character.py` records movement and command trails. `world/rebuild_world.py` can rehydrate the academy from source definitions. `tests/stress_test.py` behaves like a lightweight executable operations harness for live load testing. Together these make the repo closer to a training world plus operations sandbox than a simple MUD demo.
|
||||
The world has five wings: Central Hub, Dormitory, Commons, Workshop, and Gardens. Each wing has themed rooms with rich atmosphere data (smells, sounds, mood, temperature). Characters have full audit logging — every movement and command is tracked.
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
TELNET[Telnet clients :4000]
|
||||
WEB[Evennia web client :4001/:4005]
|
||||
PORTAL[Evennia Portal]
|
||||
SERVER[Evennia Server]
|
||||
SETTINGS[server/conf/settings.py]
|
||||
CMDSETS[commands/default_cmdsets.py]
|
||||
COMMANDS[commands/command.py]
|
||||
TYPECLASSES[typeclasses/*]
|
||||
AUDIT[typeclasses/audited_character.py]
|
||||
WORLD[world/*_wing.py]
|
||||
REBUILD[world/rebuild_world.py]
|
||||
BATCH[world/build_academy.ev]
|
||||
WEBURLS[web/urls.py]
|
||||
HERMESCFG[hermes-agent/config.yaml]
|
||||
STRESS[tests/stress_test.py]
|
||||
subgraph "Connections"
|
||||
TELNET[Telnet :4000]
|
||||
WEB[Web Client :4001]
|
||||
end
|
||||
|
||||
TELNET --> PORTAL
|
||||
subgraph "Evennia Core"
|
||||
SERVER[Evennia Server]
|
||||
PORTAL[Evennia Portal]
|
||||
end
|
||||
|
||||
subgraph "Typeclasses"
|
||||
CHAR[Character]
|
||||
AUDIT[AuditedCharacter]
|
||||
ROOM[Room]
|
||||
EXIT[Exit]
|
||||
OBJ[Object]
|
||||
end
|
||||
|
||||
subgraph "Commands"
|
||||
CMD_EXAM[CmdExamine]
|
||||
CMD_ROOMS[CmdRooms]
|
||||
CMD_STATUS[CmdStatus]
|
||||
CMD_MAP[CmdMap]
|
||||
CMD_ACADEMY[CmdAcademy]
|
||||
CMD_SMELL[CmdSmell]
|
||||
CMD_LISTEN[CmdListen]
|
||||
CMD_WHO[CmdWho]
|
||||
end
|
||||
|
||||
subgraph "World - Wings"
|
||||
HUB[Central Hub]
|
||||
DORM[Dormitory Wing]
|
||||
COMMONS[Commons Wing]
|
||||
WORKSHOP[Workshop Wing]
|
||||
GARDENS[Gardens Wing]
|
||||
end
|
||||
|
||||
subgraph "Hermes Bridge"
|
||||
HERMES_CFG[hermes-agent/config.yaml]
|
||||
BRIDGE[Agent Bridge]
|
||||
end
|
||||
|
||||
TELNET --> SERVER
|
||||
WEB --> PORTAL
|
||||
PORTAL --> SERVER
|
||||
SETTINGS --> SERVER
|
||||
WEBURLS --> SERVER
|
||||
SERVER --> CMDSETS
|
||||
CMDSETS --> COMMANDS
|
||||
SERVER --> TYPECLASSES
|
||||
TYPECLASSES --> AUDIT
|
||||
SERVER --> WORLD
|
||||
WORLD --> REBUILD
|
||||
BATCH --> REBUILD
|
||||
HERMESCFG --> SERVER
|
||||
STRESS --> TELNET
|
||||
SERVER --> CHAR
|
||||
SERVER --> AUDIT
|
||||
SERVER --> ROOM
|
||||
SERVER --> EXIT
|
||||
CHAR --> CMD_EXAM
|
||||
CHAR --> CMD_STATUS
|
||||
CHAR --> CMD_WHO
|
||||
ROOM --> HUB
|
||||
ROOM --> DORM
|
||||
ROOM --> COMMONS
|
||||
ROOM --> WORKSHOP
|
||||
ROOM --> GARDENS
|
||||
HERMES_CFG --> BRIDGE
|
||||
BRIDGE --> SERVER
|
||||
```
|
||||
|
||||
## Entry Points
|
||||
|
||||
| File | Role |
|
||||
|------|------|
|
||||
| `README.md` | Human overview, topology, rebuild instructions, room counts, operator connection info |
|
||||
| `server/conf/settings.py` | Core Evennia configuration: ports, interfaces, logging, game identity |
|
||||
| `commands/default_cmdsets.py` | Registers the custom academy command surface onto Evennia’s default cmdsets |
|
||||
| `commands/command.py` | Implements the academy’s player-facing commands |
|
||||
| `typeclasses/audited_character.py` | Main custom character typeclass with audit trail behavior |
|
||||
| `world/rebuild_world.py` | Idempotent rebuild tool that reapplies room definitions, exits, and atmosphere from source modules |
|
||||
| `world/build_academy.ev` | Evennia batch setup entrypoint |
|
||||
| `web/urls.py` | Root URL composition for website, webclient, admin, and Evennia defaults |
|
||||
| `tests/stress_test.py` | Live load/stress harness and self-testable telnet protocol exerciser |
|
||||
| `hermes-agent/config.yaml` | Bridge-side model/provider configuration snapshot for Hermes integration |
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `server/conf/settings.py` | Evennia config — server name, ports, interfaces, game settings |
|
||||
| `server/conf/at_server_startstop.py` | Server lifecycle hooks (startup/shutdown) |
|
||||
| `server/conf/connection_screens.py` | Login/connection screen text |
|
||||
| `commands/default_cmdsets.py` | Registers all custom commands with Evennia |
|
||||
| `world/rebuild_world.py` | Rebuilds all rooms from source |
|
||||
| `world/build_academy.ev` | Evennia batch script for initial world setup |
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. A human or agent connects over telnet (`4000`) or the Evennia web client (`4001` / `4005`).
|
||||
2. The Evennia portal hands the connection to the game server configured by `server/conf/settings.py`.
|
||||
3. Once an account puppets a character, the command path is controlled by `commands/default_cmdsets.py`, which mounts the academy-specific commands from `commands/command.py`.
|
||||
4. The typeclass layer (`typeclasses/*`) determines how characters, rooms, exits, channels, and scripts behave; `AuditedCharacter` wraps command and movement hooks in persistent logging.
|
||||
5. The world layer (`world/*_wing.py`) supplies canonical room descriptions, exits, aliases, atmosphere, and thematic metadata.
|
||||
6. `world/rebuild_world.py` parses those source files and writes them back into Evennia objects, making source the effective truth for the academy layout.
|
||||
7. `tests/stress_test.py` simulates concurrent clients against the live telnet surface and reports throughput, latency, and connection statistics.
|
||||
```
|
||||
Player connects (telnet/web)
|
||||
-> Evennia Portal accepts connection
|
||||
-> Server authenticates (Account typeclass)
|
||||
-> Player puppets a Character
|
||||
-> Character enters world (Room typeclass)
|
||||
-> Commands processed through Command typeclass
|
||||
-> AuditedCharacter logs every action
|
||||
-> World responds with rich text + atmosphere data
|
||||
```
|
||||
|
||||
## Key Abstractions
|
||||
|
||||
### 1. `AuditedCharacter`
|
||||
File: `typeclasses/audited_character.py`
|
||||
### Typeclasses (the world model)
|
||||
|
||||
This is the repo’s flagship abstraction. It extends `DefaultCharacter` with:
|
||||
- per-session audit logging
|
||||
- movement logging via `at_pre_move()` / `at_post_move()`
|
||||
- command tracking via `at_pre_cmd()`
|
||||
- session timing via puppet / unpuppet hooks
|
||||
- rotated in-db history (`location_history`)
|
||||
- summarized audit snapshots via `get_audit_summary()`
|
||||
| Class | File | Purpose |
|
||||
|-------|------|---------|
|
||||
| `Character` | `typeclasses/characters.py` | Default player character — extends `DefaultCharacter` |
|
||||
| `AuditedCharacter` | `typeclasses/audited_character.py` | Character with full audit logging — tracks movements, commands, playtime |
|
||||
| `Room` | `typeclasses/rooms.py` | Default room container |
|
||||
| `Exit` | `typeclasses/exits.py` | Connections between rooms |
|
||||
| `Object` | `typeclasses/objects.py` | Base object with `ObjectParent` mixin |
|
||||
| `Account` | `typeclasses/accounts.py` | Player account (login identity) |
|
||||
| `Channel` | `typeclasses/channels.py` | In-game communication channels |
|
||||
| `Script` | `typeclasses/scripts.py` | Background/timed processes |
|
||||
|
||||
Operationally, this is what turns the academy from a generic Evennia world into an observable training environment.
|
||||
### AuditedCharacter — the flagship typeclass
|
||||
|
||||
### 2. `CharacterCmdSet`
|
||||
File: `commands/default_cmdsets.py`
|
||||
The `AuditedCharacter` is the most important abstraction. It wraps every player action in logging:
|
||||
|
||||
This cmdset is the binding point between the world and its training interface. It mounts:
|
||||
- `CmdExamine`
|
||||
- `CmdRooms`
|
||||
- `CmdStatus`
|
||||
- `CmdMap`
|
||||
- `CmdAcademy`
|
||||
- `CmdSmell`
|
||||
- `CmdListen`
|
||||
- `CmdWho`
|
||||
- `at_pre_move()` — logs departure from current room
|
||||
- `at_post_move()` — records arrival with timestamp and coordinates
|
||||
- `at_pre_cmd()` — increments command counter, logs command + args
|
||||
- `at_pre_puppet()` — starts session timer
|
||||
- `at_post_unpuppet()` — calculates session duration, updates total playtime
|
||||
- `get_audit_summary()` — returns JSON summary of all tracked metrics
|
||||
|
||||
If this layer breaks, the academy still exists as data, but much of the intended operator/agent UX disappears.
|
||||
Audit trail keeps last 1000 movements in `db.location_history`. Sensitive commands (password) are excluded from logging.
|
||||
|
||||
### 3. `CmdStatus`, `CmdMap`, `CmdAcademy`, `CmdWho`
|
||||
File: `commands/command.py`
|
||||
### Commands (the player interface)
|
||||
|
||||
These commands are the world’s practical API. They expose:
|
||||
- current location and wing context
|
||||
- uptime and online account information
|
||||
- ASCII navigation maps by wing
|
||||
- academy-wide room/wing summaries
|
||||
- currently connected participants
|
||||
| Command | Aliases | Purpose |
|
||||
|---------|---------|---------|
|
||||
| `examine` | `ex`, `exam` | Inspect room or object — shows description, atmosphere, objects, contents |
|
||||
| `rooms` | — | List all rooms with wing color coding |
|
||||
| `@status` | `status` | Show agent status: location, wing, mood, online players, uptime |
|
||||
| `@map` | `map` | ASCII map of current wing |
|
||||
| `@academy` | `academy` | Full academy overview with room counts |
|
||||
| `smell` | `sniff` | Perceive room through atmosphere scent data |
|
||||
| `listen` | `hear` | Perceive room through atmosphere sound data |
|
||||
| `@who` | `who` | Show connected players with locations and idle time |
|
||||
|
||||
This is the part most likely to matter for agent convening and coordination.
|
||||
### World Structure (5 wings, 21+ rooms)
|
||||
|
||||
### 4. Wing room classes
|
||||
Files: `world/commons_wing.py`, `world/dormitory_entrance.py`, `world/workshop_wing.py`, `world/gardens_wing.py`
|
||||
**Central Hub (LIMBO)** — Nexus connecting all wings. North=Dormitory, South=Workshop, East=Commons, West=Gardens.
|
||||
|
||||
These classes encode the academy’s content model. Each room defines:
|
||||
- `self.key`
|
||||
- aliases
|
||||
- long-form description
|
||||
- `db.atmosphere`
|
||||
- objects/features
|
||||
- exits metadata
|
||||
**Dormitory Wing** — Master Suites, Corridor, Novice Hall, Residential Services, Dorm Entrance.
|
||||
|
||||
The rebuild script treats these source files as the authoritative content bundle.
|
||||
**Commons Wing** — Grand Commons Hall (main gathering, 60ft ceilings, marble columns), Hearthside Dining, Entertainment Gallery, Scholar's Corner, Upper Balcony.
|
||||
|
||||
### 5. `ROOM_CONFIG` / `WING_INFO`
|
||||
File: `world/rebuild_world.py`
|
||||
**Workshop Wing** — Great Smithy, Alchemy Labs, Woodworking Shop, Artificing Chamber, Workshop Entrance.
|
||||
|
||||
This is the world’s rehydration map. It hard-binds Evennia object IDs to source classes and wings. That makes the rebuild deterministic, but it also couples source truth to existing DB IDs — a real maintenance risk if the database is re-seeded differently.
|
||||
**Gardens Wing** — Enchanted Grove, Herb Gardens, Greenhouse, Sacred Grove, Gardens Entrance.
|
||||
|
||||
### 6. Stress-test dataclasses and `MudClient`
|
||||
File: `tests/stress_test.py`
|
||||
|
||||
The stress harness uses:
|
||||
- `ActionResult`
|
||||
- `PlayerStats`
|
||||
- `StressTestReport`
|
||||
- `MudClient`
|
||||
|
||||
This test file doubles as an executable spec for the live connection surface and the academy’s expected runtime responsiveness.
|
||||
Each room has rich `db.atmosphere` data: mood, lighting, sounds, smells, temperature.
|
||||
|
||||
## API Surface
|
||||
|
||||
### In-world commands
|
||||
Defined in `commands/command.py` and registered in `commands/default_cmdsets.py`.
|
||||
### Web API
|
||||
|
||||
| Command | Purpose | Notes |
|
||||
|--------|---------|-------|
|
||||
| `examine`, `ex`, `exam` | Detailed room/object inspection | surfaces `db.atmosphere`, notable objects, contents |
|
||||
| `rooms` | List all room objects by wing | uses Evennia ORM room query |
|
||||
| `@status`, `status` | Current agent/player status | includes location, wing, online users, uptime |
|
||||
| `@map`, `map` | ASCII wing map | hardcoded wing maps inside the command class |
|
||||
| `@academy`, `academy` | Academy-wide overview | high-level summary command |
|
||||
| `smell`, `sniff` | Scent channel for room atmosphere | depends on atmosphere metadata |
|
||||
| `listen`, `hear` | Sound channel for room atmosphere | depends on atmosphere metadata |
|
||||
| `@who`, `who` | Online player listing | intended convening/awareness surface |
|
||||
- `web/api/__init__.py` — Evennia REST API (Django REST Framework)
|
||||
- `web/urls.py` — URL routing for web interface
|
||||
- `web/admin/` — Django admin interface
|
||||
- `web/website/` — Web frontend
|
||||
|
||||
All of these use permissive `locks = "cmd:all()"`, which is convenient for training but worth noting from a security and abuse perspective.
|
||||
### Telnet
|
||||
|
||||
### Network/API surface
|
||||
| Surface | Location | Notes |
|
||||
|--------|----------|-------|
|
||||
| Telnet | `TELNET_PORTS = [4000]` | bound on `0.0.0.0` |
|
||||
| Web client | `WEBSERVER_PORTS = [(4001, 4005)]` | bound on `0.0.0.0` |
|
||||
| Django web stack | `web/urls.py` | includes website, webclient, admin, and Evennia defaults |
|
||||
| Hermes bridge config | `hermes-agent/config.yaml` | configuration-only integration point; not an executable bridge implementation inside this repo |
|
||||
- Standard MUD protocol on port 4000
|
||||
- Supports MCCP (compression), MSDP (data), GMCP (protocol)
|
||||
|
||||
## World Model
|
||||
### Hermes Bridge
|
||||
|
||||
The academy is modeled as a central hub plus four themed wings, matching the repo’s source files better than the older “five wings” phrasing in the stale genome artifact.
|
||||
|
||||
| Zone | Source | Notes |
|
||||
|------|--------|------|
|
||||
| Central Hub / Limbo | `world/rebuild_world.py` | special-case hub description and routing nexus |
|
||||
| Dormitory Wing | `world/dormitory_entrance.py` | residence/rest zone |
|
||||
| Commons Wing | `world/commons_wing.py` | social and gathering zone |
|
||||
| Workshop Wing | `world/workshop_wing.py` | crafting and alchemy zone |
|
||||
| Gardens Wing | `world/gardens_wing.py` | nature and contemplative zone |
|
||||
|
||||
Grounded repo facts:
|
||||
- README advertises `21 rooms, 43+ exits across 5 zones`
|
||||
- `ROOM_CONFIG` in `world/rebuild_world.py` maps room IDs `3..22` for wing rooms, while Limbo/hub is treated separately
|
||||
- atmosphere metadata is a first-class room feature, not cosmetic prose
|
||||
|
||||
## Verification Performed
|
||||
|
||||
Target repo verification from a fresh clone at `/tmp/timmy-academy-verify`:
|
||||
|
||||
- `python3 -m py_compile commands/command.py commands/default_cmdsets.py server/conf/settings.py typeclasses/audited_character.py world/rebuild_world.py web/urls.py`
|
||||
- `python3 tests/stress_test.py --help`
|
||||
- `python3 tests/stress_test.py --self-test`
|
||||
- `python3 ~/.hermes/pipelines/codebase-genome.py --path /tmp/timmy-academy-verify --output /tmp/timmy-academy-base.md`
|
||||
|
||||
Observed runtime-adjacent facts:
|
||||
- core modules compile as Python
|
||||
- the stress harness advertises `--self-test` and `--json` modes
|
||||
- target repo does **not** contain a checked-in `GENOME.md` at its own root
|
||||
|
||||
## Test Coverage Gaps
|
||||
|
||||
The repo still has only one test file: `tests/stress_test.py`.
|
||||
|
||||
Critical untested paths:
|
||||
1. `typeclasses/audited_character.py`
|
||||
- no direct tests for move logging, audit pruning, command counting, or session accounting
|
||||
2. `commands/command.py`
|
||||
- no command-level unit tests for `@status`, `@map`, `rooms`, `smell`, `listen`, or `@who`
|
||||
3. `world/rebuild_world.py`
|
||||
- no tests for parsing wing files, room ID mapping, exit verification, or idempotent rebuild behavior
|
||||
4. `server/conf/settings.py`
|
||||
- no configuration sanity checks for port exposure, logging handlers, or audit defaults
|
||||
5. `web/urls.py`
|
||||
- no tests confirming routing composition for website/webclient/admin
|
||||
|
||||
The existing stress harness is valuable, but it is not a substitute for unit or integration tests around the repo’s custom command/typeclass logic.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
1. Network exposure
|
||||
- `TELNET_INTERFACES = ['0.0.0.0']`
|
||||
- `WEBSERVER_INTERFACES = ['0.0.0.0']`
|
||||
These settings expose the academy to all interfaces. That may be intended on the VPS, but it shifts safety to firewall/reverse-proxy controls.
|
||||
|
||||
2. Secrets split is expected but must be enforced
|
||||
- `server/conf/settings.py` imports `secret_settings.py`
|
||||
- this is the right shape, but only if `secret_settings.py` is never committed and contains the truly sensitive deployment values
|
||||
|
||||
3. Audit log sensitivity
|
||||
- `AuditedCharacter.at_pre_cmd()` excludes password commands from audit logging
|
||||
- good safeguard, but the rest of the command stream is still intentionally retained and should be treated as sensitive behavioral telemetry
|
||||
|
||||
4. Checked-in bridge environment file
|
||||
- the repo contains `hermes-agent/.env`
|
||||
- even if it is benign now, a checked-in `.env` path is a standing secret-handling risk and should be treated carefully
|
||||
|
||||
5. Framework-level dynamic evaluation risk
|
||||
- Evennia’s config surface includes modules like `server/conf/inlinefuncs.py`
|
||||
- this is inherited framework behavior, but still part of the runtime attack surface
|
||||
|
||||
## CI / Runtime Drift
|
||||
|
||||
This repo has meaningful operational drift and missing automation:
|
||||
|
||||
1. No checked-in CI workflows
|
||||
- no `.gitea/workflows/*` or `.github/workflows/*` coverage surfaced in the fresh clone
|
||||
- the academy relies on manual rebuild and manual stress testing
|
||||
|
||||
2. Target repo root lacks its own `GENOME.md`
|
||||
- the genome issue lives in `timmy-home`
|
||||
- the analyzed repo itself still does not carry an in-repo architecture artifact
|
||||
|
||||
3. `README.md` vs command docs wording drift
|
||||
- README frames the academy as four thematic wings plus a hub/zone model
|
||||
- older generated genome wording called these “five wings”
|
||||
- the source-of-truth model is more accurately “central hub + four wings”
|
||||
|
||||
4. Bridge configuration drift
|
||||
- `hermes-agent/config.yaml` still references `anthropic/claude-opus-4.6`
|
||||
- this is a real integration snapshot inside the repo and should be treated as provider-policy drift if the surrounding stack has moved away from Anthropic
|
||||
- `hermes-agent/config.yaml` — Configuration for AI agent connection
|
||||
- Allows Hermes agents to connect as characters and interact with the world
|
||||
|
||||
## Dependencies
|
||||
|
||||
No `requirements.txt`, `pyproject.toml`, or other dependency lockfile is checked in at the repo root.
|
||||
No `requirements.txt` or `pyproject.toml` found. Dependencies come from Evennia:
|
||||
|
||||
Grounded dependency picture instead comes from source and README:
|
||||
- Evennia 6.0.0
|
||||
- Django (via Evennia)
|
||||
- Twisted (via Evennia)
|
||||
- Python 3.12.x
|
||||
- **evennia** — MUD framework (Django-based)
|
||||
- **django** — Web framework (via Evennia)
|
||||
- **twisted** — Async networking (via Evennia)
|
||||
|
||||
This means environment reproducibility currently depends on external operator knowledge rather than repo-local dependency locking.
|
||||
## Test Coverage Analysis
|
||||
|
||||
## Deployment
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Source modules | 35 |
|
||||
| Test modules | 1 |
|
||||
| Estimated coverage | 0% |
|
||||
| Untested modules | 35 |
|
||||
|
||||
README-documented rebuild path:
|
||||
Only one test file exists: `tests/stress_test.py`. All 35 source modules are untested.
|
||||
|
||||
```bash
|
||||
ssh root@167.99.126.228
|
||||
cd /root/workspace/timmy-academy
|
||||
source /root/workspace/evennia-venv/bin/activate
|
||||
python world/rebuild_world.py
|
||||
```
|
||||
### Critical Untested Paths
|
||||
|
||||
Operationally relevant deployment facts:
|
||||
- target VPS in README: `167.99.126.228`
|
||||
- telnet surface: `4000`
|
||||
- web client surface: `4001`
|
||||
- the repo assumes an Evennia virtualenv outside the repo itself
|
||||
- world rebuild is source-driven and intended to be idempotent
|
||||
1. **AuditedCharacter** — audit logging is the primary value-add. No tests verify movement tracking, command counting, or playtime calculation.
|
||||
2. **Commands** — no tests for any of the 8 commands. The `@map` wing detection, `@who` session tracking, and atmosphere-based commands (`smell`, `listen`) are all untested.
|
||||
3. **World rebuild** — `rebuild_world.py` and `fix_world.py` can destroy and recreate the entire world. No tests ensure they produce valid output.
|
||||
4. **Typeclass hooks** — `at_pre_move`, `at_post_move`, `at_pre_cmd` etc. are never tested in isolation.
|
||||
|
||||
## Technical Debt
|
||||
## Security Considerations
|
||||
|
||||
1. `ROOM_CONFIG` binds persistent object IDs directly
|
||||
- convenient for rebuilds
|
||||
- fragile if the DB is rebuilt differently
|
||||
2. only one test file for an otherwise rich custom surface
|
||||
3. no CI automation for compile/rebuild/smoke validation
|
||||
4. no explicit dependency lockfile
|
||||
5. checked-in `hermes-agent/.env` path raises secret-hygiene questions
|
||||
6. target repo has no first-party `GENOME.md`, so architecture memory still lives mostly outside the repo
|
||||
- ⚠️ Uses `eval()`/`exec()` — Evennia's inlinefuncs module uses eval for dynamic command evaluation. Risk level: inherent to MUD framework.
|
||||
- ⚠️ References secrets/passwords — `settings.py` references `secret_settings.py` for sensitive config. Ensure this file is not committed.
|
||||
- ⚠️ Telnet on 0.0.0.0 — server accepts connections from any IP. Consider firewall rules.
|
||||
- ⚠️ Web client on 0.0.0.0 — same exposure as telnet. Ensure authentication is enforced.
|
||||
- ⚠️ Agent bridge (`hermes-agent/config.yaml`) — verify credentials are not hardcoded.
|
||||
|
||||
## Configuration Files
|
||||
|
||||
- `server/conf/settings.py` — Main Evennia settings (server name, ports, typeclass paths)
|
||||
- `hermes-agent/config.yaml` — Hermes agent bridge configuration
|
||||
- `world/build_academy.ev` — Evennia batch build script
|
||||
- `world/batch_cmds.ev` — Batch command definitions
|
||||
|
||||
## What's Missing
|
||||
|
||||
1. **Tests** — 0% coverage is a critical gap. Priority: AuditedCharacter hooks, command func() methods, world rebuild integrity.
|
||||
2. **CI/CD** — No automated testing pipeline. No GitHub Actions or Gitea workflows.
|
||||
3. **Documentation** — `world/BUILDER_GUIDE.md` exists but no developer onboarding docs.
|
||||
4. **Monitoring** — No health checks, no metrics export, no alerting on server crashes.
|
||||
5. **Backup** — No automated database backup for the Evennia SQLite/PostgreSQL database.
|
||||
|
||||
---
|
||||
|
||||
This genome was refreshed against the live `timmy-academy` repository and verified with compile + stress-harness entrypoint checks, not just copied from the older auto-generated artifact.
|
||||
*Generated by Codebase Genome Pipeline. Review and update manually.*
|
||||
|
||||
70
README.md
70
README.md
@@ -112,6 +112,76 @@ pytest tests/
|
||||
```
|
||||
|
||||
### Project Structure
|
||||
## Sherlock Username Recon Wrapper
|
||||
|
||||
### Quick Usage
|
||||
|
||||
```bash
|
||||
# Opt-in via env var
|
||||
export SHERLOCK_ENABLED=1
|
||||
|
||||
# Or via explicit CLI flag
|
||||
python -m tools.sherlock_wrapper --query "alice" --opt-in --json
|
||||
|
||||
# With site whitelist
|
||||
python -m tools.sherlock_wrapper --query "alice" --opt-in --sites github twitter --json
|
||||
```
|
||||
|
||||
### What It Does
|
||||
|
||||
Builds a bounded local wrapper around the Sherlock username OSINT tool that:
|
||||
|
||||
- **Opt-in gate** — SHERLOCK_ENABLED=1 or `--opt-in` required before any external call
|
||||
- **Local-first caching** — results cached in `~/.cache/timmy/sherlock_cache.db` (TTL: 7 days)
|
||||
- **Normalized JSON** — stable schema with `found`, `missing`, `errors`, and `metadata` sections
|
||||
- **No network egress** — only makes outbound HTTP to target sites through sherlock; never phones home
|
||||
|
||||
### Output Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": "1.0",
|
||||
"query": "alice",
|
||||
"timestamp": "2025-04-26T14:23:00+00:00",
|
||||
"found": [
|
||||
{"site": "github", "url": "https://github.com/alice"}
|
||||
],
|
||||
"missing": ["twitter", "facebook"],
|
||||
"errors": [{"site": "instagram", "error": "timeout"}],
|
||||
"metadata": {
|
||||
"total_sites_checked": 50,
|
||||
"found_count": 1,
|
||||
"missing_count": 48,
|
||||
"error_count": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Setup
|
||||
|
||||
Sherlock must be installed separately:
|
||||
|
||||
```bash
|
||||
pip install sherlock-project
|
||||
```
|
||||
|
||||
The wrapper is pure Python and requires only stdlib apart from sherlock itself.
|
||||
|
||||
### Why an Opt-In Gate?
|
||||
|
||||
Sherlock makes outbound HTTP requests to dozens of third-party sites. The opt-in gate:
|
||||
1. Ensures a human operator explicitly approves this dependency
|
||||
2. Makes the outbound traffic auditable in session logs
|
||||
3. Prevents accidental invocation in automated pipelines
|
||||
|
||||
### Running the Smoke Test
|
||||
|
||||
```bash
|
||||
# Run unit + integration tests
|
||||
pytest tests/test_sherlock_wrapper.py -v
|
||||
```
|
||||
|
||||
|
||||
|
||||
```
|
||||
.
|
||||
|
||||
182
tests/test_sherlock_wrapper.py
Normal file
182
tests/test_sherlock_wrapper.py
Normal file
@@ -0,0 +1,182 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Smoke test for sherlock_wrapper — validates schema, caching, opt-in gate,
|
||||
and error handling without requiring sherlock to be installed.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "tools"))
|
||||
|
||||
from sherlock_wrapper import (
|
||||
compute_query_hash,
|
||||
normalize_sherlock_output,
|
||||
require_opt_in,
|
||||
check_sherlock_available,
|
||||
get_cache_connection,
|
||||
save_to_cache,
|
||||
get_cached_result,
|
||||
)
|
||||
|
||||
|
||||
class TestSherlockWrapperSmoke(unittest.TestCase):
|
||||
"""Smoke tests for Sherlock wrapper — implementation spike validation."""
|
||||
|
||||
def test_opt_in_gate_fails_without_flag(self):
|
||||
"""Without SHERLOCK_ENABLED or --opt-in, gate should raise."""
|
||||
with patch("sherlock_wrapper.SHERLOCK_ENABLED", False):
|
||||
with self.assertRaises(RuntimeError) as ctx:
|
||||
require_opt_in(opt_in=False)
|
||||
self.assertIn("opt-in only", str(ctx.exception).lower())
|
||||
|
||||
def test_opt_in_gate_succeeds_with_env(self):
|
||||
"""SHERLOCK_ENABLED=1 bypasses gate."""
|
||||
with patch("sherlock_wrapper.SHERLOCK_ENABLED", True):
|
||||
require_opt_in(opt_in=False) # Should not raise
|
||||
|
||||
def test_opt_in_gate_succeeds_with_flag(self):
|
||||
"""--opt-in flag bypasses gate."""
|
||||
with patch("sherlock_wrapper.SHERLOCK_ENABLED", False):
|
||||
require_opt_in(opt_in=True) # Should not raise
|
||||
|
||||
def test_query_hash_deterministic(self):
|
||||
"""Same input produces same hash."""
|
||||
h1 = compute_query_hash("alice")
|
||||
h2 = compute_query_hash("alice")
|
||||
self.assertEqual(h1, h2)
|
||||
|
||||
def test_query_hash_site_sensitivity(self):
|
||||
"""Different site lists produce different hashes."""
|
||||
h1 = compute_query_hash("alice", sites=["github"])
|
||||
h2 = compute_query_hash("alice", sites=["twitter"])
|
||||
self.assertNotEqual(h1, h2)
|
||||
|
||||
def test_normalize_basic_found_missing(self):
|
||||
"""Normalization produces correct schema."""
|
||||
raw = {
|
||||
"github": {"status": "found", "url": "https://github.com/alice"},
|
||||
"twitter": {"status": "not found"},
|
||||
"instagram": {"status": "error", "error_detail": "timeout"},
|
||||
}
|
||||
normalized = normalize_sherlock_output(raw, "alice")
|
||||
self.assertEqual(normalized["query"], "alice")
|
||||
self.assertEqual(normalized["metadata"]["found_count"], 1)
|
||||
self.assertEqual(normalized["metadata"]["missing_count"], 1)
|
||||
self.assertEqual(normalized["metadata"]["error_count"], 1)
|
||||
self.assertEqual(len(normalized["found"]), 1)
|
||||
self.assertEqual(normalized["found"][0]["site"], "github")
|
||||
self.assertIn("twitter", normalized["missing"])
|
||||
self.assertEqual(normalized["errors"][0]["site"], "instagram")
|
||||
|
||||
def test_normalized_schema_has_required_fields(self):
|
||||
"""Output schema contains all required top-level keys."""
|
||||
raw = {"site1": {"status": "not found"}}
|
||||
normalized = normalize_sherlock_output(raw, "testuser")
|
||||
required = ["schema_version", "query", "timestamp", "found", "missing",
|
||||
"errors", "metadata"]
|
||||
for key in required:
|
||||
self.assertIn(key, normalized)
|
||||
self.assertIsInstance(normalized["timestamp"], str)
|
||||
self.assertIsInstance(normalized["found"], list)
|
||||
self.assertIsInstance(normalized["missing"], list)
|
||||
self.assertIsInstance(normalized["errors"], list)
|
||||
self.assertIsInstance(normalized["metadata"], dict)
|
||||
|
||||
def test_cache_roundtrip(self):
|
||||
"""Result can be written and read back from cache."""
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with patch("sherlock_wrapper.CACHE_DB", Path(tmp) / "cache.db"):
|
||||
test_result = {
|
||||
"schema_version": "1.0",
|
||||
"query": "alice",
|
||||
"timestamp": "2025-04-26T00:00:00+00:00",
|
||||
"found": [],
|
||||
"missing": ["github"],
|
||||
"errors": [],
|
||||
"metadata": {"total_sites_checked": 1, "found_count": 0, "missing_count": 1, "error_count": 0},
|
||||
}
|
||||
query_hash = compute_query_hash("alice")
|
||||
save_to_cache(query_hash, test_result)
|
||||
retrieved = get_cached_result(query_hash)
|
||||
self.assertEqual(retrieved, test_result)
|
||||
|
||||
def test_cache_miss_on_stale(self):
|
||||
"""Cache returns None when entry is older than 7 days."""
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
db_path = Path(tmp) / "cache.db"
|
||||
with patch("sherlock_wrapper.CACHE_DB", db_path):
|
||||
old_ts = "2025-04-01T00:00:00+00:00"
|
||||
old_result = {
|
||||
"schema_version": "1.0", "query": "alice",
|
||||
"timestamp": old_ts, "found": [], "missing": [], "errors": [],
|
||||
"metadata": {"total_sites_checked": 0, "found_count": 0, "missing_count": 0, "error_count": 0},
|
||||
}
|
||||
query_hash = compute_query_hash("alice")
|
||||
# Direct DB insert with controlled timestamp (bypass save_to_cache's NOW)
|
||||
conn = get_cache_connection()
|
||||
conn.execute(
|
||||
"INSERT INTO cache (query_hash, result_json, timestamp) VALUES (?, ?, ?)",
|
||||
(query_hash, json.dumps(old_result), old_ts)
|
||||
)
|
||||
conn.commit()
|
||||
retrieved = get_cached_result(query_hash)
|
||||
self.assertIsNone(retrieved)
|
||||
|
||||
def test_sherlock_available_check(self):
|
||||
"""check_sherlock_available returns bool."""
|
||||
available = check_sherlock_available()
|
||||
self.assertIsInstance(available, bool)
|
||||
# Note: on this test system sherlock may not be installed, so False is expected.
|
||||
# The important thing is the function returns a bool.
|
||||
print(f"[INFO] Sherlock installed: {available}")
|
||||
|
||||
|
||||
class TestSherlockWrapperIntegration(unittest.TestCase):
|
||||
"""Integration tests with mocked sherlock module."""
|
||||
|
||||
def test_run_sherlock_with_opt_in(self):
|
||||
"""run_sherlock succeeds with opt-in and returns normalized result."""
|
||||
fake_sherlock = MagicMock()
|
||||
fake_sherlock.sherlock = MagicMock(return_value={
|
||||
"github": {"status": "found", "url": "https://github.com/alice"},
|
||||
"twitter": {"status": "not found"},
|
||||
})
|
||||
with patch.dict("sys.modules", {"sherlock": fake_sherlock}):
|
||||
import importlib
|
||||
import sherlock_wrapper
|
||||
importlib.reload(sherlock_wrapper)
|
||||
with patch.dict(os.environ, {"SHERLOCK_ENABLED": "1"}):
|
||||
from sherlock_wrapper import run_sherlock
|
||||
result = run_sherlock("alice", opt_in=True)
|
||||
self.assertEqual(result["query"], "alice")
|
||||
self.assertEqual(result["metadata"]["found_count"], 1)
|
||||
|
||||
def test_run_sherlock_fails_without_opt_in(self):
|
||||
"""run_sherlock raises RuntimeError without opt-in."""
|
||||
from sherlock_wrapper import run_sherlock
|
||||
with self.assertRaises(RuntimeError) as ctx:
|
||||
run_sherlock("alice", opt_in=False)
|
||||
self.assertIn("opt-in only", str(ctx.exception).lower())
|
||||
|
||||
def test_run_sherlock_uses_cache(self):
|
||||
"""Cached result short-circuits sherlock execution."""
|
||||
cached = {
|
||||
"schema_version": "1.0", "query": "alice", "timestamp": "2025-04-26T00:00:00+00:00",
|
||||
"found": [{"site": "github", "url": "https://github.com/alice"}],
|
||||
"missing": ["twitter"],
|
||||
"errors": [],
|
||||
"metadata": {"total_sites_checked": 2, "found_count": 1, "missing_count": 1, "error_count": 0},
|
||||
}
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
with patch("sherlock_wrapper.CACHE_DB", Path(tmp) / "cache.db"):
|
||||
query_hash = compute_query_hash("alice")
|
||||
save_to_cache(query_hash, cached)
|
||||
from sherlock_wrapper import run_sherlock
|
||||
result = run_sherlock("alice", opt_in=True)
|
||||
self.assertEqual(result, cached)
|
||||
@@ -1,67 +0,0 @@
|
||||
"""Lock timmy-academy genome to current verified repo facts. Ref: #678."""
|
||||
from pathlib import Path
|
||||
|
||||
GENOME = Path("GENOME-timmy-academy.md")
|
||||
|
||||
|
||||
def read_genome() -> str:
|
||||
assert GENOME.exists(), "timmy-academy genome must exist at repo root"
|
||||
return GENOME.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def test_genome_exists():
|
||||
assert GENOME.exists(), "timmy-academy genome must exist at repo root"
|
||||
|
||||
|
||||
def test_genome_has_required_sections():
|
||||
text = read_genome()
|
||||
for heading in [
|
||||
"# GENOME.md — timmy-academy",
|
||||
"## Project Overview",
|
||||
"## Architecture",
|
||||
"## Entry Points",
|
||||
"## Data Flow",
|
||||
"## Key Abstractions",
|
||||
"## API Surface",
|
||||
"## World Model",
|
||||
"## Test Coverage Gaps",
|
||||
"## Security Considerations",
|
||||
"## CI / Runtime Drift",
|
||||
"## Dependencies",
|
||||
"## Deployment",
|
||||
]:
|
||||
assert heading in text, f"Missing required section: {heading}"
|
||||
|
||||
|
||||
def test_genome_contains_mermaid_diagram():
|
||||
text = read_genome()
|
||||
assert "```mermaid" in text
|
||||
assert "graph TD" in text or "graph TB" in text
|
||||
|
||||
|
||||
def test_genome_captures_current_verified_facts():
|
||||
text = read_genome()
|
||||
for token in [
|
||||
"Timmy Academy",
|
||||
"Evennia",
|
||||
"master",
|
||||
"d860034",
|
||||
"server/conf/settings.py",
|
||||
"commands/default_cmdsets.py",
|
||||
"typeclasses/audited_character.py",
|
||||
"world/rebuild_world.py",
|
||||
"tests/stress_test.py",
|
||||
"python3 tests/stress_test.py --self-test",
|
||||
"TELNET_PORTS = [4000]",
|
||||
"WEBSERVER_PORTS = [(4001, 4005)]",
|
||||
"0.0.0.0",
|
||||
"secret_settings.py",
|
||||
"hermes-agent/config.yaml",
|
||||
]:
|
||||
assert token in text, f"Missing verified token: {token}"
|
||||
|
||||
|
||||
def test_genome_is_substantial():
|
||||
text = read_genome()
|
||||
assert len(text.splitlines()) >= 120
|
||||
assert len(text) >= 7000
|
||||
0
tools/__init__.py
Normal file
0
tools/__init__.py
Normal file
249
tools/sherlock_wrapper.py
Normal file
249
tools/sherlock_wrapper.py
Normal file
@@ -0,0 +1,249 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Sherlock username recon wrapper — opt-in, cached, normalized JSON output.
|
||||
|
||||
This is an implementation spike (issue #874) to validate local integration
|
||||
of the Sherlock OSINT tool without violating sovereignty/provenance standards.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Optional, Dict, Any, List
|
||||
|
||||
# Opt-in gate: must have SHERLOCK_ENABLED=1 or --opt-in flag
|
||||
SHERLOCK_ENABLED = os.environ.get("SHERLOCK_ENABLED", "0") == "1"
|
||||
|
||||
# Cache location
|
||||
CACHE_DIR = Path.home() / ".cache" / "timmy"
|
||||
CACHE_DB = CACHE_DIR / "sherlock_cache.db"
|
||||
|
||||
# Normalized output schema version
|
||||
SCHEMA_VERSION = "1.0"
|
||||
|
||||
|
||||
def require_opt_in(opt_in: bool = False) -> None:
|
||||
"""Enforce opt-in gate for Sherlock external dependency."""
|
||||
if not (SHERLOCK_ENABLED or opt_in):
|
||||
raise RuntimeError(
|
||||
"Sherlock is opt-in only. Set SHERLOCK_ENABLED=1 or pass --opt-in."
|
||||
)
|
||||
|
||||
|
||||
|
||||
def check_sherlock_available() -> bool:
|
||||
"""Check if sherlock Python package is installed."""
|
||||
try:
|
||||
import sherlock # type: ignore # noqa: F401
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
|
||||
def get_cache_connection() -> sqlite3.Connection:
|
||||
"""Initialize cache directory and return DB connection."""
|
||||
CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
conn = sqlite3.connect(str(CACHE_DB))
|
||||
conn.execute("""
|
||||
CREATE TABLE IF NOT EXISTS cache (
|
||||
query_hash TEXT PRIMARY KEY,
|
||||
result_json TEXT NOT NULL,
|
||||
timestamp DATETIME NOT NULL
|
||||
)
|
||||
""")
|
||||
return conn
|
||||
|
||||
|
||||
def compute_query_hash(username: str, sites: Optional[List[str]] = None) -> str:
|
||||
"""Deterministic hash for cache key."""
|
||||
components = [username.lower().strip()]
|
||||
if sites:
|
||||
components.extend(sorted(sites))
|
||||
raw = "|".join(components)
|
||||
return hashlib.sha256(raw.encode()).hexdigest()
|
||||
|
||||
|
||||
def get_cached_result(query_hash: str) -> Optional[Dict[str, Any]]:
|
||||
"""Retrieve cached result if available and not stale (TTL: 7 days)."""
|
||||
conn = get_cache_connection()
|
||||
cur = conn.execute(
|
||||
"SELECT result_json, timestamp FROM cache WHERE query_hash = ?",
|
||||
(query_hash,)
|
||||
)
|
||||
row = cur.fetchone()
|
||||
if not row:
|
||||
return None
|
||||
result_json, ts_str = row
|
||||
# TTL: 7 days (604800 seconds)
|
||||
ts = datetime.fromisoformat(ts_str)
|
||||
age_seconds = (datetime.now(timezone.utc) - ts).total_seconds()
|
||||
if age_seconds >= 604800:
|
||||
return None
|
||||
return json.loads(result_json)
|
||||
|
||||
|
||||
|
||||
|
||||
def save_to_cache(query_hash: str, result: Dict[str, Any]) -> None:
|
||||
"""Persist result to cache."""
|
||||
conn = get_cache_connection()
|
||||
conn.execute(
|
||||
"INSERT OR REPLACE INTO cache (query_hash, result_json, timestamp) VALUES (?, ?, ?)",
|
||||
(query_hash, json.dumps(result), datetime.now(timezone.utc).isoformat())
|
||||
)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
def normalize_sherlock_output(
|
||||
raw_result: Dict[str, Any],
|
||||
username: str,
|
||||
sites_checked: Optional[List[str]] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Convert raw sherlock output into a stable, normalized schema.
|
||||
|
||||
Expected sherlock result shape (via Python API):
|
||||
{
|
||||
"site_name": {"url": "...", "status": "found"|"not found"|"error", ...},
|
||||
...
|
||||
}
|
||||
"""
|
||||
found: List[Dict[str, str]] = []
|
||||
missing: List[str] = []
|
||||
errors: List[Dict[str, str]] = []
|
||||
|
||||
for site_name, site_data in raw_result.items():
|
||||
status = site_data.get("status", "")
|
||||
url = site_data.get("url", "")
|
||||
if status == "found" and url:
|
||||
found.append({"site": site_name, "url": url})
|
||||
elif status == "not found":
|
||||
missing.append(site_name)
|
||||
else:
|
||||
errors.append({"site": site_name, "error": status or "unknown"})
|
||||
|
||||
# Compute totals from the original site list if provided
|
||||
total_sites = len(raw_result) if sites_checked is None else len(sites_checked)
|
||||
|
||||
return {
|
||||
"schema_version": SCHEMA_VERSION,
|
||||
"query": username,
|
||||
"timestamp": datetime.now(timezone.utc).isoformat(),
|
||||
"found": found,
|
||||
"missing": missing,
|
||||
"errors": errors,
|
||||
"metadata": {
|
||||
"total_sites_checked": total_sites,
|
||||
"found_count": len(found),
|
||||
"missing_count": len(missing),
|
||||
"error_count": len(errors),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def run_sherlock(
|
||||
username: str,
|
||||
sites: Optional[List[str]] = None,
|
||||
timeout: Optional[int] = None,
|
||||
opt_in: bool = False
|
||||
) -> Dict[str, Any]:
|
||||
"""
|
||||
Execute Sherlock wrapper with opt-in gate, caching, and normalization.
|
||||
"""
|
||||
require_opt_in(opt_in)
|
||||
|
||||
# Compute cache key
|
||||
query_hash = compute_query_hash(username, sites)
|
||||
|
||||
# Check cache first — avoids dependency requirement on cache hit
|
||||
cached = get_cached_result(query_hash)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
# Only require sherlock on cache miss
|
||||
if not check_sherlock_available():
|
||||
raise RuntimeError(
|
||||
"Sherlock Python package not installed. "
|
||||
"Install with: pip install sherlock-project"
|
||||
)
|
||||
|
||||
# Call sherlock
|
||||
try:
|
||||
import sherlock
|
||||
from sherlock import sherlock as sherlock_main # type: ignore
|
||||
|
||||
if sites:
|
||||
result = sherlock_main(username, site_list=sites, timeout=timeout or 10)
|
||||
else:
|
||||
result = sherlock_main(username, timeout=timeout or 10)
|
||||
|
||||
normalized = normalize_sherlock_output(result, username, sites)
|
||||
save_to_cache(query_hash, normalized)
|
||||
return normalized
|
||||
|
||||
except Exception as e:
|
||||
raise RuntimeError(f"Sherlock execution failed: {e}") from e
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Sherlock username OSINT wrapper — opt-in, cached, normalized JSON"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--query", "-q", required=True,
|
||||
help="Username to search across sites"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--opt-in", action="store_true",
|
||||
help="Explicit opt-in flag (alternatively set SHERLOCK_ENABLED=1)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--sites", "-s", nargs="+",
|
||||
help="Specific sites to check (default: all supported)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--timeout", "-t", type=int, default=10,
|
||||
help="Request timeout per site (default: 10)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--json", action="store_true",
|
||||
help="Output normalized JSON to stdout"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-cache",
|
||||
action="store_true",
|
||||
help="Bypass cached result (if any)"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
result = run_sherlock(
|
||||
username=args.query,
|
||||
sites=args.sites,
|
||||
timeout=args.timeout,
|
||||
opt_in=args.opt_in
|
||||
)
|
||||
if args.json:
|
||||
print(json.dumps(result, indent=2))
|
||||
else:
|
||||
print(f"Query: {result['query']}")
|
||||
print(f"Found: {result['metadata']['found_count']} site(s)")
|
||||
print(f"Missing: {result['metadata']['missing_count']} site(s)")
|
||||
print(f"Errors: {result['metadata']['error_count']} site(s)")
|
||||
for f in result['found']:
|
||||
print(f" [{f['site']}] {f['url']}")
|
||||
return 0
|
||||
except RuntimeError as e:
|
||||
print(f"ERROR: {e}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user