Compare commits

..

2 Commits

Author SHA1 Message Date
Alexander Whitestone
599904d945 fix: log plugin memory provider fallback failure at debug level
All checks were successful
Lint / lint (pull_request) Successful in 9s
Address review feedback on PR #1002: replace silent `except Exception: pass`
with `logger.debug(...)` so plugin loading failures are visible in debug logs.

Refs #990

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:54:10 -04:00
Alexander Whitestone
df1bfe433a fix: add register_memory_provider to PluginContext — fixes #990
PluginContext was missing register_memory_provider(), causing any
user plugin (e.g. MemPalace) that followed the documented pattern
of calling ctx.register_memory_provider(provider) in its register()
function to fail at startup with:

  'PluginContext' object has no attribute 'register_memory_provider'

Changes:
- hermes_cli/plugins.py: Add register_memory_provider() to PluginContext,
  _plugin_memory_provider field to PluginManager, and module-level
  get_plugin_memory_provider() accessor function.
- run_agent.py: After failing to find a provider in plugins/memory/,
  fall back to checking get_plugin_memory_provider() — mirrors how
  context engine plugins are resolved.
- tests: Add TestRegisterMemoryProvider with four regression tests
  covering success, accessor, duplicate rejection, and type validation.

Fixes #990

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-22 10:54:10 -04:00
7 changed files with 158 additions and 225 deletions

View File

@@ -211,6 +211,43 @@ class PluginContext:
}
logger.debug("Plugin %s registered CLI command: %s", self.manifest.name, name)
# -- memory provider registration ----------------------------------------
def register_memory_provider(self, provider) -> None:
"""Register a memory provider supplied by this plugin.
The provider must be an instance of ``agent.memory_provider.MemoryProvider``.
Only one plugin-registered memory provider is accepted; a second
attempt is rejected with a warning.
The registered provider is retrievable via
``get_plugin_memory_provider()`` and is picked up by ``run_agent.py``
when ``memory.provider`` in *config.yaml* matches the provider's
``name`` property.
"""
from agent.memory_provider import MemoryProvider
if not isinstance(provider, MemoryProvider):
logger.warning(
"Plugin '%s' tried to register a memory provider that does not "
"inherit from MemoryProvider. Ignoring.",
self.manifest.name,
)
return
if self._manager._plugin_memory_provider is not None:
logger.warning(
"Plugin '%s' tried to register a memory provider, but one is "
"already registered by another plugin. Only one plugin-supplied "
"memory provider is allowed at a time.",
self.manifest.name,
)
return
self._manager._plugin_memory_provider = provider
logger.info(
"Plugin '%s' registered memory provider: %s",
self.manifest.name, provider.name,
)
# -- context engine registration -----------------------------------------
def register_context_engine(self, engine) -> None:
@@ -323,6 +360,7 @@ class PluginManager:
self._plugin_tool_names: Set[str] = set()
self._cli_commands: Dict[str, dict] = {}
self._context_engine = None # Set by a plugin via register_context_engine()
self._plugin_memory_provider = None # Set by a plugin via register_memory_provider()
self._discovered: bool = False
self._cli_ref = None # Set by CLI after plugin discovery
# Plugin skill registry: qualified name → metadata dict.
@@ -699,6 +737,11 @@ def get_plugin_context_engine():
return get_plugin_manager()._context_engine
def get_plugin_memory_provider():
"""Return the plugin-registered memory provider, or None."""
return get_plugin_manager()._plugin_memory_provider
def get_plugin_toolsets() -> List[tuple]:
"""Return plugin toolsets as ``(key, label, description)`` tuples.

View File

@@ -1,190 +0,0 @@
---
name: adversarial-ux-test
description: Roleplay the most difficult, tech-resistant user for your product. Browse the app as that persona, find every UX pain point, then filter complaints through a pragmatism layer to separate real problems from noise. Creates actionable tickets from genuine issues only.
version: 1.0.0
author: Omni @ Comelse
license: MIT
metadata:
hermes:
tags: [qa, ux, testing, adversarial, dogfood, personas, user-testing]
related_skills: [dogfood]
---
# Adversarial UX Test
Roleplay the worst-case user for your product — the person who hates technology, doesn't want your software, and will find every reason to complain. Then filter their feedback through a pragmatism layer to separate real UX problems from "I hate computers" noise.
Think of it as an automated "mom test" — but angry.
## Why This Works
Most QA finds bugs. This finds **friction**. A technically correct app can still be unusable for real humans. The adversarial persona catches:
- Confusing terminology that makes sense to developers but not users
- Too many steps to accomplish basic tasks
- Missing onboarding or "aha moments"
- Accessibility issues (font size, contrast, click targets)
- Cold-start problems (empty states, no demo content)
- Paywall/signup friction that kills conversion
The **pragmatism filter** (Phase 3) is what makes this useful instead of just entertaining. Without it, you'd add a "print this page" button to every screen because Grandpa can't figure out PDFs.
## How to Use
Tell the agent:
```
"Run an adversarial UX test on [URL]"
"Be a grumpy [persona type] and test [app name]"
"Do an asshole user test on my staging site"
```
You can provide a persona or let the agent generate one based on your product's target audience.
## Step 1: Define the Persona
If no persona is provided, generate one by answering:
1. **Who is the HARDEST user for this product?** (age 50+, non-technical role, decades of experience doing it "the old way")
2. **What is their tech comfort level?** (the lower the better — WhatsApp-only, paper notebooks, wife set up their email)
3. **What is the ONE thing they need to accomplish?** (their core job, not your feature list)
4. **What would make them give up?** (too many clicks, jargon, slow, confusing)
5. **How do they talk when frustrated?** (blunt, sweary, dismissive, sighing)
### Good Persona Example
> **"Big Mick" McAllister** — 58-year-old S&C coach. Uses WhatsApp and that's it. His "spreadsheet" is a paper notebook. "If I can't figure it out in 10 seconds I'm going back to my notebook." Needs to log session results for 25 players. Hates small text, jargon, and passwords.
### Bad Persona Example
> "A user who doesn't like the app" — too vague, no constraints, no voice.
The persona must be **specific enough to stay in character** for 20 minutes of testing.
## Step 2: Become the Asshole (Browse as the Persona)
1. Read any available project docs for app context and URLs
2. **Fully inhabit the persona** — their frustrations, limitations, goals
3. Navigate to the app using browser tools
4. **Attempt the persona's ACTUAL TASKS** (not a feature tour):
- Can they do what they came to do?
- How many clicks/screens to accomplish it?
- What confuses them?
- What makes them angry?
- Where do they get lost?
- What would make them give up and go back to their old way?
5. Test these friction categories:
- **First impression** — would they even bother past the landing page?
- **Core workflow** — the ONE thing they need to do most often
- **Error recovery** — what happens when they do something wrong?
- **Readability** — text size, contrast, information density
- **Speed** — does it feel faster than their current method?
- **Terminology** — any jargon they wouldn't understand?
- **Navigation** — can they find their way back? do they know where they are?
6. Take screenshots of every pain point
7. Check browser console for JS errors on every page
## Step 3: The Rant (Write Feedback in Character)
Write the feedback AS THE PERSONA — in their voice, with their frustrations. This is not a bug report. This is a real human venting.
```
[PERSONA NAME]'s Review of [PRODUCT]
Overall: [Would they keep using it? Yes/No/Maybe with conditions]
THE GOOD (grudging admission):
- [things even they have to admit work]
THE BAD (legitimate UX issues):
- [real problems that would stop them from using the product]
THE UGLY (showstoppers):
- [things that would make them uninstall/cancel immediately]
SPECIFIC COMPLAINTS:
1. [Page/feature]: "[quote in persona voice]" — [what happened, expected]
2. ...
VERDICT: "[one-line persona quote summarizing their experience]"
```
## Step 4: The Pragmatism Filter (Critical — Do Not Skip)
Step OUT of the persona. Evaluate each complaint as a product person:
- **RED: REAL UX BUG** — Any user would have this problem, not just grumpy ones. Fix it.
- **YELLOW: VALID BUT LOW PRIORITY** — Real issue but only for extreme users. Note it.
- **WHITE: PERSONA NOISE** — "I hate computers" talking, not a product problem. Skip it.
- **GREEN: FEATURE REQUEST** — Good idea hidden in the complaint. Consider it.
### Filter Criteria
1. Would a 35-year-old competent-but-busy user have the same complaint? → RED
2. Is this a genuine accessibility issue (font size, contrast, click targets)? → RED
3. Is this "I want it to work like paper" resistance to digital? → WHITE
4. Is this a real workflow inefficiency the persona stumbled on? → YELLOW or RED
5. Would fixing this add complexity for the 80% who are fine? → WHITE
6. Does the complaint reveal a missing onboarding moment? → GREEN
**This filter is MANDATORY.** Never ship raw persona complaints as tickets.
## Step 5: Create Tickets
For **RED** and **GREEN** items only:
- Clear, actionable title
- Include the persona's verbatim quote (entertaining + memorable)
- The real UX issue underneath (objective)
- A suggested fix (actionable)
- Tag/label: "ux-review"
For **YELLOW** items: one catch-all ticket with all notes.
**WHITE** items appear in the report only. No tickets.
**Max 10 tickets per session** — focus on the worst issues.
## Step 6: Report
Deliver:
1. The persona rant (Step 3) — entertaining and visceral
2. The filtered assessment (Step 4) — pragmatic and actionable
3. Tickets created (Step 5) — with links
4. Screenshots of key issues
## Tips
- **One persona per session.** Don't mix perspectives.
- **Stay in character during Steps 2-3.** Break character only at Step 4.
- **Test the CORE WORKFLOW first.** Don't get distracted by settings pages.
- **Empty states are gold.** New user experience reveals the most friction.
- **The best findings are RED items the persona found accidentally** while trying to do something else.
- **If the persona has zero complaints, your persona is too tech-savvy.** Make them older, less patient, more set in their ways.
- **Run this before demos, launches, or after shipping a batch of features.**
- **Register as a NEW user when possible.** Don't use pre-seeded admin accounts — the cold start experience is where most friction lives.
- **Zero WHITE items is a signal, not a failure.** If the pragmatism filter finds no noise, your product has real UX problems, not just a grumpy persona.
- **Check known issues in project docs AFTER the test.** If the persona found a bug that's already in the known issues list, that's actually the most damning finding — it means the team knew about it but never felt the user's pain.
- **Subscription/paywall testing is critical.** Test with expired accounts, not just active ones. The "what happens when you can't pay" experience reveals whether the product respects users or holds their data hostage.
- **Count the clicks to accomplish the persona's ONE task.** If it's more than 5, that's almost always a RED finding regardless of persona tech level.
## Example Personas by Industry
These are starting points — customize for your specific product:
| Product Type | Persona | Age | Key Trait |
|-------------|---------|-----|-----------|
| CRM | Retirement home director | 68 | Filing cabinet is the current CRM |
| Photography SaaS | Rural wedding photographer | 62 | Books clients by phone, invoices on paper |
| AI/ML Tool | Department store buyer | 55 | Burned by 3 failed tech startups |
| Fitness App | Old-school gym coach | 58 | Paper notebook, thick fingers, bad eyes |
| Accounting | Family bakery owner | 64 | Shoebox of receipts, hates subscriptions |
| E-commerce | Market stall vendor | 60 | Cash only, smartphone is for calls |
| Healthcare | Senior GP | 63 | Dictates notes, nurse handles the computer |
| Education | Veteran teacher | 57 | Chalk and talk, worksheets in ring binders |
## Rules
- Stay in character during Steps 2-3
- Be genuinely mean but fair — find real problems, not manufactured ones
- The pragmatism filter (Step 4) is **MANDATORY**
- Screenshots required for every complaint
- Max 10 tickets per session
- Test on staging/deployed app, not local dev
- One persona, one session, one report

View File

@@ -1193,6 +1193,18 @@ class AIAgent:
from plugins.memory import load_memory_provider as _load_mem
self._memory_manager = _MemoryManager()
_mp = _load_mem(_mem_provider_name)
# Fall back to a user plugin that called register_memory_provider()
if _mp is None:
try:
from hermes_cli.plugins import get_plugin_memory_provider as _gpm
_candidate = _gpm()
if _candidate and _candidate.name == _mem_provider_name:
_mp = _candidate
except Exception as _gpm_err:
logger.debug(
"get_plugin_memory_provider() failed during fallback lookup: %s",
_gpm_err,
)
if _mp and _mp.is_available():
self._memory_manager.add_provider(_mp)
if self._memory_manager.providers:

View File

@@ -19,6 +19,7 @@ from hermes_cli.plugins import (
PluginManifest,
get_plugin_manager,
get_pre_tool_call_block_message,
get_plugin_memory_provider,
discover_plugins,
invoke_hook,
)
@@ -609,3 +610,105 @@ class TestPreLlmCallTargetRouting:
# in PluginContext (hermes_cli/plugins.py). The tests referenced _plugin_commands,
# commands_registered, get_plugin_command_handler, and GATEWAY_KNOWN_COMMANDS
# integration — all of which are unimplemented features.
# ── TestRegisterMemoryProvider ─────────────────────────────────────────────
class TestRegisterMemoryProvider:
"""Regression tests for PluginContext.register_memory_provider() — issue #990.
The MemPalace plugin (and any user plugin following the developer guide)
calls ``ctx.register_memory_provider(provider)`` inside ``register(ctx)``.
Before the fix, PluginContext had no such method and the plugin failed to
load with: 'PluginContext' object has no attribute 'register_memory_provider'.
"""
def _make_memory_plugin(self, plugins_dir: "Path", name: str) -> None:
"""Write a minimal user plugin that registers a stub MemoryProvider."""
from agent.memory_provider import MemoryProvider
plugin_dir = plugins_dir / name
plugin_dir.mkdir(parents=True, exist_ok=True)
(plugin_dir / "plugin.yaml").write_text(
f"name: {name}\nversion: 0.1.0\ndescription: Stub memory plugin\n"
)
# The register() body imports and calls register_memory_provider — this
# is the exact pattern documented in memory-provider-plugin.md and used
# by third-party plugins such as MemPalace.
(plugin_dir / "__init__.py").write_text(
"from agent.memory_provider import MemoryProvider\n"
"\n"
"class _StubProvider(MemoryProvider):\n"
" @property\n"
f" def name(self): return '{name}'\n"
" def is_available(self): return True\n"
" def initialize(self, session_id, **kw): pass\n"
" def get_tool_schemas(self): return []\n"
"\n"
"def register(ctx):\n"
" ctx.register_memory_provider(_StubProvider())\n"
)
def test_register_memory_provider_succeeds(self, tmp_path, monkeypatch):
"""A user plugin calling register_memory_provider() loads without error."""
plugins_dir = tmp_path / "hermes_test" / "plugins"
self._make_memory_plugin(plugins_dir, "mempalace")
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hermes_test"))
mgr = PluginManager()
mgr.discover_and_load()
assert "mempalace" in mgr._plugins
assert mgr._plugins["mempalace"].enabled, (
mgr._plugins["mempalace"].error
)
def test_plugin_memory_provider_stored(self, tmp_path, monkeypatch):
"""The provider instance is accessible via get_plugin_memory_provider()."""
import hermes_cli.plugins as plugins_mod
plugins_dir = tmp_path / "hermes_test" / "plugins"
self._make_memory_plugin(plugins_dir, "mempalace")
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hermes_test"))
mgr = PluginManager()
# Swap the singleton so get_plugin_memory_provider() sees our manager
monkeypatch.setattr(plugins_mod, "_plugin_manager", mgr)
mgr.discover_and_load()
provider = get_plugin_memory_provider()
assert provider is not None
assert provider.name == "mempalace"
def test_second_registration_rejected(self, tmp_path, monkeypatch):
"""Only one plugin-registered memory provider is accepted."""
plugins_dir = tmp_path / "hermes_test" / "plugins"
self._make_memory_plugin(plugins_dir, "first_provider")
self._make_memory_plugin(plugins_dir, "second_provider")
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hermes_test"))
mgr = PluginManager()
mgr.discover_and_load()
# The manager should hold exactly one provider
assert mgr._plugin_memory_provider is not None
assert mgr._plugin_memory_provider.name in {"first_provider", "second_provider"}
def test_non_provider_rejected(self, tmp_path, monkeypatch):
"""Passing a non-MemoryProvider object logs a warning and is ignored."""
plugins_dir = tmp_path / "hermes_test" / "plugins"
plugin_dir = plugins_dir / "bad_provider"
plugin_dir.mkdir(parents=True, exist_ok=True)
(plugin_dir / "plugin.yaml").write_text("name: bad_provider\n")
(plugin_dir / "__init__.py").write_text(
"def register(ctx):\n"
" ctx.register_memory_provider('not-a-provider')\n"
)
monkeypatch.setenv("HERMES_HOME", str(tmp_path / "hermes_test"))
mgr = PluginManager()
mgr.discover_and_load()
# Plugin still loads (warning only), but no provider is stored
assert mgr._plugin_memory_provider is None

View File

@@ -1,25 +0,0 @@
from pathlib import Path
from tools.skills_hub import OptionalSkillSource
REPO_ROOT = Path(__file__).resolve().parents[1]
def test_optional_skill_source_scans_adversarial_ux_test():
source = OptionalSkillSource()
metas = {meta.identifier: meta for meta in source._scan_all()}
assert "official/dogfood/adversarial-ux-test" in metas
assert metas["official/dogfood/adversarial-ux-test"].name == "adversarial-ux-test"
assert "tech-resistant user" in metas["official/dogfood/adversarial-ux-test"].description
def test_optional_skill_catalog_docs_list_adversarial_ux_test():
optional_catalog = (REPO_ROOT / "website" / "docs" / "reference" / "optional-skills-catalog.md").read_text(encoding="utf-8")
bundled_catalog = (REPO_ROOT / "website" / "docs" / "reference" / "skills-catalog.md").read_text(encoding="utf-8")
assert "**adversarial-ux-test**" in optional_catalog
assert "official/dogfood/adversarial-ux-test" in optional_catalog
assert "`adversarial-ux-test`" in bundled_catalog
assert "dogfood/adversarial-ux-test" in bundled_catalog

View File

@@ -16,7 +16,6 @@ For example:
```bash
hermes skills install official/blockchain/solana
hermes skills install official/dogfood/adversarial-ux-test
hermes skills install official/mlops/flash-attention
```
@@ -57,12 +56,6 @@ hermes skills uninstall <skill-name>
| **blender-mcp** | Control Blender directly from Hermes via socket connection to the blender-mcp addon. Create 3D objects, materials, animations, and run arbitrary Blender Python (bpy) code. |
| **meme-generation** | Generate real meme images by picking a template and overlaying text with Pillow. Produces actual `.png` meme files. |
## Dogfood
| Skill | Description |
|-------|-------------|
| **adversarial-ux-test** | Roleplay the most difficult, tech-resistant user for a product — browse in-persona, rant, then filter through a RED/YELLOW/WHITE/GREEN pragmatism layer so only real UX friction becomes tickets. |
## DevOps
| Skill | Description |

View File

@@ -59,12 +59,9 @@ DevOps and infrastructure automation skills.
## dogfood
Internal dogfooding and QA skills used to test Hermes Agent itself.
| Skill | Description | Path |
|-------|-------------|------|
| `dogfood` | Systematic exploratory QA testing of web applications — find bugs, capture evidence, and generate structured reports. | `dogfood/dogfood` |
| `adversarial-ux-test` | Roleplay the most difficult, tech-resistant user for a product — browse in-persona, rant, then filter through a RED/YELLOW/WHITE/GREEN pragmatism layer so only real UX friction becomes tickets. | `dogfood/adversarial-ux-test` |
| `hermes-agent-setup` | Help users configure Hermes Agent — CLI usage, setup wizard, model/provider selection, tools, skills, voice/STT/TTS, gateway, and troubleshooting. | `dogfood/hermes-agent-setup` |
## email