Alexander Whitestone ebf69d155b feat: GPU Inference Scheduler — Multi-Model Resource Management ...

Fixes #645

Queue-based model loading with priority lanes and VRAM budget tracking.
Prevents GPU OOM crashes when multiple projects compete for VRAM.

## Features

### Priority Lanes
- REALTIME (1): LPM, live video, interactive sessions
- INTERACTIVE (2): Playground, chat, user-facing
- BATCH (3): Harvester, overnight jobs, background

### VRAM Management
- Tracks total/used/available VRAM
- Reserves VRAM when job starts
- Releases VRAM when job completes
- CPU fallback when GPU full

### Model Registry
Pre-registered models:
- Video Forge: SD XL (8GB), HeartMuLa (4GB), Wan2.1 (12GB)
- LPM: Video Gen (16GB), A2A (8GB)
- Local: Llama 3 70B (40GB), Llama 3 8B (8GB), MiMo v2 Pro (16GB)
- Playground: SDXL Turbo (6GB)

### Cross-Project Scenarios Handled
1. Video Forge batch + LPM live → LPM gets priority
2. 3 Video Forge jobs → Sequential with shared cache
3. Night harvester + playground → Batch runs on idle cycles

## Files
- tools/gpu_scheduler.py: InferenceScheduler class, CLI interface
- tests/tools/test_gpu_scheduler.py: 19 tests, all passing

## Usage
```python
from tools.gpu_scheduler import InferenceScheduler, Priority

scheduler = InferenceScheduler(vram_budget_mb=49152)  # 48GB
scheduler.submit_job("job-1", "lpm", "llama3_8b", Priority.REALTIME)
job = scheduler.get_next_job()
scheduler.start_job(job)
# ... do inference ...
scheduler.complete_job(job)
```

2026-04-14 21:15:58 -04:00

..

__init__.py

…

test_ansi_strip.py

fix: strip ANSI at the source — clean terminal output before it reaches the model

2026-03-23 07:43:12 -07:00

test_approval.py

fix: block agent from self-destructing gateway via terminal (#6666)

2026-04-14 15:43:31 -07:00

test_base_environment.py

feat(environments): unified spawn-per-call execution layer

2026-04-08 17:23:15 -07:00

test_browser_camofox_persistence.py

fix: remove 115 verified dead code symbols across 46 production files

2026-04-10 03:44:43 -07:00

test_browser_camofox_state.py

fix(tests): fix 78 CI test failures and remove dead test (#9036)

2026-04-13 10:50:24 -07:00

test_browser_camofox.py

fix: remove 115 verified dead code symbols across 46 production files

2026-04-10 03:44:43 -07:00

test_browser_cdp_override.py

feat: switch managed browser provider from Browserbase to Browser Use (#5750)

2026-04-07 08:40:22 -04:00

test_browser_cleanup.py

fix(doctor): only check the active memory provider, not all providers unconditionally (#6285)

2026-04-08 13:44:58 -07:00

test_browser_console.py

fix: add browser_console to browser toolset and core tools list (#1084)

2026-03-17 02:02:57 -07:00

test_browser_content_none_guard.py

fix(browser): guard LLM response content against None in snapshot and vision (#3642)

2026-03-28 17:25:04 -07:00

test_browser_hardening.py

fix(browser): hardening — dead code, caching, scroll perf, security, thread safety

2026-04-10 13:05:44 -07:00

test_browser_homebrew_paths.py

fix(browser): add termux PATH fallbacks

2026-04-14 16:55:55 -07:00

test_browser_orphan_reaper.py

fix: reap orphaned browser sessions on startup (#7931)

2026-04-11 14:02:46 -07:00

test_browser_secret_exfil.py

fix: rewrite test mock secrets and add redaction fixture

2026-04-01 12:03:56 -07:00

test_browser_ssrf_local.py

fix(browser): skip SSRF check for local backends (Camofox, headless Chromium) (#4292)

2026-03-31 10:40:13 -07:00

test_budget_config.py

test(tools): add unit tests for budget_config module

2026-04-11 02:58:48 -07:00

test_checkpoint_manager.py

fix: normalize checkpoint manager home-relative paths

2026-04-11 14:50:44 -07:00

test_clarify_tool.py

…

test_clipboard.py

feat(gateway): WSL-aware gateway with smart systemd detection (#7510)

2026-04-10 21:15:47 -07:00

test_code_execution.py

fix: resolve remaining 4 CI test failures (#9543)

2026-04-14 02:18:38 -07:00

test_command_guards.py

fix: remove 115 verified dead code symbols across 46 production files

2026-04-10 03:44:43 -07:00

test_config_null_guard.py

fix: guard config.get() against YAML null values to prevent AttributeError (#3377)

2026-03-27 04:03:00 -07:00

test_credential_files.py

fix: remove 115 verified dead code symbols across 46 production files

2026-04-10 03:44:43 -07:00

test_cron_prompt_injection.py

…

test_cronjob_tools.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_daytona_environment.py

fix: update tests for unified spawn-per-call execution model

2026-04-08 17:23:15 -07:00

test_debug_helpers.py

…

test_delegate_toolset_scope.py

fix(security): restrict subagent toolsets to parent's enabled set (#3269)

2026-03-26 14:50:26 -07:00

test_delegate.py

feat(delegation): add configurable reasoning_effort for subagents

2026-04-10 21:16:53 -07:00

test_docker_environment.py

fix(tests): fix several failing/flaky tests on main (#6777)

2026-04-09 13:17:06 -07:00

test_docker_find.py

fix: Docker backend fails when docker is not in PATH (macOS gateway)

2026-03-10 20:45:13 -07:00

test_env_passthrough.py

fix: remove 115 verified dead code symbols across 46 production files

2026-04-10 03:44:43 -07:00

test_file_operations_edge_cases.py

fix(tools): remove dead code in _is_likely_binary and harden _check_lint against brace paths

2026-04-10 21:16:53 -07:00

test_file_operations.py

fix(patch): harden V4A patch parser and fuzzy match — 9 correctness bugs

2026-04-10 16:47:44 -07:00

test_file_read_guards.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_file_staleness.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_file_sync_perf.py

test: add reproducible perf benchmark for file sync overhead

2026-04-10 03:01:46 -07:00

test_file_sync.py

test(file_sync): add tests for bulk_upload_fn callback

2026-04-10 21:14:32 -07:00

test_file_tools_live.py

feat(environments): unified spawn-per-call execution layer

2026-04-08 17:23:15 -07:00

test_file_tools.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_file_write_safety.py

fix(file_tools): block /private/etc writes on macOS symlink bypass

2026-04-13 05:15:05 -07:00

test_force_dangerous_override.py

fix(skills): honor policy table for dangerous verdicts

2026-03-14 11:27:02 -07:00

test_fuzzy_match.py

fix(patch): harden V4A patch parser and fuzzy match — 9 correctness bugs

2026-04-10 16:47:44 -07:00

test_gpu_scheduler.py

feat: GPU Inference Scheduler — Multi-Model Resource Management

2026-04-14 21:15:58 -04:00

test_hidden_dir_filter.py

…

test_homeassistant_tool.py

fix: clean up description escaping, add string-data tests

2026-04-13 04:45:07 -07:00

test_interrupt.py

fix: resolve remaining 4 CI test failures (#9543)

2026-04-14 02:18:38 -07:00

test_llm_content_none_guard.py

fix: guard aux LLM calls against None content + reasoning fallback + retry (salvage #3389) (#3449)

2026-03-27 15:28:19 -07:00

test_local_env_blocklist.py

fix: add macOS Homebrew paths to browser and terminal PATH resolution

2026-03-23 22:45:55 -07:00

test_local_tempdir.py

fix(termux): honor temp dirs for local temp artifacts

2026-04-09 16:24:53 -07:00

test_managed_browserbase_and_modal.py

refactor(tests): re-architect tests + fix CI failures (#5946)

2026-04-07 17:19:07 -07:00

test_managed_media_gateways.py

test(tools): isolate approval and audio gateway env

2026-04-11 02:03:20 -07:00

test_managed_modal_environment.py

feat(environments): unified spawn-per-call execution layer

2026-04-08 17:23:15 -07:00

test_managed_server_tool_support.py

fix(tests): fix several failing/flaky tests on main (#6777)

2026-04-09 13:17:06 -07:00

test_managed_tool_gateway.py

feat: switch managed browser provider from Browserbase to Browser Use (#5750)

2026-04-07 08:40:22 -04:00

test_mcp_dynamic_discovery.py

fix(mcp): make server aliases explicit

2026-04-14 17:19:20 -07:00

test_mcp_oauth.py

feat: implement MCP OAuth 2.1 PKCE client support (#5420)

2026-04-05 22:08:00 -07:00

test_mcp_probe.py

fix: remove stale test skips, fix regex backtracking, file search bug, and test flakiness

2026-04-04 10:18:57 -07:00

test_mcp_stability.py

fix: add vLLM/local server error patterns + MCP initial connection retry (#9281)

2026-04-13 18:46:14 -07:00

test_mcp_structured_content.py

fix(mcp): combine content and structuredContent when both present (#7118)

2026-04-10 03:44:35 -07:00

test_mcp_tool_issue_948.py

fix: remove stale test skips, fix regex backtracking, file search bug, and test flakiness

2026-04-04 10:18:57 -07:00

test_mcp_tool.py

fix(mcp): make server aliases explicit

2026-04-14 17:19:20 -07:00

test_memory_tool_import_fallback.py

fix(tools): keep memory tool available when fcntl is unavailable

2026-04-14 10:18:05 -07:00

test_memory_tool.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_mixture_of_agents_tool.py

refactor: tighten MoA traceback logging scope (#1307)

2026-03-14 07:53:56 -07:00

test_modal_bulk_upload.py

perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014)

2026-04-12 06:18:05 +05:30

test_modal_sandbox_fixes.py

fix: update tests for unified spawn-per-call execution model

2026-04-08 17:23:15 -07:00

test_modal_snapshot_isolation.py

fix(tests): update mocks for file sync changes

2026-04-10 03:01:46 -07:00

test_notify_on_complete.py

fix: suppress duplicate completion notifications when agent already consumed output via wait/poll/log (#8228)

2026-04-12 00:36:22 -07:00

test_osv_check.py

feat: OSV malware check for MCP extension packages (#5305)

2026-04-05 12:46:07 -07:00

test_parse_env_var.py

fix(docker): add explicit env allowlist for container credentials (#1436)

2026-03-17 02:34:35 -07:00

test_patch_parser.py

fix(patch): harden V4A patch parser and fuzzy match — 9 correctness bugs

2026-04-10 16:47:44 -07:00

test_process_registry.py

fix(gateway): propagate user identity through process watcher pipeline

2026-04-11 13:46:16 -07:00

test_read_loop_detection.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_registry.py

fix: make tool registry reads thread-safe

2026-04-13 23:52:32 -07:00

test_rl_training_tool.py

fix: call _stop_training_run on early-return failure paths

2026-03-10 17:09:51 -07:00

test_search_hidden_dirs.py

fix: exclude hidden directories from find/grep search backends (#1558)

2026-03-17 02:02:57 -07:00

test_send_message_missing_platforms.py

fix(tests): fix several failing/flaky tests on main (#6777)

2026-04-09 13:17:06 -07:00

test_send_message_tool.py

feat(cron): support Discord thread_id in deliver targets

2026-04-10 03:20:05 -07:00

test_session_search.py

test: add multi-word query tests for truncation match strategy

2026-04-13 04:54:42 -07:00

test_singularity_preflight.py

fix(tests): use case-insensitive regex in singularity preflight tests

2026-03-16 19:01:39 +03:00

test_skill_env_passthrough.py

fix: remove 115 verified dead code symbols across 46 production files

2026-04-10 03:44:43 -07:00

test_skill_improvements.py

feat(skills): size limits for agent writes + fuzzy matching for patch (#4414)

2026-04-01 04:19:19 -07:00

test_skill_manager_tool.py

refactor: extract shared helpers to deduplicate repeated code patterns (#7917)

2026-04-11 13:59:52 -07:00

test_skill_size_limits.py

feat(skills): size limits for agent writes + fuzzy matching for patch (#4414)

2026-04-01 04:19:19 -07:00

test_skill_view_path_check.py

…

test_skill_view_traversal.py

…

test_skills_guard.py

fix(skills): preserve trust for skills-sh identifiers + reduce resolution churn (#3251)

2026-03-26 13:40:21 -07:00

test_skills_hub_clawhub.py

fix: improve clawhub skill search matching

2026-03-14 23:15:04 -07:00

test_skills_hub.py

fix: update 6 test files broken by dead code removal

2026-04-10 03:44:43 -07:00

test_skills_sync.py

fix(skills): read name from SKILL.md frontmatter in skills_sync

2026-04-11 01:21:20 -07:00

test_skills_tool.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_ssh_bulk_upload.py

perf(ssh,modal): bulk file sync via tar pipe and tar/base64 archive (#8014)

2026-04-12 06:18:05 +05:30

test_ssh_environment.py

fix(tests): update mocks for file sync changes

2026-04-10 03:01:46 -07:00

test_symlink_prefix_confusion.py

…

test_terminal_exit_semantics.py

feat: add exit code context for common CLI tools in terminal results (#5144)

2026-04-04 16:57:24 -07:00

test_terminal_foreground_timeout_cap.py

fix: reject foreground timeout above cap instead of clamping

2026-04-10 02:58:54 -07:00

test_terminal_none_command_guard.py

fix(terminal): guard invalid command values

2026-04-08 21:37:51 -07:00

test_terminal_requirements.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_terminal_timeout_output.py

fix(terminal): preserve partial output when command times out (#3868)

2026-03-29 21:51:44 -07:00

test_terminal_tool_pty_fallback.py

feat: add tested Termux install path and EOF-aware gh auth

2026-04-09 16:24:53 -07:00

test_terminal_tool_requirements.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_terminal_tool.py

fix: handle empty sudo password and false prompts

2026-04-09 02:50:07 -07:00

test_threaded_process_handle.py

feat(environments): unified spawn-per-call execution layer

2026-04-08 17:23:15 -07:00

test_tirith_security.py

fix: send_animation metadata, MarkdownV2 inline code splitting, tirith cosign-free install (#1626)

2026-03-16 23:39:41 -07:00

test_todo_tool.py

fix(tools): enforce ID uniqueness in TODO store during replace operations

2026-04-11 16:22:50 -07:00

test_tool_backend_helpers.py

test(tools): add unit tests for tool_backend_helpers module

2026-04-11 02:58:48 -07:00

test_tool_call_parsers.py

refactor(tests): re-architect tests + fix CI failures (#5946)

2026-04-07 17:19:07 -07:00

test_tool_result_storage.py

fix(tools): neutralize shell injection in _write_to_sandbox via path quoting (#7940)

2026-04-11 14:26:11 -07:00

test_transcription_tools.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_transcription.py

feat(auth): same-provider credential pools with rotation, custom endpoint support, and interactive CLI (#2647)

2026-03-31 03:10:01 -07:00

test_tts_mistral.py

feat(tools): add Voxtral TTS provider (Mistral AI)

2026-04-11 01:56:55 -07:00

test_tts_speed.py

test(tts): add speed config tests for Edge, OpenAI, and MiniMax

2026-04-12 16:46:18 -07:00

test_url_safety.py

fix(security): add SSRF protection to vision_tools and web_tools (hardened)

2026-03-23 15:40:42 -07:00

test_vision_tools.py

refactor: remove dead code — 1,784 lines across 77 files (#9180)

2026-04-13 16:32:04 -07:00

test_voice_cli_integration.py

fix(tests): fix 78 CI test failures and remove dead test (#9036)

2026-04-13 10:50:24 -07:00

test_voice_mode.py

fix(termux): tighten voice setup and mobile chat UX

2026-04-09 16:24:53 -07:00

test_watch_patterns.py

feat: background process monitoring — watch_patterns for real-time output alerts

2026-04-11 03:13:23 -07:00

test_web_tools_config.py

Merge branch 'main' into rewbs/tool-use-charge-to-subscription

2026-03-31 08:48:54 +09:00

test_web_tools_tavily.py

fix(tests): fix several failing/flaky tests on main (#6777)

2026-04-09 13:17:06 -07:00

test_website_policy.py

fix: resolve 7 failing CI tests (#3936)

2026-03-30 08:10:14 -07:00

test_windows_compat.py

…

test_write_deny.py

…

test_yolo_mode.py

fix(gateway): scope /yolo to the active session

2026-04-10 03:38:44 -07:00

test_zombie_process_cleanup.py

fix(tests): fix 78 CI test failures and remove dead test (#9036)

2026-04-13 10:50:24 -07:00