forked from Rockachopa/Timmy-time-dashboard
feat: Multi-modal support with automatic model fallback
- Add MultiModalManager with capability detection for vision/audio/tools
- Define fallback chains: vision (llama3.2:3b -> llava:7b -> moondream)
tools (llama3.1:8b-instruct -> qwen2.5:7b)
- Update CascadeRouter to detect content type and select appropriate models
- Add model pulling with automatic fallback in agent creation
- Update providers.yaml with multi-modal model configurations
- Update OllamaAdapter to use model resolution with vision support
Tests: All 96 infrastructure tests pass
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
"""Infrastructure models package."""
|
||||
|
||||
from infrastructure.models.registry import (
|
||||
CustomModel,
|
||||
ModelFormat,
|
||||
ModelRegistry,
|
||||
ModelRole,
|
||||
model_registry,
|
||||
)
|
||||
from infrastructure.models.multimodal import (
|
||||
ModelCapability,
|
||||
ModelInfo,
|
||||
MultiModalManager,
|
||||
get_model_for_capability,
|
||||
get_multimodal_manager,
|
||||
model_supports_tools,
|
||||
model_supports_vision,
|
||||
pull_model_with_fallback,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
# Registry
|
||||
"CustomModel",
|
||||
"ModelFormat",
|
||||
"ModelRegistry",
|
||||
"ModelRole",
|
||||
"model_registry",
|
||||
# Multi-modal
|
||||
"ModelCapability",
|
||||
"ModelInfo",
|
||||
"MultiModalManager",
|
||||
"get_model_for_capability",
|
||||
"get_multimodal_manager",
|
||||
"model_supports_tools",
|
||||
"model_supports_vision",
|
||||
"pull_model_with_fallback",
|
||||
]
|
||||
|
||||
Reference in New Issue
Block a user