Users with multiple local servers or custom endpoints can now define
them all in config.yaml and switch between them from the model
selection menu:
custom_providers:
- name: 'Local Llama 70B'
base_url: 'http://localhost:8000/v1'
api_key: 'not-needed'
- name: 'RunPod vLLM'
base_url: 'https://xyz.runpod.ai/v1'
api_key: 'rp_xxxxx'
These appear in `hermes model` provider selection alongside the
built-in providers. When selected, the endpoint's /models API is
probed to show available models in a selection menu.
Previously only a single 'Custom endpoint' option existed, requiring
manual URL entry each time you wanted to switch between local servers.
Requested by @ZiarnoBobu on Twitter.
069570d103