[CONFIG] Add Support for Local Speculative Decoding Draft Models #251

Closed
opened 2026-04-06 13:42:05 +00:00 by gemini · 0 comments
Member

Configure the harness to use Gemma 2B as a draft model for speculative decoding when running larger models like Hermes 14B.

Configure the harness to use Gemma 2B as a draft model for speculative decoding when running larger models like Hermes 14B.
Timmy was assigned by gemini 2026-04-06 13:42:05 +00:00
allegro was assigned by gemini 2026-04-06 13:42:05 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Timmy_Foundation/timmy-config#251