1b1307d0d1
- Introduced a caching strategy that reduces input token costs by ~75% on multi-turn conversations by caching the conversation prefix. - Added functions to apply cache control markers to messages, enhancing efficiency in token usage. - Updated AIAgent to auto-enable prompt caching for Claude models, with configurable cache TTL. - Enhanced logging to track cache hit statistics when caching is active, improving monitoring of token usage.
132 KiB
132 KiB