Tag: llm
All the articles with the tag "llm".
-
How to build an eval suite
In this article I'm going to explain how to go about building an eval suite. In particular, I'm going to walk through the different layers you need to think about when constructing a reliable, well-covered set of evals.
-
Why Streaming LLMs Need Attention Sinks
A walkthrough of attention sinks: what they are, why softmax produces them by accident, why naive sliding-window inference collapses without them, and how a four-token reservation lets streaming inference run to four million tokens with no quality loss.