Tag: llm
All the articles with the tag "llm".
-
How to Build an Eval Suite
In this article, I'm going to explain how to build an eval suite. Specifically, I'm going to walk through the different things you need to think about when constructing a reliable set of evals.
-
Why Streaming LLMs Need Attention Sinks
If you evict the first few tokens from the KV cache, a streaming LLM can fall apart. Here is why that happens, and why holding on to four early tokens is enough to make sliding-window inference work.