Context Engineering

• Agents need context (e.g., instructions, external knowledge, tool feedback) to perform tasks

Context engineering is the art and science of filling the context window with just the right information at each step of an agent’s trajectory

• Common strategies
  • Writing context - saving it outside the context window to help an agent perform a task.
  • Selecting context - pulling it into the context window to help an agent perform a task.
  • Compressing context - retaining only the tokens required to perform a task.
  • Isolating context - splitting it up to help an agent perform a task.
• LangGraph is designed to support them

Context Engineering Defined

https://x.com/tobi/status/1935533422589399127

https://x.com/karpathy/status/1937902205765607626

https://blog.langchain.com/the-rise-of-context-engineering/

Definition

*[Context engineering is the] ”…delicate art and science of filling the context window with just the right information for the next step.”*

Analogy

• Karpathy: LLMs are a new kind of OS
  • LLM is CPU
  • Context window is RAM or “working memory” and has limited capacity to handle context
  • Curation of what fits into RAM is analogous to “context engineering” as mentioned above

Types of context

Umbrella discipline that captures a few different types of context: