An LLM can only reason about information inside its context window. It doesn't automatically know my docs, APIs, databases, Jira tickets or company data. Something has to decide what information gets shown to the model.
💡
That is the Context Layer.
Core Concepts
| Cue | Notes |
|---|---|
| Context Layer? | My current mental model is that the Context Layer is responsible for deciding what information enters the model's working memory. |
| Context Package? | The final collection of facts, documents, relationships, and tool results that gets sent to the model. Ideally only the information needed to answer the question. |
| Why not send everything? | More context isn't automatically better. It increases cost, latency and noise. Relevance matters more than volume. |
| Context Engineering? | The practice of deciding what information should enter the context window, what should be excluded, how sources should be ranked and how information should be structured. |
| Where can context come from? | Documents, databases, knowledge graphs, APIs, tools, memory systems, and other knowledge sources. |
| What's RAG? | Retrieval-Augmented Generation. Instead of stuffing everything into context, retrieve only the relevant pieces when needed. |
| What problem does RAG solve? | Large collections of information that won't fit inside the context window. |
| Typical RAG sources | Confluence pages, PDFs, Jira tickets, documentation, wikis, and databases. |
| What are embeddings? | Numerical representations of meaning. Similar concepts end up near each other in vector space. |
| Why do embeddings matter? | Most semantic search and RAG systems depend on embeddings to find relevant information. |
| vector database? | A database optimized for storing and searching embeddings. |
| Knowledge Graph? | A system that stores relationships between entities, not just the entities themselves. |
| Mental model for Knowledge Graphs | A database tells me a customer exists. A Knowledge Graph tells me how that customer relates to campaigns, products, segments, and other customers. |
| What problem does a Knowledge Graph solve? | Understanding how things connect and influence one another. |
| Is a Knowledge Graph the same as RAG? | No. RAG retrieves information. Knowledge Graphs model relationships. |
| Can RAG and Knowledge Graphs work together? | Yes. A Knowledge Graph can simply be another source used by the Context Layer. |
| What are tools? | Systems the model can call when it needs information it doesn't already have. |
| Examples of tools | Snowflake, Jira, GitHub, CRM systems, internal APIs and databases. |
| What is MCP? | Model Context Protocol. A standard way for AI systems to discover and interact with tools. |
| Why use tools instead of storing everything in context? | Some information changes constantly. It's usually better to fetch it on demand than try to keep it in memory. |
| Examples of tool usage | Current revenue, latest churn rate, open incidents, deployment status, customer profile lookups. |
| Separation of concerns | Facts should remain factual. Relationships belong in graphs. Live values come from tools. Interpretation happens in the model. |
Diagram
Knowledge Sources
│
├── Documents
├── Databases
├── Knowledge Graphs
├── APIs
└── Tools
↓
Context Layer
↓
Context Package
↓
Context Window
↓
LLM
↓
Answer
Summary
- Context Layer is the orchestrator.
- RAG retrieves information.
- Knowledge Graphs provide relationships.
- Tools provide real-time data.
- The Context Layer decides what enters the model's working memory.
- The LLM reasons over whatever makes it through.