Context management with LangGraph

01Why context management is the real product

02What "context" means here

• Must fit — model max input tokens (hard ceiling).
• Must matter — extra tokens can dilute attention ("lost in the middle").
• Must be cheap enough — every agent step re-sends most of the working set.
• Must be durable when needed — HITL and multi-day threads need resume without replaying the world.

03Six approaches (combine them)

04How LangGraph models context (three layers)

• Static runtime context — user id, tenant, DB connections, feature flags. Passed into invoke/stream. Immutable for that run. Not chat history.
• Dynamic state (short-term) — messages, summary, scratchpad, pending tool calls. Lives in the graph state. With a checkpointer, it persists across turns under a thread_id (resume, HITL, time-travel).
• Store (long-term) — user preferences, profiles, durable facts under namespace + key. Cross-thread. Recall into state when needed — don't paste the whole profile every turn.

05Why LangGraph makes this easy

06Pattern: trim and summarize in the graph

# Conceptual sketch — trim before the model call
from langchain_core.messages import trim_messages

def call_model(state, config):
    # Keep ~4k tokens of recent dialogue for the LLM
    window = trim_messages(
        state["messages"],
        strategy="last",
        token_counter=count_tokens,
        max_tokens=4000,
        start_on="human",
        include_system=True,
    )
    return {"messages": [llm.invoke(window)]}

# Summarize path: a node writes state["summary"], then call_model
# builds prompt = [System(summary)] + recent_messages
# Optional: RemoveMessage to shrink checkpointer working set after summarize.

• Trigger — message count, token estimate, or fraction of model max input.
• Keep — last N messages verbatim so the bot doesn't "forget" the last ask.
• Rolling summary — feed prior summary + new turns into the summarizer so it stays incremental.
• Cheaper model — summarize with a small model; reserve the frontier model for the user-facing reply.

07The #1 footgun: UI history vs LLM RAM

08Long-term memory with the store

# Conceptual — long-term preference recall
def load_prefs(state, config, *, store):
    user_id = config["configurable"]["user_id"]
    item = store.get(("users", user_id), "preferences")
    prefs = item.value if item else {}
    return {"prefs": prefs}  # later folded into the system message

def save_pref(state, config, *, store):
    user_id = config["configurable"]["user_id"]
    store.put(("users", user_id), "preferences", state["prefs"])
    return {}

• When to use store — prefs, profiles, learned procedures, cross-session facts.
• When to keep in state — this conversation's tool traces, current plan, HITL pending action.
• Production — PostgresStore (or equivalent) beside PostgresSaver.

09Worked example: support agent thread

1. Static context: tenant_id, tool clients, model name.
2. State: messages, summary, pending_refund.
3. Checkpointer: Postgres, thread_id = conversation_id.
4. After each tool call: if payload > 2KB, store in S3, replace message content with pointer + 3-line summary.
5. Before call_model: if tokens > 6k, run summarize node → update summary, trim working messages.
6. Store: user language + VIP flag under ("users", user_id).
7. UI: always reads append-only chat_messages table dual-written on each turn.
8. Refund tool: interrupt → human approves hours later → same thread resumes.

10Cost and performance levers

11Rapid-fire interview Q&A

12Interview Q&A by level

13Wrapping up