Comparisons / Haystack vs LangChain
Haystack vs LangChain: Which Agent Framework to Use?
Haystack haystack by deepset is a framework for building nlp and llm pipelines. LangChain langchain is the most popular agent framework. Here is how they compare — and what the same patterns look like in plain Python.
By the numbers
Haystack
24.7k
2.7k
Python
Apache-2.0
2019-11-14
deepset
LangChain
132.3k
21.8k
Python
MIT
2022-10-17
Harrison Chase
Sequoia Capital, Benchmark
$25M Series A (2023), $25M Series B (2024)
3.5M
LangSmith (observability), LangServe (deployment)
Yes
Used by: Notion, Elastic, Instacart
github.com/langchain-ai/langchain →GitHub stats as of April 2026. Stars indicate community interest, not necessarily quality or fit for your use case.
| Concept | Haystack | LangChain | Plain Python |
|---|---|---|---|
| Agent | Agent component with ChatGenerator, tool definitions, and message routing | AgentExecutor with LLMChain, PromptTemplate, OutputParser | A function that POSTs to /chat/completions and dispatches tool_calls |
| Tools | Tool dataclass with function reference, name, description, parameters schema | @tool decorator, StructuredTool, BaseTool class hierarchy | A dict of callables: tools = {"search": lambda q: ...} |
| Pipeline Architecture | Pipeline() with add_component() and connect() — a directed graph of typed components | — | A sequence of function calls: output = step_b(step_a(input)) |
| RAG / Retrieval | DocumentStore + Retriever + PromptBuilder + Generator wired in a Pipeline | — | Embed the query, search a list, inject matches into the prompt, call the LLM |
| Memory | ChatMessageStore with ConversationMemory component in pipeline | VectorStoreRetrieverMemory, ConversationEntityMemory | A messages list that persists outside the function |
| Deployment | Pipeline YAML serialization, Hayhooks REST server | — | A Python script behind FastAPI or any HTTP server |
| Agent Loop | — | AgentExecutor.invoke() with internal iteration | A while loop: call LLM, check for tool_calls, execute, repeat |
| Conversation | — | ConversationBufferMemory, ConversationSummaryMemory | A messages list that persists outside the function |
| State | — | LangGraph state channels with typed reducers | A dict updated inside the loop: state["turns"] += 1 |
| Guardrails | — | OutputParser, PydanticOutputParser, custom validators | Two lists of lambda rules checked before and after the LLM call |
What both do in plain Python
Every concept in the table above — agent, tools, loop, memory, state — maps to a handful of Python primitives: a function, a dict, a list, and a while loop. Both Haystack and LangChain wrap these primitives in their own class hierarchies and APIs. The underlying pattern is the same ~60 lines of code. The difference is how much ceremony each framework adds on top.
When to use Haystack
Haystack earns its complexity when you're building RAG pipelines with multiple retrieval stages, document processing, and production deployment needs. But for straightforward agents with a few tools, the plain Python version is simpler to write and debug.
What Haystack does
Haystack models NLP/LLM applications as directed graphs of components. You create a Pipeline, add components (retrievers, generators, converters, rankers), and connect their inputs and outputs. Each component is a Python class with a @component decorator and a run() method with typed inputs and outputs. The framework handles data routing between components, input validation, and pipeline serialization to YAML. Haystack ships with document stores (Elasticsearch, Qdrant, Pinecone, Weaviate), embedding models, converters for PDFs and HTML, and a ChatGenerator that wraps LLM API calls. The Agent component adds tool-using capabilities on top. For RAG pipelines with multiple retrieval stages, re-ranking, and document processing, the component model provides clear separation of concerns.
The plain Python equivalent
A Haystack Pipeline is a sequence of function calls. The Retriever is a function that takes a query, embeds it, and searches a list of documents. The Generator is a function that calls the LLM API. The PromptBuilder is string formatting. Connecting components is passing the output of one function as the input to the next. The Agent component is the same while loop every agent framework uses — call the LLM, check for tool_calls, dispatch, append results, repeat. Document stores are a list you search with cosine similarity, or a database query. YAML serialization is saving your config to a file. The entire RAG pipeline — embed query, retrieve documents, build prompt, call LLM — fits in about 30 lines. Add tool-using agent behavior and you're at 60.
When to use LangChain
LangChain adds value when you need production integrations (vector stores, specific LLM providers, deployment tooling). But if you want to understand what's happening — or your use case is straightforward — the plain Python version is easier to debug, modify, and reason about.
What LangChain does
LangChain provides a unifying interface across LLM providers, a class hierarchy for tools and memory, and orchestration via AgentExecutor and LangGraph. The core value proposition is interchangeable components: swap OpenAI for Anthropic by changing one class, plug in a vector store for retrieval, add memory without rewriting your loop. It also ships with dozens of integrations — document loaders, text splitters, embedding models, vector stores — that save you from writing boilerplate HTTP calls. For teams that need to compose many integrations quickly, this catalog is genuinely useful. The tradeoff is that you inherit a large dependency tree and a set of abstractions that sit between you and the actual API calls.
The plain Python equivalent
Every LangChain abstraction maps to a small piece of plain Python. AgentExecutor is a while loop that calls the LLM, checks for tool_calls in the response, executes the matching function from a tools dict, appends the result to a messages array, and repeats. Memory is a dict you inject into the system prompt. Output parsing is a function that validates the LLM's response before returning it. The entire agent — tool dispatch, conversation history, state tracking, guardrails — fits in about 60 lines of Python. No base classes, no decorators, no chain composition. Just a function, a dict, a list, and a loop. When something breaks, you read your 60 lines instead of navigating a class hierarchy.
Or build your own in 60 lines
Both Haystack and LangChain implement the same 8 patterns. An agent is a function. Tools are a dict. The loop is a while loop. The whole thing composes in ~60 lines of Python.
No framework. No dependencies. No opinions. Just the code.
Build it from scratch →