Chapter 21 · LangChain LangChain Track CLI Project

A test designer that checks what already exists before writing case one.

JIRA story in, structured TestPlan out. The agent fetches the story from JIRA, searches the QA Copilot knowledge base for similar existing test cases, then designs new coverage that reuses your conventions and skips near-duplicates.

01 What students build

The designer from the LangChain production trio. fetch_jira_story pulls the story via JIRA REST v3 and flattens Atlassian Document Format into plain text. qa_copilot_search retrieves the top-k most similar items from a local Chroma knowledge base of 200+ VWO test cases and bug precedents (embedded with Ollama nomic-embed-text). Groq's llama-3.3-70b-versatile with with_structured_output then emits a typed Pydantic TestPlan: feature, scope, risks, and concrete cases with steps and expected results.

JIRA Native

Fetches the real story by key (project VWO), including type, components and description, with an ADF-to-text flattener for the v3 API.

RAG Grounded

Prior test cases from the QA Copilot KB go straight into the prompt, so new cases follow existing ids, style and team conventions.

Typed TestPlan

Pydantic structured output: every case has id, type, priority, preconditions, steps and expected result. No free-text mush.

02 Architecture

JIRA keyVWO-105
──▶
fetch_jira_storyREST v3 · ADF → text
──▶
qa_copilot_searchChroma · 200+ items
──▶
Designer chainGroq · structured output
──▶
TestPlanscope · risks · cases
story + prior cases in one prompt · reuse conventions · avoid duplicating known coverage

03 Live demo (simulated with sample data)

This is a UI walkthrough with pre-baked sample data so you can feel the flow without JIRA credentials or a local KB. In the course you run the real thing: JIRA REST + Chroma + Ollama embeddings + Groq.

python src/prod/test_designer_agent_prod.py VWO-105 demo · sample data

04 Inputs & outputs

What goes in

  • A JIRA story key (project VWO), e.g. VWO-105
  • JIRA URL, email and API token via .env
  • The QA Copilot KB (built once with python -m rag.ingest)
  • Top-k similar prior cases pulled automatically

What comes out

  • A typed TestPlan: feature, scope, risks
  • Concrete cases across Functional, Negative, Edge, Security
  • P0-P2 priorities, preconditions, numbered steps, expected results
  • Convention reuse: ids and style match the existing suite
  • Near-duplicates of KB cases skipped, not rewritten

05 How it's built

Build the knowledge base once: python -m rag.ingest chunks 200+ VWO test cases and bugs into a local Chroma DB via Ollama nomic-embed-text.
Fetch the story with JIRA REST v3 and flatten its Atlassian Document Format description into plain text.
Search the KB with the full story text as the query, pulling the top-5 most similar prior cases.
Define TestCase and TestPlan Pydantic models so the LLM must return valid structure.
Prompt the designer with BOTH the story and the prior cases: reuse conventions, avoid duplicates, no fluff.
Invoke chain = prompt | llm.with_structured_output(TestPlan) and print the plan case by case.

06 Tech stack

Python · LangChainGroq llama-3.3-70b-versatilePydantic structured outputJIRA REST v3 + ADF parserChroma vector DBOllama nomic-embed-text@tool functionsdotenv config