Who wants free courses? 👀 Help us pick the topics!

1: From Test Execution - System
Validation
Shift mindset first.
Learn:
Tokens, embeddings, temperature
Determinism vs variability
Why traditional testing breaks for GenAl
Core failure modes: hallucination, bias, prompt
sensitivity, latency & cost instability
Hands-on:
Build an LLM prompt-response evaluator
Compare outputs across temperature changes
Log prompts, responses & metadata
 2 - LLM Evaluation & Metrics (The Core
Upgrade)
GenAl quality * pass/fail.
Learn evaluation dimensions:
Correctness
Faithfulness
Relevance
Context recall
Accuracy vs usefulness
Hands-on:
Build automated evaluation pipelines
Run batch prompt experiments
Compare model versions objectively
Tools:
RAGAS . DeepEval . Braintrust
Deliverable: Your first LLM evaluation report
3 -- RAG & Knowledge Reliability Testing
Most hallucinations are retrieval failures, not model
failures.
Learn:
RAG architecture & failure points
Chunking issues
Embedding mismatch
Retrieval drift
Hands-on:

Test retrieval precision & recall
Inject corrupted documents
Validate answer faithfulness to sources
A now validates data pipelines, not just app logic
4 - Observability & Production Readiness
Logs are not enough for LLMs.
Learn:
Prompt lineage & versioning
Behavior drift detection
Tracing prompt tool - response chains
Hands-on:
Detect latency spikes & token explosions
Compare behavior across deployments
Tools:
LangSmith . Arize
Deliverable: Production-ready GenAl observability
dashboard
5-- Safety, Guardrails & Risk-Based Testing
This is where GenAl becomes enterprise critical.
Learn:
Data leakage risks
Unsafe instructions
Compliance & policy testing
Prompt fixes vs system controls
Hands-on:
Build red-team prompt suites
Validate refusal behavior
Test boundary violations
Tools:
Guardrails Al + policy-as-code checks

 6 - Testing Al Agents (The Future of QA)
Agents plan, act, fail, retry.. and surprise you.
Learn:
Agent architectures (planner, executor, memory)
Non-deterministic workflows
Why step-based test cases fail
Hands-on:
Test multi-step agents for:
Goal completion rate
Unsafe action rate
Recovery from failure
Human-in-the-loop gates