Design your AI agent use-case in testing

AI Agent Use Case – “ReproGenie”

Real Testing Problem - Recurring Pain:

Writing high-quality, reproducible bug reports from exploratory or regression testing sessions.

Especially in:

• Complex business logic
• Mobile UI issues
• API mismatches between Android & iOS
• Edge-case failures after regression runs

Common issues:

• Steps are incomplete
• Logs/screenshots not properly attached
• Environment details missing
• Reproducibility inconsistency
• Back-and-forth with devs

5W1H Framework

WHAT – What will the agent do?

ReproGenie will:

• Convert raw tester inputs (notes, logs, screen recordings, console output)
• Into a clean, structured, dev-ready bug report

It will:

• Extract reproduction steps
• Detect missing info
• Identify environment details
• Suggest expected vs actual behavior
• Classify severity
• Attach relevant logs
• Cross-check if similar bug exists
• Suggest possible impacted modules

WHY – Why should this be an agent?

Business Value

• Faster bug resolution
• Reduced dev clarification loops
• Cleaner Jira backlog
• Improved sprint predictability
• Better regression traceability

Practitioner Value (YOU)

As a tester:

• Saves 20–30% reporting time
• Improves credibility
• Reduces cognitive load after long test cycles
• Maintains consistency across releases
• Helps junior testers improve quality

Especially useful in:

• Complex feature areas
• Multi-platform testing
• Animated UI automation issues

WHEN - Agency Level?

Semi-autonomous Agent

Why not fully autonomous?

Because:

• Bug severity sometimes needs human judgment
• Business context matters
• Reproducibility must be verified

So flow is:

• Tester → Agent draft → Tester review → Submit

WHERE – Where in SDLC/STLC?

It fits in:

• During Exploratory Testing
• During Regression Testing
• After Automation Failures
• During UAT bug triage

Specifically:

• Between Test Execution → Defect Logging

WHO – Who controls or reviews it?

Primary: QA Engineer

Secondary:

• QA Lead
• Product Owner (if high severity)

Agent never submits automatically without review.

HOW – Rough Architecture

Core Components

1. LLM Layer
   • Parses natural tester notes
   • Extracts structured steps
   • Rewrites for clarity
2. Deterministic Layer
   • Template enforcement
   • Severity matrix logic
   • Required fields validation
   • Duplicate check logic
3. Tools & APIs
   • Jira API
   • Appium logs ingestion
   • Android logcat parsing
   • API response capture
   • Git commit linking
4. Memory
   • Stores:
     • Past bugs
     • Similar module failures
     • Known flaky areas
   • Improves classification over time

Agent Name

ReproGenie

Architecture Sketch (Lightweight)

Tester Input (Notes / Logs / Screenshot)
          ↓
Input Parser
          ↓
LLM (Structure + Clarify + Improve)
          ↓
Validation Engine (Missing info? Required fields?)
          ↓
Duplicate Detector (Jira API check)
          ↓
Severity Engine (Rule + Context based)
          ↓
Draft Bug Report
          ↓
QA Review → Submit

Failure Modes & Guardrails

Mini Prototype Snippet (PoC Idea)

Example prompt structure:

prompt = f"""
You are a QA assistant.

Convert the following raw tester notes into a structured bug report.

Notes:
{tester_notes}

App Version:
{version}

Environment:
{environment}

Ensure:
- Clear reproduction steps
- Expected vs Actual
- Pre-conditions
- Attach log suggestions
- No hallucinations
"""

Enhancement:

• Add Jira API integration
• Add log similarity detection
• Add severity rule engine

Advanced Version (Future Roadmap)

• Auto-watch failed Appium test runs
• Convert failure stack trace → human-readable repro
• Identify flaky vs real bug
• Suggest impacted regression areas
• Generate negative test cases automatically

🤖 RegGuard — AI Agent for Manual Regression Testing

The Real Problem I work as a Manual QA Engineer on a fintech mobile app. Every release, we run regression testing manually across Stage, Beta, and Production. We have critical financial flows — where even a one rupee mismatch can cause compliance failures or financial loss.

Some of these flows are automated and covered in our CI/CD pipeline. But several critical flows cannot be automated payment behaviour, real transaction edge cases, and UI flows that change every sprint. These are mandatory checks before every release. A human must verify them every single time.

The problem? Test cases exist in Zephyr but testers skip them and bulk-mark everything as PASS from memory. No one catches it. A mandatory critical payment flow gets a PASS without anyone actually testing it. And that's how compliance risks reach production in a fintech product — silently.

WHAT will it do? RegGuard monitors test execution in Zephyr, detects when someone bulk-marks test cases suspiciously fast, checks coverage across all 3 environments, and generates a report for the QA Lead before sign-off. It also uses AI to read the sprint PRD and generate a mandatory test checklist — so no critical flow is ever missed.

WHY should it be an agent? It combines time-based detection rules + AI-powered PRD analysis + Zephyr integration. No single tool does all of this. It saves hours of manual policing and gives the QA Lead objective, evidence-based coverage data before every release — especially critical in a fintech product where missing a test is not just a quality issue, it is a compliance risk.

WHEN — Agency Level? Semi-Autonomous. RegGuard monitors and alerts automatically. The QA Lead reviews and makes the final release decision.

WHERE in the STLC? Test Execution → Coverage Gate → QA Sign-off. Active across Stage, Beta, and Production.

WHO controls it? QA Lead reviews all alerts and approves release. Testers receive real-time nudges and mandatory checklists before execution begins.

HOW does it work? JavaScript (Node.js) for bulk-mark detection and coverage reporting. Claude.ai / ChatGPT for reading PRDs and generating test checklists. Zephyr API for test case data. All free tools — zero budget. No coding needed for daily use.

Failure Modes

• False positives on fast testers → QA Lead can whitelist sessions
• Testers gaming the system → randomised evidence spot-checks required
• Alert fatigue → daily digest mode after week one

Bonus — Working Prototype Built and running. All 3 phases work with one command: node regguard.js

🔗 github.com/Sahithya5as/AI-for-Testers/tree/main/RegGuard

All yours — just copy and paste! 🚀

🤖 RegGuard — AI Agent for Manual Regression Testing

The Real Problem I work as a Manual QA Engineer on a fintech mobile app. Every release, we run regression testing manually across Stage, Beta, and Production. We have critical financial flows  where even a one rupee mismatch can cause compliance failures or financial loss.

Some of these flows are automated and covered in our CI/CD pipeline. But several critical flows cannot be automated real transaction edge cases, and UI flows that change every sprint. These are mandatory checks before every release. A human must verify them every single time.

The problem? Test cases exist in Zephyr but testers skip them and bulk-mark everything as PASS from memory. No one catches it. A mandatory critical payment flow gets a PASS without anyone actually testing it. And that's how compliance risks reach production in a fintech product — silently.

WHAT will it do? RegGuard monitors test execution in Zephyr, detects when someone bulk-marks test cases suspiciously fast, checks coverage across all 3 environments, and generates a report for the QA Lead before sign-off. It also uses AI to read the sprint PRD and generate a mandatory test checklist — so no critical flow is ever missed.

WHY should it be an agent? It combines time-based detection rules + AI-powered PRD analysis + Zephyr integration. No single tool does all of this. It saves hours of manual policing and gives the QA Lead objective, evidence-based coverage data before every release — especially critical in a fintech product where missing a test is not just a quality issue, it is a compliance risk.

WHEN — Agency Level? Semi-Autonomous. RegGuard monitors and alerts automatically. The QA Lead reviews and makes the final release decision.

WHERE in the STLC? Test Execution → Coverage Gate → QA Sign-off. Active across Stage, Beta, and Production.

WHO controls it? QA Lead reviews all alerts and approves release. Testers receive real-time nudges and mandatory checklists before execution begins.

HOW does it work? JavaScript (Node.js) for bulk-mark detection and coverage reporting. Claude.ai / ChatGPT for reading PRDs and generating test checklists. Zephyr API for test case data. All free tools — zero budget. No coding needed for daily use.

Failure Modes

• False positives on fast testers → QA Lead can whitelist sessions
• Testers gaming the system → randomised evidence spot-checks required
• Alert fatigue → daily digest mode after week one

Bonus — Working Prototype Built and running. All 3 phases work with one command: node regguard.js

🔗 github.com/Sahithya5as/AI-for-Testers/tree/main/RegGuard

All yours — just copy and paste! 🚀

Hi, everyone!

Thank you all for all of your submissions, all of them have been creative and interesting.

However, there can be only one winner. And I’m happy to announce today that the winner is ​@dharmendratak 

Congratulations, Dharmendra. You deserve it.

Thank you everyone for participating in the challenge, and stay tuned for more exciting events and challenges. 

Hello, congratulations @dharmendratak, 🥳good luck👏👏👏

Hi ​@Mustafa , thank you for this honour. I am glad I could do something good here.