Challenge: What are the three biggest challenges for quality in the AI era?

1. We're moving fast, but thinking slow
Yes, AI speeds up our work. But, AI also turns off our brains. That is not good while testing.
2. More tests, less confidence
Yes, AI creates 100 tests in seconds. But, is this the correct set of 100 tests?
3. Nobody owns the bug anymore
Yes, AI created the bug, AI tested it, AI passed it. But, if it fails, then who is responsible?…...

From a practical QA perspective, three challenges I see are:

1.AI can create a false sense of coverage by generating many tests without ensuring meaningful validation.

2. It can amplify flakiness in already unstable systems, making failures harder to debug.

3. And over-reliance on AI risks reducing deep product understanding, which is critical for identifying real quality gaps

In the AI era, quality is no longer just about validating functionality — it’s about ensuring trust in systems that are inherently unpredictable.

First, AI systems are non-deterministic, so the same input can produce different outputs. This makes traditional “expected vs actual” validation insufficient, requiring us to define acceptable response boundaries instead.

Second, there’s the challenge of correctness vs plausibility — AI can generate outputs that sound convincing but are factually wrong, making validation much harder than in deterministic systems.

Third, the focus is shifting from just testing outputs to controlling behavior. Techniques like context engineering and structured system design are becoming essential to reduce unpredictability and ensure consistent, reliable results.

Ultimately, the biggest challenge is moving from testing features to building systems that are trustworthy at scale.

1. (Re-)defining the quality in the era of AI Engineering
2. Keeping up with the development of AI tools and effectively selecting/integrating useful ones
3. Volume + velocity

3 biggest challenges for quality in AI era can be following:

1.> Data Integrity:  What kind of data is fed to models and how much they are trained, it becomes difficult to verify and validate what’s given to the model.

2.> Biased Output: Output can’t be trusted because machine has produced result on the basis of what’s fed to it.So, it’s become difficult to validate output.

3.> Accountability & Transparency: Quality comes into a questionable state because, you need to work on a large data at once and if something fails it becomes difficult to pinpoint single use case and accountability and transparency comes into questionable form.

1. Data quality
2. Bias & Transparency 
3. Regulations & compliance

• Speed over substance:
  AI makes it easy to produce fast, but quality suffers when we skip thinking, refining, and depth.

• Sameness over originality:
  AI outputs tend to sound alike, making it harder to stand out with authentic, human ideas.

• Confidence over truth:
  AI can sound right even when it’s wrong, so maintaining accuracy and trust becomes harder.

You need someone who can test intelligence, behavior and unpredictability.

Take on Filip’s challenge for a chance to win! The lucky winner will walk away with a gift box from us!🎁

What are the three biggest challenges for quality in the AI era?

Drop your answers in the comments below. 

Practically, IMHO
- AI systems don’t produce consistent outputs for the same input, which makes traditional assertion-based testing ineffective. I approach this by focusing on intent-based validation and semantic checks instead of exact matches.

- There’s often no single correct answer in AI systems, making it difficult to define pass/fail criteria. I address this by using multi-dimensional evaluation looking at correctness, relevance, and factual accuracy along with human-in-the-loop validation where needed.

-System performance is heavily dependent on data, and it can degrade over time as data changes. I treat data as a first-class test artifact by versioning datasets, prompts, and models, and by continuously monitoring for drift and quality drops.

Three Challenges i see

1. Every Agent solutioning should be evaluated for Hallusinations, Toxicity, Bias and other such quality metrix. AI infused testing is a skill in itself. 
2. The response may differ for different models same set of steps as the reasoning ability varies from model to model. 
3. AI solutioning generally works well for prototyping and POC but need to be mindful while building the prod grade application where the system constaints are also to be considered.