What is your biggest pain point when it comes to automation testing?

Selection of Proper Test Cases/Scenarios
Requirement Clarity 
expectations from the stakeholders

script maintenance, UI Changes

Scaling the automation FW with the never ending new features being added to the product.

Tech debt in maintaining the automation framework due to everchanging dependencies and their new versions.

• Script maintenance is extremely time consuming
• Flaky tests decreases the reliability in automation
• Shrinking size of the QA team within development teams along with QAs being hybrid that do both manual and Automation and hence left with very less time to accomplish in-sprint testing and no time to take care of test backlog/test maintenance.

​@Mukesh_Otwani  ​@Mustafa 

Based on my experience in automation testing, sharing my answer:

I would say my biggest pain point in automation testing is dealing with test flakiness and maintenance, particularly in our web application testing suite.

Let me explain the main challenges I face:

Test Flakiness: The most frustrating issue is when tests pass in one run but fail in another without any actual changes to the application. This often happens due to:

•    Dynamic loading times causing timing issues
•    Network latency in API calls
•    Browser rendering differences
•    Async operations not being handled properly

I've tried several approaches to handle this:

1.    Added smart waits instead of hard-coded sleeps
2.    Improved element synchronization
3.    Implemented retry mechanisms for flaky operations

But even with these solutions, maintaining stable tests requires constant attention.

->Another significant challenge is keeping up with frequent UI changes. When developers update the application:

•    Locators break unexpectedly
•    New features get added without proper test coverage planning
•    Existing workflows change, making test scenarios outdated

For example, last month our team updated our React framework version, which changed how some elements were rendered in the DOM. This broke about 30% of our test cases, and it took considerable effort to update all the locators and test logic.

To minimize these issues, I've started:

•    Using more robust locator strategies (like data-testid attributes)
•    Having regular sync-ups with developers about upcoming changes
•    Implementing a modular framework design where common elements are maintained in one place

However, this is still a work in progress, and it requires constant collaboration with the development team.
 

Flaky Tests, TDM, Finding the Right Framework & Tools, Initial Investment, Skill Gaps, sometimes due to Unrealistic Expectations, etc.

There are so many reasons test automation may fail, biggest issue is always a lack of knowledge of testing and automation.

Next to that is that most applications are not designed for testability and in some situations the developers are unwilling to adapt the code to improve observability.

Other issue is that the amount of test scripts is only growing. Test wear out, especially regression tests. So keep on identificating (residual) risks and adapt your test strategy to those.

Another issue is the underestimating of the amount of work test automation will take, especially if management want to test everything automated. Which results in a lot of maintenance and reduced added value.

Last remark: it depends ….

To choose the better tool , resource selection based on the tool and more importantly the type of script we are going to write for the project.

It was really great session and depicted many unsung areas to cover like automation plan, understand, filter realistic test cases, end goal of automation, ROI etc. It should be adapted across the team then only automation will fetch ROI else it will be POC only.

There are many challenges starting to end. To name a few like 

1. Automation is considered as silver bullet by many leaders on paper who have not practical experience to technology field. Expecting 100% automation which is not even practical any where.
2. Application is not stable and undergoing the repetitive changes every now and then.
3. App is not functionally stable.
4. Before moving to UI automation, API layer is not stable enough.
5. API should be the first priority to automate then UI.
6. Time allocated to automation activities are very less, unrealistic expectation to keep covering everything within sprint 
7. No time allocated to maintain the script
8. Flaky tests

• Maintaining automation scripts
• Reducing flaky test count
• Automating complex, multi-feature scenarios with intricate prerequisites.

My biggest pain point in automation testing is the constant need to maintain and update test scripts. As the application evolves with new features and UI changes, the tests become brittle and break frequently.

This requires significant effort to identify the broken tests, understand the changes in the application, and then modify the scripts accordingly.

This ongoing maintenance can be time-consuming and resource-intensive, often overshadowing the initial benefits of automation.

Ignorance and Misunderstanding by management of the purpose and process of "automation", This causes them to make decisions based on criteria that lead to low ROI. Then they do what all management do which is blame the testers/developers for failing to implement correctly. Managing leadership expectations is a far underutilised skill, and training generally focuses on "hard" skills such as language or framework knowledge rather than soft skills. Ironically, and contrary to the physical world, good automation stands stronger on a foundation of soft rather than hard skills.

Biggest challange I face is maintanance of script. Too many changes on UI/UX, unstale code by developers.

Also, limited resources. I am the only person who is doing all things for multiple projects at once and trying to manage all. I always miss something or else due to this.

The hardest thing is maintaining test automation so that the locators are targeting the correct thing.  About 5% of the test suite ends up failing even though no UI has changed in the app we are testing.

As per my opinion, below are few pain points of automation testing:

• Environment Stability: Ensuring that the test environment mirrors the production environment and is stable can be an ongoing challenge.
• Maintenance Overhead: As the application evolves, test scripts often require frequent updates. This can become quite labor-intensive, especially if the test suite is extensive.
• Flaky Tests: Intermittent test failures due to timing issues or dependencies on external systems can be a major frustration, as they undermine confidence in the test results.
• Changes in Requirements: Rapid changes in project requirements can lead to instability and require continuous updates to the test scripts, making it challenging to maintain a robust test automation framework.

Thank you everyone for your answers and contributions.

We’re happy to announce that the winner of this competition is ​@hrdave. I’ve already gotten in touch with him over the email to arrange his prize.

Congratulations to you ​@hrdave  and better luck next time for everyone else. Thank you!

Thank you ​@Mustafa and Tricentis for organizing such thoughtful webinar. It was truly awesome and insightful. I am glad to be part of such community. Thanks once again for wonderful gift and look forward to participate more in near future. 

Hello ​@Mukesh_Otwani ,

From my experience, developers often finish tickets on the last day of the sprint, which makes it extremely hectic to cover all scenarios within the limited time.

In my opinion, they need to prioritize tasks accordingly based on the story points.

Cheers,

Ramanan