We're thrilled to announce our first blogathon , and we warmly invite you to participate and showcase your exceptional technical and writing skills!

This is not your average competition, folks. It's a chance to connect with like-minded individuals, flaunt your expertise, give your article the spotlight it deserves, and who knows, you might just snag some prizes along the way!

To encourage more people towards blog writing, ShiftSync is launching a one-month Blogathon contest from July 19, 2023, to August 18, 2023. 

Top 3 winners will get monetary prizes, badges, and certificates of recognition.  

• 🥇First prize – 300 USD + Swag 

• 🥈Second prize – 150 USD + Swag 

• 🥉Third prize – 50 USD + Book of the industry expert 

In addition, we will have some outstanding entries who will receive certificates of participation and ShiftSync badges (based on your points). 

 If you have any concerns or need assistance, feel free to reach out to shiftsync@tricentis.com or message @Kat or @Daria.

You must watch out for these dates: 

• Blog submission last date – July 30, 2023 

• ShiftSync assessment – August 16, 2023 

• Winner announcement – August 18, 2023 

After blog submission, you can vote for your fellow users’ blogs. The most popular blogs in terms of likes and comments will be shortlisted. Public voting is open for 14 days (about 2 weeks) after submission. In other words, if you submit the blog on July 19, your votes will be counted for 14 days, starting from July 19 to August 1. If you submit your blog on July 30, you will still get 14 days of voting, which means your votes will be counted from July 30 till August 12.  

Please make sure that you submit your blogs on or before July 30, 2023. Post July 30, your blogs won’t be accepted.  

ShiftSync will finalize the top 3 winners based on public voting.  

An opportunity for you all to create blogs on the new community and win prizes. We’re encouraging you to create some meaningful and engaging content with utmost brevity. Get ready to awaken your creative muse. Grab the opportunity to educate your fellow users as well as encourage them to be a part of it.  

How to be a part of this Blogathon? 

1. Submit your blog to the ShiftSync community. Please reply to the blogathon topic. Screenshot attached. 

 

2. Create your original blog posts (between 300-1000 words) and cover ONE of the following topics: 

• 📃Test Automation for Mobile Applications –This is a broad topic, so feel free to pick your own point of view: from how it contributes to company success or share some examples from your experience (both positive or challenges and learnings).
• 📃What's enough testing?
• 📃Integrating OAuth 2.0 in C# Microservices: A Step-by-Step Guide. 

3. Follow the rules:

• An author can submit multiple blogs; however, they must cover the topics listed above   
• The entry should be in English 
• The entry should not have multiple authors 
• Don't forget to mention the title of the blog that you chose  
• Please include images (not more than 5) to make your blog visually appealing. You can use images from pexels or unsplash 

Who can vote? 

• Anyone 
• You cannot vote your own blog but go ahead and activate your network and friends, have them vote for you and get your name and article to the top of the list! 

Doesn’t this topic sounds straightforward? People will say that when you stop finding bugs, the testing should stop, OR when all the tests have been executed and pass, testing is enough. Many other theories also can become answers. As we finish testing, or feel like it is done, new test ideas come up. Hence is it very very important to know when to stop testing. With my experience as a junior tester, senior tester and testing manager over 20 years, I have found few ways where you can find out if the testing that is performed is enough or not.

1. All the test ideas discussed in meetings, or provided by managers or tossed in between testing are implemented :-  You went through each of the sample scenarios that came with the story to test. This means that you only executed the test cases created by someone else. This might feels like something is left, often it is the case in teams with many seniors and junior testers. Junior testers are asked to remain within scope of testing, hence they know when to stop when the list of testing is complete.
2. Testing time is over :-  With all the SDLC, Agile and other practices in place, testing is often a time bound activity. As the race to Market intensifies, someone can always declare that time is up and the software needs to be shipped. Like it or not, we have a finite amount of time for testing, so these may be good indicators that it’s time to stop. In such cases, testing is done based on priority of test cases.
3. Testing is not produced desirable results:- Its been months of repeating same tests over and over again. Automation may have been implemented. If the management has stopped adding new features, the testing also need not be done as regressively as it was earlier. You know that even there is a bug hidden, it would be a very simple error hence no point wasting resources on testing. We can stop.
4. Tired testers:- Whether it is towards the end of day, or towards the end of product life, testers can feel exhausted and that is the time we should stop. If I cannot find a bug today after 5-6 hrs of testing, chances are that I am losing focus because I am mentally tired. I may want to stop testing , for today, and continue tomorrow. 
5.  Irrelevant Test ideas:- Sometimes the tests ideas that we put in front of management can be dismissed and we hear that these tests are not worthy and the product can live with any bug related to those tests. In such cases we can stop testing that particular functional area.

Apart from the general guidelines, there are several other situations which determine the extent and duration of software testing. After conducting series of tests you might be satisfied with the quality of codes or the bug percentage but the reality might be a completely different picture. The quality, bugs percentage etc. could be pointed in just the opposition direction. If you release the software in this state, it may prove expensive and detrimental to your business. You can control it by evaluating the possibility of finding more defects. This is inferred from the analysis of the results obtained so far. The analysis is important because

●       A newfound defect is a strong indication to continue testing as often they lead to other defects.

●       If you found defects with a small portion of overall functionality, you should continue the testing.

●       If multiple testing of the functionality of the software is not showcasing any defects, then it is the right time to stop testing the software.

●       Before putting a stop to software testing, you need to ensure that its significant features are mostly or entirely tested for satisfactory performance.

●       One should always rely on comprehensive tests and results and not based their decision on a false confidence veneer. It is not uncommon that many developers completely skip on tests stating that as long as the codes comply; it has reached the acceptance testing benchmark.

Success comes from identifying the risks early. It is a good indicator of when to stop software testing. The risk factors will determine your level of testing. If in various testing like Unit testing, System testing, Regression testing etc., you are getting positive results, then you can stop testing.

We can easily surmise that you can understand it is time to stop software testing when you are confident of finding any additional defects and when various tests give you a high degree of confidence that the software is ready to be released. 

Conclusion:

It can be easily seen that there are no predefined rules that you can use every time to find how much testing is enough. Visualize project situations, project readiness and timelines to decide on your own.

Why MAST is a must?

Mobile applications are a huge deal these days. Nowadays there is an app for almost everything. Want to know what was the song you’ve listened to on a radio recently? Or what is the nearest tram station to your location right now? Or with whom did your partner meet the other day when he/she claimed they went to the gym alone? Well, there is an app for that as well. But that is not the topic of today’s article. With over 255 billion mobile applications downloaded in 2022, there is just no denial that mobile applications are prevalent. This puts a lot of pressure on the mobile application testing. That doesn’t mean only the functional testing, but the non-functional testing as well. In today’s article, I’d like to talk about security testing and show an example of how an attacker can leverage a poorly tested application.

Introducing the problem

Imagine you are a small company that came up with the idea to create an app that would allow the users to rate the pubs and breweries they have visited. Due to the immense popularity of Android within your target user group, you have decided to create an application on Android, publish it on Google Play and have the users download the app from there.

Many people perceive the mobile applications as secure by nature. The developers usually write the Java (or Kotlin) code, compile it, bundle it together with other stuff and publish it as an executable file. An executable file is just the combination of unreadable mess for a normal human being. If something goes horribly wrong during the development, no one should be able to figure that out right? Wrong.

Let’s return back to our story. The company was able to publish the application, but it also attracted the attention of some shady guys on the internet. We will play the role of these individuals. The first thing we have to obtain is the application itself. Luckily, there are sites such as APKpure that allows us to obtain them.

But the file that gets downloaded is just an APK. In order to get to the code, the attacker would use a decompiler which is a software that allows us to the reverse process to code compilation. There are various decompilers available, but in this case a simple JADX (Dex to Java) decompiler should do the trick. What is great about this tool is that it also has a GUI version which makes the reverse engineering process much easier.

Having a code available, the attacker can start looking for interesting parts within the database. In this case, it might be the secrets. Secret is for example a password for a database or an API token the third party service you might be using. It is something you wouldn’t want any attacker to get their hands on. There are many ways for an attacker to look for secrets. One of them would be write a simple regex looking for strings with a given pattern (for example AWS tokens are usually in the form of „AKIA” followed by 16 characters that can be either numbers or capital letters). In this example, we have a commercial secret scanner.

A simple mistake that could have been prevented right in the development phase was discovered from the final executable. In the worst case scenario, this mistake could bring down the small company and their service. This puts a lot of emphacise on testing and more specifically an automated security testing.

MAST as solution

So now that we have explained a real-life scenario and showcased the problém, we should also speak about how to prevent it. And the prevention should start during the development by the usage of various automatic MAST tools. We have used the term MAST, but what exactly does it stand for?

MAST is „Mobile application security testing“ and it should be part of every organization’s mobile application development process. It consists of the combination of various tools that can be used automatically throughout the build and testing phase of the SDLC. Most of these tools can be seemlessly integrated into the CI/CD and based on their output, we can let the build fail for not meeting the security requirements.

Speaking about tools, let’s do a quick rundown of each tool category:

• Static application security testing (SAST) is used for scanning the application source code to identify the vulnerabilities. These tools can be run really early in your CI/CD or even as an IDE plugin while coding and are considered as „white box“ testing.
• Dynamic application security testing (DAST) is used for checking the security at runtime by testing common attack types against the running apps. In comparison to SAST, they can only be used much later in the development process and are an example of a „black box“ testing.
• Interactive application security testing (IAST) is used for checking security at runtime via application scanning and analysis of internal application flows. It is a blend of white box and blackbox testing as it links the foundings of DAST to the source code scanned by SAST.
• Software composition analysis (SCA) is used for tracking third-party code dependencies, which is useful when your solution relies on many open source libraries. Developers can use these tools to discover components, their supporting libraries and indirect dependencies that might result in vulnerabilities and possible exploits.

Ideally, you would like to put these tools into your development pipeline in the succession like this one:

Conclusion

With the rise of mobile applications, i tis vital to perform not only the functional testing, but the security testing as well. Considering the magnitude of attack vectors, the only feasible way forward is to use automated testing tools that can perform various scanning activities throughout your whole development phase.

What’s enough testing?

Testing: The term can be used as a synonym for fixing the problems. In Software Development, testing means evaluating the product and verifying that the product does what it's supposed to do. 

But now a million dollar question arises!

At  what time do we have to stop testing or what’s enough testing?

The answer lies in 2 factors: the QA functionality and how scrupulously that Functionality is applied in testing as “Quality matters more than quantity!”

“Enough Testing” refers to the amount of testing which ensures that a software product meets the requirements as defined. 

Testing is all about finding the errors and using the right test cases enables this. This increases the effectiveness of the work and also increases the accuracy rate but couldn’t guarantee the 100% error free result.

In my opinion, there are two main reasons why any software/application can not be tested completely.

1. For a particular input, there may be “n” number of tests and,
2. There are too many ways through which the product can be tested.

Testing Myths:

1. Know that, the things which are covered in the client meeting are all to develop the error free software/application: Often we think that the points which are discussed in the meetings are enough to get the 100% accuracy but this is wrong. There might be few edge cases that were missed while testing.
2. If a tester feels that his role is to verify that the product should work perfectly, then he is doing injustice to himself because in the real world any program is not hundred percent bug free.

Most of the developers catch and fix maximum of their mistakes in a product before deploying it for testing, so it is the role of the tester to find the remaining. This suppose, 1% may be neglected by the developers because they assume the end-user wouldn’t take that route while using the product but as a tester you can’t. The main goal is to have a top-level stable product.

Approaches to define Enough Testing:

1. Time for the testing ran out: While providing the time estimation of the work to the client, it has been clearly mentioned that this is the time allotted for the testing of the particular program and work should be completed within it.
2. Ideas that were discussed in the meetings: There are chances that the new ideas which were given or discussed by the manager, TLs, or other members of the team were enough for testing.
3. Getting the fall off results: Suppose a long time ago you stopped finding bugs, and the test ideas which you have are sufficiently hard to run and you think they will yield lower results. Better yet, even if they did have errors, the errors would be small, or only happen with a complex and rare setup. It’s not worth continuing testing.
4. Ideas put on the table were out of scope or unrelated to the given work.

Conclusion:  Although the testing is one of the never ending process but keeping in mind the project situations, the amount of work that is ready for delivery and project deadlines can be considered to decide when to stop the testing.                          

Hi @Kat - I am unable to post my blog article as it is showing 30000 Character limits. Tried with both Reply and Quote & Reply.  MS Word says including spaces it is having 5K Characters. Attached the screenshot for your reference. Please do the needful. Thank you so much for your support.

 

Hi, the problem may be in images if they are too large. Please just publish the article with less (or better without) pictures. 

Hi @Kat - I am unable to post my blog article as it is showing 30000 Character limits. Tried with both Reply and Quote & Reply.  MS Word says including spaces it is having 5K Characters. Attached the screenshot for your reference. Please do the needful. Thank you so much for your support.

 

Hi, the problem may be in images if they are too large. Please just publish the article with less (or better without) pictures. 

Hi @Kat - Thank you for your response. I tried to publish the document without image as per your suggestion. It got submitted, however unable to view the submitted document in this page. Please let me know what needs to be done. Thanks!

Hi @Kat - I am unable to post my blog article as it is showing 30000 Character limits. Tried with both Reply and Quote & Reply.  MS Word says including spaces it is having 5K Characters. Attached the screenshot for your reference. Please do the needful. Thank you so much for your support.

 

Hi, the problem may be in images if they are too large. Please just publish the article with less (or better without) pictures. 

Hi @Kat - Thank you for your response. I tried to publish the document without image as per your suggestion. It got submitted, however unable to view the submitted document in this page. Please let me know what needs to be done. Thanks!

I messaged you 😊

The article is written by @arunkumardutta 

Introduction:

After working in the IT industry over 17.5 years as a tester across many parts of the globe and publishing over 64 blog articles, 2 community books, 20+ international conferences and 11 global webinars, I still feel that “what’s enough testing” is a very thought-provoking topic.

Back in Aug 2020, I published an article in EuroSTAR Huddle – “Testing everything is impossible” (https://huddle.eurostarsoftwaretesting.com/testing-everything-is-impossible/). In that article, I talked about “In today’s fast paced industry, where accelerating delivery is the only way to survive, testing everything or every combination is simply impossible. Even in most favourable cases, where we assume that we tested everything, there will always be a chance of missing something. In both cases, continuous monitoring will assist to uncover missing cases well in advance and proactively before the actual end-users are affected.”

I am glad that Tricentis ShiftSync is conducting its first Blogathon contest and asked the testing community to pen down their thought about this interesting topic- “what’s enough testing”.

Enough testing- is simply Impossible:

While there is no doubt that even at the last stages of SDLC- Software Development Life Cycle, we can confirm that we conducted enough testing and there is no more testing required. We even can’t say that there will not be any more bugs or issues as we already conducted enough testing. Even in a software tester’s dream this is impossible.

If we look closely, there are so many possible combinations of testing (many technology platforms, many operating systems, many browsers, many devices, many versions, many networks) and testing all these is simply unattainable. Yes, leveraging automation adds value and can save an incredible amount of time but even still, testing all of these is not possible to conduct in a shorter sprint duration in accelerated delivery. Adding additional resources is also not possible due to the project budget. Test early, test often, test fast, test as much as possible, test even in production and test the right things will be the overall objective.

I personally think quality always matters more than quantity. So, we should not think about what enough testing is and when we can say enough testing done rather, we should think about the product end-users. As a tester, our objective is to ensure that end-users are happy in terms of different quality attributes like functionality, performance, security, usability, availability, reliability, accessibility for example.

Enough testing- No Data points rather Consent as a Team:

Simply, neither there is (or there will be) any data points for what enough testing is. Enough testing is just a consent by a group of different types of testers after conducting different types of testing in a specified time frame to go live. Ultimately, what I am trying to say is that there is no such formula or data points for enough testing.

I think testers should be in a destructive attitude towards the software product to make it stronger and better for its end-user.

At the end, software products become live only when product risks are verified by conducting different types of comprehensive testing and as a group all agree to move to production.

Enough testing- Comprehensive Test Strategy Document can assist:

While we, the testers, can’t say ever that enough testing is done. However, testers can confirm whether applications will be live or not after all their test ideas are thoroughly tested, followed by discussion as a team.

Please note that testing should be everyone’s job- when I say everyone, I mean starting from architect to developers, database folks to different types of testers to product owners to operation team to businesspeople - literally everyone.

Test ideas include all positive, negative, and applicable out of box thinking scenarios and all these must be conducted in a specified time as mentioned on the test strategy document. So, a comprehensive test strategy document is helpful to go live in a production environment but not unfolding what’s enough testing is (as no matter how comprehensive test ideas are, there will always be a chance to miss some test ideas).

Enough testing- Continuous Monitoring can assist:

While enough testing is impossible to confirm, continuous monitoring in all aspects-

a. continuous project monitoring of the test process (progress, compare, re-prioritize, confirm).

b. continuous application performance monitoring- dashboards, trending charts, automated alerts for proactively resolving the issues across all application components, APIs, Interfaces even before impacting end-users.

c. continuous security monitoring – again to identify security vulnerabilities, source code, attacks in advance before it affects end-users.

d. full stack observability- to get full visibility across the whole live system.

For me, what’s enough testing in today’s world is seen as full stack observability to proactively support end-users.

Conclusion:

While I don’t know what’s enough testing is even after working as a passionate tester for so long but for me what’s enough testing is a combination of comprehensive test strategy, collaborative as a whole team and moving towards full stack observability to ensure 3E-enhanced end-user experience, endure in the market for long to establish as a brand.

About Arun Kumar Dutta

Arun has over 17 and half years of managing end to end performance testing delivery experiences. He has been selected in multiple international testing conferences and global webinars. His multiple blogs have been published in different global testing forums.  Additionally, he contributed to 2 community books with other amazing testers across the globe.  He also won various internal and external awards.

Currently working as an Associate Director at Enterprise Performance & Resiliency Testing Practice in LTIMindtree.

IMPORTANT NOTICE: We want to ensure fairness for all participants in this blog contest. After careful consideration, we have decided to make some adjustments to the rules to enhance fairness and provide a better chance for every participant. Going forward, community votes will merge with evaluations from our Tricentis expert board. This ensures a level field and a more impartial selection of the winning article. Good luck!