Attention ShiftSync community! We are excited to announce a brand new testing challenge prepared exclusively for you by the renowned expert, James Bach. The purpose of this challenge is to provide you with a way to showcase your analytical testing skills for other people to review and evaluate. You will get a winner badge and a certificate of achievement.Â
Voting goes by likes. Like answers, which in your opinion provide the best solution to the challenge. The replies with the greatest number of likes will get winner badges.
Voting will be closed on 24.07.2023. However, even after the deadline, you may submit your solution in the comments and receive points.
Â
Chapter
Here is a parking cost calculator for an airport: https://developsense.com/parkcalc/index.php (This was copied from a real airport website, some years ago.)
There is a rumor that, because of poor input validation, you can make this calculator show huge parking costs (into the billions of dollars). Try to reproduce that. What is the highest amount you can make it quote you for parking?
Protocol
This is a hard problem meant to challenge you. There is no fixed time limit for solving it.
Use any resource you like to solve this problem. Tackling this with a team of people tends to work well.
Deliverable
Please explain your process in the comment section.
Take screenshots of your most interesting and extreme results and attach them to your comment.
Â
About James Bach: Â
James Bach@satisfice is a highly influential figure in software testing, renowned for his pioneering work on exploratory testing. His book, "Lessons Learned in Software Testing," co-authored with Cem Kaner and Bret Pettichord, is a go-to resource for testers worldwide. Bach emphasizes the value of skilled testing and encourages critical thinking, challenging traditional approaches. With his unwavering dedication, Bach has reshaped testing standards and inspired countless testers to think outside the box.
After reading the description of the assignment, I had doubts about the exact assignment. Is it about the highest possible parking costs. Or the most interesting and extreme results? Both are mentioned.Â
Â
I am always looking for extreme results so I used the calculator and entered various inputs per field and left the other fields blank.
I soon found out that with >=11 spaces or >=11 or more special characters, the calculater app was blocked and an Access Denied message occurred. With >=11 digits or >=11 letters, the app did not blocked. This applies to all fields separately.
Application Under Test:  PARKING CALCULATOR URL:  https://developsense.com/parkcalc/index.php Environment:  Windows 11 | Google Chrome Version 114.0.5735.134 (Official Build) (64-bit) standard & incognito separately | Using a VPN with a locale of the Netherlands Test Session: ~ 1 hour
Recommendation: Based on issues outlined below & questions regarding pricing, this feature should not be pushed to production as is.  Â
Critical Multiple issues due to lack of field validation. Â Recreated some of the issues outlined by other posts including ways to have fees in the trillions to get a feel for the application. Instead of attempting further tests that would be corrected by implementing field validation, checked the js for the page. Â Page uses a pre-release version of the calc [.8] https://www.rainforestnet.com/ The release history outlines bugs fixed in more current versions. Â Recommend updating to the newest stable release and retest. https://www.rainforestnet.com/datetimepicker/datetimepicker-release.htm
Â
Critical Due to the lack of field validation it is possible for the call to timeout. Â Where the timeout would be less likely by including field validation it is important to call this out separately due to the possibility that automated checks would prove invalid. When the timeout occurs, the page fully crashes, but the call returns 200 OK. Â If automated tests are only checking for a 200 OK response, they would falsely report as passing.
1. Do not update any fields except enter "9999999999" (without quotes) in leaving date field 2. Click Calculate Additional debug would be needed to determine root cause, but able to repro with 11 1s or 10 9s as mentioned in steps above.
Â
Critical [UX] Upon clicking calculate, the UI reverts to Short Term Lot which will be confusing to a user if they had selected a different lot. 1. Lot > select Valet 2. Date Entry > Tomorrow @ 8:00am (using the date picker popup) 3. Date Leave > Day after tomorrow @ 5:00pm (using the date pikcer popup) 4. Click Calculate Total is calculated & displayed, Â but the UI reverts the Lot to Short Term (the first in the list) which is confusing for the user, plus the network traffic shows the correct lot in the URL, but displays mismatched lots.
Â
Critical Valid UI inputs revert & update which can compound to inaccurate pricing upon clicking Calculate, the fields update in an odd pattern. Â Further debug required. 1. Leave Choose a Lot as the default (Short-Term Parking) 2. Date Entry > Today @ 12:01pm (using the date picker popup) 3. Date Leave > Day after tomorrow @ 1:01pm (using the date picker popup) 4. Click Calculate > Leave updates to 25:01 (24H 5pm), radio button reverts to AM 5. Change the Leave radio button to PM (leave everything else as is) > Click Calculate again > Leave time updates to 37:01 This will continue when changing the radio button to PMÂ
Â
Critical [UX] Able to force the date picker to navigate & select negative years / previous years / invalid date options 1. In Entry Date field replace YYYY with 0001 2. Without pressing the <enter key> > Click the calendar button 3. When the popup calendar opens > click the < button next to the yearÂ
This would need to be considered as part of field validation - we would also want to block the user from the ability to choose a date that would fail validation. Â
Â
Questions: Â What is the expecation for pricing? Â Outlined below is 1 hour, 2 hours & 24 hours. As a user I would expect the pricing for Economy to be less than Long Term Surface & Garage, but this does not appear to be the case. Â Is there a special formula for Valet parking since the cost of 1 & 2 hours is the same? Â Additional testing & investigation is needed around pricing (outside of validation / threshhold testing)
1 Hour Short-Term Parking = $2 Economy Parking = $4 Long Term Surface = $2 Long Term Garage = $2 Valet = $12
2 Hours Short-Term Parking = $4 Economy Parking = $8 Long Term Surface = $4 Long Term Garage = $4 Valet = $12
24 Hours Short-Term Parking = $28 Economy Parking = $9 Long Term Surface = $10 Long Term Garage = $12 Valet = $42
Additional testing notes:
Spot checked opening the site and checking the calendar in iOS mobile Chrome and found issues with navigation & visual placement. Â Would need to consider if mobile browser usage is important to stakeholders and if further testing should be pursued.
Other stakeholder expectations should be considered and evaluated against actual behavior.
Play with the Parking Calculator application and report the method for returning the highest quoted amount for parking.
Start
6/26/23 9:30 PM MST
Tester
Andrew Clark
Task Breakdown
Duration: 2 hours
Test Design and Execution: 70
Bug Investigation and Reporting: 45
Session Setup: 5
Test Notes
Immediately found the input validation to be weak. You can use letters, improperly formatted dates and times, interesting values in exponential format.
Continued trying different combinations of input. You can also modify the URL’s query parameters.
After calculating, the app set my large leave time number to be 1.0E+304. Going to follow this and see where it goes.
I’m continuing to increase the given exponential value. Sometimes I see a response of NaN or INF. If this was observed I would attempt to increase the value by a smaller amount until I hit the NaN response (ex. 1.051E+304).
This reminds me of an old game designer principle, “The Rule of Doubles”. Going to call this a heuristic and use it to keep making bigger numbers.
Up to (2.0806631388889E+303 Days, 0 Hours, 0 Minutes)
Took a break. This is tedious manipulating the numbers by hand. Played with the date field which has little to no impact on the quoted cost as the the leave time increased.
Back to increasing the number by hand. Spent some time trying to manipulate hours and minutes. I believe I’ve maxed out the most possible hours after plenty of trial and error.
Up to (2.0806633505351E+303 Days, 5.5440008598187E+288 Hours, 0 Minutes)
This gives me an idea to build a tool to automatically calculate the largest possible value here. I could build a script to make the query, examine if the response is NaN, and adjust the next precision point to be higher, lower, or move on to the next precision point. Repeat until we’ve exhausted the possibilities.
I’ve hit a point where increasing the leave time nor setting an earlier entry time affects the quoted cost. I believe I’ve hit the max or are very close to it.
Entry Date & Time: 0:00 6/27/23 (the date doesn’t matter) Leaving Date & Time: 4.0102587079508772682714375E+304:5.9E+305 6/27/23 (the date doesn’t matter)
Choose Valet Parking. This is important. Use it! Its an easy quote multiplier. Remember this gets reset after each calculation.
Â
Bugs
The PM button is not preserved after calculating the cost. This also increases the leaving time by 12 minutes after selecting PM for either entry or leaving fields.
The selection in the Choose a Lot dropdown is not preserved after running a calculation.
You can modify the URL’s query parameters and set invalid dates and times (ex. &EntryDate=20%2F17%2F2006 - the 17th day in the 20th month of the year 2006)
Thought Experiment
I know the ask here was to find the highest quoted cost, but I just finished James RST Risk class and this got me thinking about victims (btw, you should take the RST classes if you haven’t already - it changed my life). From the airport’s perspective, they could also be someone who is harmed by an app like this. In this thought experiment, I assumed the airport must honor what they quote (obviously they wouldn’t, but this was for fun).
Doing some research on the Grand Rapids Airport (from James earlier comment), and poking around their parking web pages, you will find this exact application was in service between June of 2006 and July 2015. https://web.archive.org/web/20060617125320/http://www.grr.org/ParkCalc.php https://web.archive.org/web/20150706144049/http://www.grr.org/ParkCalc.php
Although the numbers are not profound, you can manipulate the calculator to ensure you pay zero dollars for a 9 year period (the estimated time this app was available).
Now, if we look at the highest cost from the airport’s perspective, they lost out on $25k. Oof!
