Attention ShiftSync community! We are excited to announce a brand new testing challenge prepared exclusively for you by the renowned expert, James Bach. The purpose of this challenge is to provide you with a way to showcase your analytical testing skills for other people to review and evaluate. You will get a winner badge and a certificate of achievement.
Voting goes by likes. Like answers, which in your opinion provide the best solution to the challenge. The replies with the greatest number of likes will get winner badges.
Voting will be closed on 24.07.2023. However, even after the deadline, you may submit your solution in the comments and receive points.
Chapter
Here is a parking cost calculator for an airport: https://developsense.com/parkcalc/index.php (This was copied from a real airport website, some years ago.)
There is a rumor that, because of poor input validation, you can make this calculator show huge parking costs (into the billions of dollars). Try to reproduce that. What is the highest amount you can make it quote you for parking?
Protocol
This is a hard problem meant to challenge you. There is no fixed time limit for solving it.
Use any resource you like to solve this problem. Tackling this with a team of people tends to work well.
Deliverable
Please explain your process in the comment section.
Take screenshots of your most interesting and extreme results and attach them to your comment.
About James Bach:
James Bach@satisfice is a highly influential figure in software testing, renowned for his pioneering work on exploratory testing. His book, "Lessons Learned in Software Testing," co-authored with Cem Kaner and Bret Pettichord, is a go-to resource for testers worldwide. Bach emphasizes the value of skilled testing and encourages critical thinking, challenging traditional approaches. With his unwavering dedication, Bach has reshaped testing standards and inspired countless testers to think outside the box.
The sample parking calculator seems to be of an old application/website.
At first sight, it’s UI provides already some tips:
the OLD UI is a hot tip telling that it may not be responsive; which in fact, it’s not so it’s not going to work well with mobile devices and/or devices with different screen sizes
there seems to be several input that use the keyboard and also some shortcuts (e.g., for opening a calendar)
Focusing on the input validation, there are at least 2 ways of inputing data: keyboard and mouse. The fact that we can use both may present a challenge in terms of consistence between values edited using either of them.
Besides it, and probably more critical, is the fact that there seems to be no input validation whenever editing the fields using the keyboard; they are just text fields (i.e., “input” elements of “text” type), instead of proper HTML elements that could restrict it to just numbers or even a regular expression.
Knowing this, we can enter some invalid times like:
15:00 AM (which is a value out of bounds)
15:00 PM (which becomes 27:00 AM after submitting)
25:00 AM (which is a value out of bounds)
-25:00 AM (which is a value out of bounds)
More,
the date field doesn’t have a validation, therefore we can enter a date in a whatever format. For example, in some countries it’s common to use YYYY-MM-DD, such as 2023/06/27. If we enter that for the end date, for example, it will accept it, and returns a cost of more than 1,6 Million USD for a few days.
Additionally, we can see in the browser that submitting the values for calculation performs an HTTP GET, and the parameters can easily be modified on the browser address bar.
We can modify the type of Lot to some unexpected value like STPPP and use that to calculate the parking cost… which won’t give an error and return $0, such in the following example.
More deep testing would be needed but the fact that there are some critical validations that aren’t being done, this parking calculator should not be released and we can even say that probably this needs to be fixed ASAP before more exhaustive testing is performed.
Having an adequate “Leaving Date and Time“ threshold would really help, e.g. 01/01/2100. Also, please embed the link to the Parking Calc in the “Chapter” section, so that one doesn’t need to copy->paste the link or use “Open In New Tab OR Window”
Few descriptions are already provided on top.
The “Choose Leaving Date and Time” field accept real time data, with no restriction on decimal values, in fact it worked till 21 decimal values
result is here
I interacted around 45 minutes with the calculator straight and also during writing this report (around another 45 minutes). Warning: I’m not used to AM/PM times therefore I might confuse some things which are normal for this.
My currently highest amount (It might even go higher): 5,777,922,043,452.00
Observations:
Dates are not checked for any sanity
A crash when I set the years date to high. The pages goes white after some seconds, having no UI anymore.
setting one of the dates to (-)350,000,000 let the site go white => It would need further investigations to find the exact limit.
Making a successful calculation with Valet Parking resets the option to Short-Term Parking
After a calculation with PM the UI is set back to AM and sometimes is the time also changed:
Warning: I’m not used to AM/PM times
00:00 PM goes to 12:00 AM
25:99 PM goes to 37:99 AM
Requesting the URL with working values returns the HTML immediately but it does not contains a cost value. With high, working dates takes the response some seconds.
How do I came to this?
At first I played a bit around to learn how this calculator works. What are the UI elements.
My first idea was that day dates in general and specifically extrem years are a fast way to create high amount.
I tried out if dates before 1.1.1970 (Unix date start) where an issue but even for that I could generate costs. Having the Leaving date before the Entry date was gave an error message as often as I tried it (maybe in combination with the time can be found more issues).
Now I started playing around with the dates. At first going back in the Entry year like 999, -9. With the first -9 I just add 9 to reach the maximum which was -999,999,999 at that time, crashing the site. As I observed with every 9 added the response took longer.
Now I set the Entry date to around -999 and tried in batches to approach the limit.
Maybe I should have stopped pushing the boundaries way earlier. Having Entry dates quite back in the past when no cars makes no sense at all. Also a Leaving date more than 100 years in the future is not that different from 1,000 years.
My time is out now. I have to stop the report here. Maybe I can continue and improve this report tomorrow.
I think I’ll leave out some of the details about the general overview of this calculator (i.e. the UI/UX shortcomings etc.) that have already been addressed here and jump straight into my findings.
First of all, it needs to be noted that the only input validation I’ve found in this calculator is the general check that the entry date shouldn’t be later than the exit date (and vice versa). That’s it. The inputs are not validated for non-digit characters which leaves plenty of room for bugs to crawl in (and hopefully be discovered).
So I started playing with numbers, particularly dates. My initial thought was to set the entry date as early as possible and the exit date as late as possible, hoping that the date difference would end up in a huge cost value returned. That was partially true, and I was able to get to something around a trillion. Adding up nines to the year section led to slower response time, so I decided to switch to time inputs.
It quickly became obvious that the the server did not have any restrictions on seconds part, and the number grew way quicker.
That’s when I noticed that the number of days actually had scientific notation which was denoted by E+16 part. I wondered, well, if the server returns such a value to the client, then maybe it could process it if the client sends it to the server? So I immediately tested that out by adding E+199 to the exit time which resulted in something like this…
As we can see, the cost number returned was so huge it didn’t even fit the page and broke the layout. I went further and got curious what would happen if I entered an even bigger number in scientific notation, something around E+99999? Below is the result
The return value turned out so huge that the calculator returned NAN dollars for infinite number of days.
That’s how far I’ve gone so far. Still wondering if there’s a way to make that thing return a negative value.
the calculator is using simple formula to calculate number of hours of parking and then multiplies it by a unit amount. The unit amount part looked ok to me
Time parked = (End Date End time - Start Date Start Time)
There is no validation on End Date End time or Start Date Start Time.
It can be difficult to break out of our preconceived ideas of software limitations: for instance, the proper formatting of the date field.
This is what hackers do, though. This is why cybersecurity is so hard. Developers and product owners like to say “no user would do that.” But of course in cybersecurity terms, hackers are users, and they are motivated to find vulnerabilities in our products.
@Shrihari You are very close to the maximum I was able to get. My max is more than twice as large as yours, though.
@Sebastian Stautz For project purposes, I agree you don’t need to go any further. Still, it can be a good learning experience to try.
@darktelecom I like your report. It’s written clearly and seems well-reasoned. I agree that if we are simulating a realistic project situation, you don’t need to go any further. However, sometimes I like to push things beyond the practical on projects because I want to enhance my street cred as a bug investigator/hacker.
@detestable I don’t get a reply at all with that input.
@satisfice ohhh, maybe it was just me, the request hanged for like 30 seconds and got me a blank response
bummer
@satisfice my mistake when reporting (tried to be the cool kid), doesn't work directly by url, but if you use 1.0E+304:1.0E+306 (obviously copied from Shrihari) as exit date and today as entry date and submit I think you'll get it to hang
This was a fun little excercise. At first it was clear that some information is missing to fully test this. What are the supposed prices for every lot? Should I be able to stay for a Year on a Short Term Parking?
But then I though about the real issue I was supposed to investigate and decided to leave those questions for another time.
My first approach was to simply check how much could I adjust the dates. I tried to calculate a 30 year booking. Wich didn’t give me any unreasonable numbers. So I then decided to check if I could book a leaving date before the entry date. To my surprise this seemed to work quite nicely. So then I though about the time. (I also noticed that no error is displayed when the user invents dates like 29th of Febuary 2023)
Something that catched my eye is the AM/PM selector (I was thinking of inputing 24:00 AM). When I clicked the timers, I noticed I was able to write on them. So I started with the obvious 999999999999999999999 and the issue was found.
But I wasn’t done. I thought “Maybe if I am able to go that high I can go way down” so I decided to try with a big negative number “-99999999999999999” seemed reasonable.
Also, robert seems to be a valid date, and quite costly
No correct validation for displayed date format: possible days, months, years: min and max values.
It looks like 00-99 range - when typed 2 symbols (DD, MM) and 0000-9999 range - 4 digits for YYYY.
But as we know range for DD: 01-31; MM: 01-12; YYYY: most possible start day - today. It means at least 2023 and till real year time (not 9999). But now you can type 0000 and it will be applied.
The maximum possible sum of money case:
Start date: 00/00/0000 End date: 99/99/9999
Sum: 76,041,838.00
Here is my report on this nice challenge.
My first goal was to find out how the app behaves with different date format(s) so I’ve tried the following formats:
DD/MM/GGGG
GGGG/MM/DD
GGGG/DD/MM
YYYY-MM-DD
DD-MM-YYYY
MM-DD-YYYY
DD.MM.YYYY
MM.DD.YYYY
YYYY.MM.DD
YYYY.DD.MM
M.D.YYYY
D.M.YYYY
YYYY.M.D
YYYY.D.M
DD MM YYYY
MM DD YYYY
YYYY MM DD
YYYY DD MM
D M Y
D M YYYY
DD MM 'YY
DD-MM-YYYY HHss
DD/MM/YYYY HHss
DD.MM.YYYY HHss
Next step was to find out how big value can ‘Date’ section accept before breaking and after some experimentation I came to the conclusion that the max/min values are in range, inclusive, -999999999 <> -999999999.
The next step was to fiddle with the ‘Time’ inputs. There were some interesting finds here too. For example:
Start date can be empty and will be set to ‘:00’ after calculation is performed
If ‘:XX’ is entered in the Leaving time and the ‘PM’ is selected the value will default to ‘12:XX’ after calculation
Setting start date to 00:00 AM and end date to 99:99 PM. After calculation, the 'end date' is set to '111:99'
The value in the 'Leaving date' increases with each calculation – 'PM' needs to be selected to achieve this (the pictures below show some of the results have ‘AM’ selected, that is because the picture was taken after the calculation was performed and the form resets some of the values after the calculation)
Then, there were some general issues like:
The 'Choose a Lot' resets to 'Short-Term Parking' after each calculation
Entered values don’t reset after the page reload
Of course, the best one is saved for last. These are the two highest values I’ve managed to obtain (was inspired by results/inputs Shrihari obtained/used):
@Sebastian Stautz For project purposes, I agree you don’t need to go any further. Still, it can be a good learning experience to try.
Somehow I found it more interesting to find bugs by playing with the UI than to look for the maximum cost :-) Which might still be an interesting challenge on its own.
I did in the usual format I note down findings when testing (something I learned from RST). One can argue my “improvements” section doesn’t make sense but I seem them as something additional to the current scope. Everybody knows there’s a third party plugin out there somewhere that can do everything this calculator does.
Bugs:
The "choose a lot" selection always resets to "Short-term Parking"
It takes negative values as input and also 0 as input
It takes time values greater that 24 as input
There's no way to refresh the thing since every parameter gets embedded in the URL
AM/PM should be decided based on time value selected. 1300 cannot be an AM value
Time value should auto format from 1200 to 12:00. Regex to detect minutes is missing, it takes the value entered as hours
The only way to get a time value in PM is to enter the AM equivalent, select PM radio button which then adds 12 hours to the value
If you input a PM value for time and then select the "PM" radio button, it will go beyond 2400
AM/PM radio buttons reset to AM
It even takes text as input for time and date
Improvements:
The date picker should be placed alongside the date field
The date picker should also include a time picker
There should be a max time limit to park your car
I have chosen a simple combination of digits like 1 and 9 to check extremities randomly, manipulated the time data only and here is the result.
Why only a combination of 1s and 9s in the date fields?
Well, as a kid, we can think of only 1 and 9 to play with the smallest and largest values excluding 0.
And a wild attempt!
Tests:
Here are my few random tests and after seeing this kind of profit, I would love to start a business.
When all the inputs are in the wrong format.
One Invalid date like 2/38/2100 is no barriers, and we are in profit.
Even negative signs did not matter!
Following formats but with unexpected values and it works!
No proper entries but the value of Zeros. Wondering how the calculation logic is working!
Note: I am adding my observations later! Stay tuned.
Observations:
What happens with a click on “Calculate”
Choose any of the lots other than “Short-term Parking” from the dropdown, it will reset the value to the same.
Time Fields
By standard norms, time follows the format of hh:mm.
I tried entering hhss format and it got reset after clicking to calculate.
Enter any time for the entry or leaving time field. (Default selecting is AM).
Select the ‘PM’ button and calculate.
You will get 12 hours added to ‘hh’.
The only value that remains the same without change is 12:00.
You can try any number from single zero to unlimited ranges.
If you add only strings to the time field and calculate by selecting AM, it will show string:00
If you add only strings to the time field and calculate by selecting PM, the value will reset to 12:00.
If you add only time in hh:mm format followed by strings to the time field and calculate by selecting AM, it will remain the same.
If you add time in hh format followed by strings to the time field and calculate by selecting AM, it will show the value as hhstring:00.
If you add only time in hh:mm format followed by strings to the time field and calculate by selecting PM, it will show (hh+12):00string.
If you add time in hh format followed by strings to the time field and calculate by selecting PM, it will remove the string, add 12 to hh and, set the value to (hh+12):00.
How the date field works
No validation of date ranges, invalid date, invalid date format, date combined with strings, and special chars.
If you simply enter just numbers (for easy fundamentals- pick anything between 1 to 12), it considers the number as the month like 1 stand for Jan, 2 for Feb, and so on, and calculates the total days of the entry month.
Note: If you select the entry date field with the value 0 and leave date field as 1, it will give you a default value of 31.
If you enter the entry date in -/-/- format and 2nd field digit(s) is considered as day only.
The leave date if entered with a number between 1 and 12, is considered as month.
On calculation, the total days are counted as
{total number of the month (from leave date field)- days (from the entry date 2nd field)}
For the below example, total days = (31-12) = 19 days.
If you enter both the entry and leave date in -/-/- format.
On calculation, the total days are counted normally.
No matter whatever value you add after -/-/- this, the values are counted based on the values entered within the -/-/- format.
Note: My other few points are already iterated by many of the others.