Week 3 Exercise - Refining Prompt for Testing Task

Refined Prompt: "Provide a set of test data for functional testing of a mobile banking app. The data should include valid and invalid user credentials, transaction details, and account information. Please provide the data in a CSV format suitable for automated testing scripts."

Hi ​@parwalrahul ,
regarding the given prompt - "Give me some test data for this app." 
I have listed a few issues with the given prompt - 

• prompt does not specify what kind of app is being tested.
• It does not mention the type of testing
• It does not specify if the test data should be in which format to be.
• It does not indicate what kind of test cases the data should support.
• It does not define normal, boundary, or invalid test data requirements.

The prompt that I came up with is - 
"Generate test data for a login feature in an web application. The data should cover valid login credentials, invalid passwords along with negative scenarios.
Provide the test data  with columns for 'Username', 'Password', and 'Expected Outcome'."

And I have given both prompts in gpt and tested which is more specific.
1. "Give me some test data for this app."
output -

Sure! Could you specify what kind of app you're testing? Is it a web app, mobile app, or desktop app? Also, what type of data do you need—user credentials, form inputs, transaction records, etc.?

2. Generate test data for a login feature in an web application. The data should cover valid login credentials, invalid passwords along with negative scenarios.
Provide the test data  with columns for 'Username', 'Password', and 'Expected Outcome'.

output - 

Hello

From my perspective, I would say the following

Identified Issues with the Prompt:
No App Context
No Testing Techniques
No Expected Tests
No Usage Context
No Output Format
No Testing Scope

Improved Prompt:
Develop test data for a web-based insurance calculator application that includes user registration, product search, and checkout features. The test cases should consider all relevant scenarios, including typical user inputs, erroneous inputs, and edge cases. The data should be delivered in a CSV file with clearly labeled columns for seamless integration into manual and automated testing workflows

Regards 
Frank

When I used a vague prompt like "give me test data for app," the response included data for different types of apps, along with valid and invalid scenarios. However, the depth of these scenarios was quite limited, and there was a lot of unnecessary detail that wasn’t relevant.

On the other hand, when I provided a detailed prompt specifying the app and its specific feature, the AI generated highly relevant test cases. The response was well-structured in an Excel format with expected outcomes, which was quite impressive. The test cases covered all critical aspects, including in-depth scenarios and edge cases.

This experience showed me that giving clear and detailed prompts makes AI much more useful. A well-written prompt helps get accurate, well-structured, and complete answers, making AI a more powerful and efficient tool.

​@ghanesh : nice, quick and really fast submission !

My only suggestion to you would be that try adding a persona e.g. ask it to “Act as a tester specialised in test data generation” and check if it makes a difference.

​@Harsha23 : Nice one. what I like about your response is that you doubled down on “being specific”.

Your prompt shows that you know what you wanted.

Good one.

One great place to use as a reference for amazing test data is Test Heuristics Cheat Sheet | Ministry of Testing

You can also try giving some stuff from here for a good in-context learning and contextualized prompt.

​@Frank Kokoska nice one, mate. this is by far one of the best prompts added by someone on this challenge.

the bar is high now.

looking forward to more submissions now!

Issues:

• Lack of Context – No details about the app (e.g., type, purpose, functionality).
• Unclear Testing Scope – Doesn't specify which feature or aspect needs test data.
• No Test Coverage Details – Fails to mention types of test cases (valid, invalid, boundary, etc.).
• Missing Output Format – Doesn't specify how the data should be structured (e.g., JSON, CSV).
• No Expected Outcome – Lacks information on how the test data will be used or validated.

Improved Prompt:
Provide a set of test data for a [type of app, e.g., e-commerce checkout system], covering valid, invalid, boundary, and edge cases. Ensure the data aligns with [specific feature, e.g., payment processing] and includes necessary formats (e.g., JSON, CSV). Highlight expected outcomes for each test scenario.

​@shashwata you have identified the issues very well.

Also, like your template for creating a prompt for test data needs as per the [context].

I also see that you have asked it to “highlight expected outcomes for each test scenario”, which is a nice oracle to ask for.

Have you tried running it for any specific use case and evaluating if the outcome matches your expectations?

Generate test data for the below fields :

STest1 ID - Text(13) (External ID) (Unique Case Sensitive)
S4Test Id - Number(10) (External ID) (Case Sensitive)

Example:   2000009626

Create 20  records

The above fields are updated in the Salesforce System through Integration.