Question

Assertions


Userlevel 6
Badge +4

 

 

Where do you put your assertions in your automation code?


3 replies

Userlevel 3
Badge +1

When I was working on Selenium, I have used below Assert categories. 

  • Assert Equals
  • Assert Boolean
Userlevel 4
Badge +4

I don’t use assertions, as such.

Instead, I usually write code that generates data for use in an automated assay of the product.

Then I write code that applies that data to the product. When I’m doing GUI-level work, I sometimes use a page-object design. I collect data about how the product behaves and save it. I don’t find it helpful to put a lot of asserts into that code because I’m usually working with a stable GUI for which I’m creating regression checks (thus low-level asserts would almost never fail, and all that same checking I can do on a higher level anyway). Also, asserts are testing logic, and I like to separate testing logic from logic that operates the product.

As a separate process, I write code to analyze that data and flag interesting things about it. When this takes the form of pass and fail, I call the whole process “checking.” When it takes the form of alerts and visualizations, I call it “flagging.” I like to focus on complex oracles rather than simple assertions.

One problem I have with the strategy of peppering code with asserts is that everyone who does that seems to lose track of what they are or are not checking. You have a zillion asserts… what does it amount to? By writing dedicated programs to serve as oracles, I find I can keep better track of that.

Another thing to note is that I write my own frameworks, instead of using PyTest or that sort of thing.

Userlevel 5
Badge +3

Always in the test cases or test steps. This places it squarely at the individual project’s requirement responsibility & enables maximum reuse, shared understanding + clarity (ways of working of where to expect the assertions) then ease of maintenance. 

 

  • POM level: 
    • projectA adds a method to Application123’s LoginPage POM, which asserts that submitted credentials result in a successful login and the dashboard page then showing with “welcome, UserName” shown.
    • projectA then needs to add negative test cases, either the POM method now needs a conditional branch or a duplicated method reversing the assertion
    • ProjectB with another team wants to interact with the same Application123 from their project. Their validation criteria is to check the look & feel of the page - the dashboard page displays the company logo in the right place, the font style & background colours are as expected (they don’t care about the welcome, UserName” text)
  • Test Automation Framework: can be the same
  • Helper methods: depend on where they’re located - are they test case, test data, POM, reporting or other? In my view test case & test step helper methods work to have assertions, other places not

 

In the above example, designing a LoginPage POM with methods such as isIncorrectLoginMessageShown(), getTextofIncorrectLoginMessage(), isUserNameEmpty(), isSubmitDisplayed(), then a DashboardPage POM with methods such as isPageDisplayedAndLoaded(), getWelcomeText() etc. will support more projects & give them each the responsibility of deciding what is a pass or failure criteria depending upon their own requirements.

Reply