Solved

jenkins tosca testcase execution fails, but passes using tosca execution list...

  • 27 February 2024
  • 7 replies
  • 186 views

Badge

I am integrating Tosca with jenkins creating a windows batch command, when I execute the build jenkins will show build success but the results.xml file show that the testcase has failed. 

I can run the testcase through tosca exectionlist and the test case will pass, but fails when using jenkins.

why does it fail using jenkins but passes using tosca?

note, that no browser opens when I execute using jenkins when running via jenkins but open when I run using tosca executionlist ...is the browser supposed to execute in headless mode? if so is there a ‘switch’ to do so in the batch commands?

currently I am using a local jenkins install, jenkins is not installed on separate server (not ready).

 

thanks for your help in advance!

James

 

icon

Best answer by brendon.russell 4 March 2024, 21:08

View original

7 replies

Userlevel 3
Badge +1

It’s hard to diagnose just yet, there are a lot of details missing here.

What batch command are you using in Jenkins?

Are you trying to have Jenkins execute the tests locally? Normally these days you would have Jenkins run the Tosca Execution Client (or CI Client on older versions of Tosca) to run the tests remotely via DEx.
 

What errors do you see in the results.xml and/or Tosca execution log when the test fails?

Badge

Thanks Brendon for the reply.

 

I am using the ‘Execute Windows batch command’ in jenkins...here is the commands I am using:

"C:\Program Files (x86)\TRICENTIS\Tosca Testsuite\ToscaCommander\ToscaCI\Client\ToscaCIClient.exe" -m local -r "C:\Users\write\Desktop\ToscaBatchTesting\result.xml

I am executing the tests locally using jenkins as I cannot use a remote server yet. 

Jenkins runs ok...I get a build success message after jenkins finishes execution. it is in the results file where I see that it errors on finding the webpage login page which I am assuming is because the webpage is not being loaded either headless or as a physical browser. note, the testcase runs ok using tosca. 

 

error in tosca execution:

 

Here is the results xml file error:

 

error bolded in red

<testsuites>

<testsuite name="Build 240227 1637" tests="1" failures="1" skipped="0" time="22.536" timestamp="2024-02-27T16:37:26.4519849-07:00" id="3a10ff1a-fcb0-c5c4-d6c8-26f6d6253bd5" log="created: 2024-02-27 4:37:26 PM">

<testcase name="TableTest" time="22.536" timestamp="2024-02-27T16:37:28.8140706-07:00" log="- Failed TableTest {LogInfo='Execution will be continued with the next TestCase, because there was an error in the current TestCase.'} + Passed Kill browser {LogInfo='Started 'taskkill' with arguments '/im chrome.exe''} + Passed TBox Wait {LogInfo='Waited for 2000ms.'} + Passed TBox Set Buffer + Passed Username {LogInfo='Buffer with name: "Username" has been set to value: "Admin"',UsedValue='Admin'} + Passed Open OrangeHRM - Failed login {LogInfo='Could not find HtmlDocument 'login' with the following properties: - Title: OrangeHRM The Browser could not be found. Please check if you have opened the browser and prepared it according to the documentation.'} ">

<failure message="Test failure">- Failed TableTest {LogInfo='Execution will be continued with the next TestCase, because there was an error in the current TestCase.'} + Passed Kill browser {LogInfo='Started 'taskkill' with arguments '/im chrome.exe''} + Passed TBox Wait {LogInfo='Waited for 2000ms.'} + Passed TBox Set Buffer + Passed Username {LogInfo='Buffer with name: "Username" has been set to value: "Admin"',UsedValue='Admin'} + Passed Open OrangeHRM - Failed login {LogInfo='Could not find HtmlDocument 'login' with the following properties: - Title: OrangeHRM The Browser could not be found. Please check if you have opened the browser and prepared it according to the documentation.'} </failure>

</testcase>

</testsuite>

</testsuites>

 

 

Userlevel 3
Badge +1

Hmmm, it looks like the ‘Open OrangeHRM’ step isn’t opening the browser - I presume this step is supposed to do that, using the OpenUrl module?

I’ve never used the ToscaCIClient in local execution mode so I’m not sure if there are any limitations or extra setup to get it working. I can only suggest re-checking the manual and searching the Tricentis support site for any relevant knowledge base articles.

In any case, it doesn’t look like a Jenkins issue per se. You should be able to run the same command via command prompt, and I’d expect that to have the same behaviour.

Which Version of Tosca are you using?
Which Type of Remote Execution are you using?
    execution via Remote Service?
    execution with Tosca Distributed Execution?
    execution on the build server?
The issue you are describing I have only encountered if either RDP was not configured/not working or the executing user was not logged in.
So, first make sure you are logged in as the user which is running either DEX-Client or ToscaCIRemoteExecutionService and start from this user the Jenkins queue. The user you are using to log into Jenkins does not matter.
Also, be aware that logging in via Remote Desktop Login and then disconnecting will also show the behaviour you described, if not having configured RDP. And RDP can only used with DEX, not with ToscaCIRemoteExecutionService - at least I wouldn’t know how.
Regards,

Badge

Thanks tstets...

Here some answers:

Which Version of Tosca are you using?

2023.1.3


Which Type of Remote Execution are you using?
    execution via Remote Service?
    execution with Tosca Distributed Execution?
    execution on the build server?

 

I am doing this all on my local machine.,,no dex and no remote service. I don’t have a test server setup yet but wanted to do some jenkins \tosca executions on my local machine if that was possible,  but it appears it is not possible to execute locally. I am not sure if their is a switch parameter to ensure the browser is headless or if the browser can be executed visibly.

note, the testcases pass when running through tosca execution module.

Thanks for your help,

James

 

 

Userlevel 3
Badge +1

Thanks tstets...

Here some answers:

Which Version of Tosca are you using?

2023.1.3


Which Type of Remote Execution are you using?
    execution via Remote Service?
    execution with Tosca Distributed Execution?
    execution on the build server?

 

I am doing this all on my local machine.,,no dex and no remote service. I don’t have a test server setup yet but wanted to do some jenkins \tosca executions on my local machine if that was possible,  but it appears it is not possible to execute locally. I am not sure if their is a switch parameter to ensure the browser is headless or if the browser can be executed visibly.

note, the testcases pass when running through tosca execution module.

Thanks for your help,

James

 

 

Hi James,

Tosca will always execute in browser mode unless you specifically set it up to run in headless mode via WebDriver. I don’t know of any reason why you should need to do this for local execution via CI Client though.

If local execution is just to test out Jenkins and you will eventually be using distributed execution (DEx), then it’s probably not worth the bother of troubleshooting that aspect much further. If you can get Jenkins to successfully execute the CI Client and read in the result XML file at the end, then it is doing what it’s supposed to in terms of Jenkins-Tosca integration.

Badge

Thanks Brendon! Yes, All I was doing was testing it out. If there was a setting config to run in headless mode all the better but thanks for the info you gave me all I needed to know. I will be setting jenkins up on a server for execution with DEX soon. Thanks for the help!

 

James

Reply