Get Trained and start executing UFT tests from Jenkins. Click Here for more details.
Hello Friends,
In this post, we will discuss integrating Jenkins CI tool with HP UFT.
Introduction
Jenkins is a CI tool which basically monitors external job executions and it’s mainly used by developers for building and deployment of code. It provides several functions like scheduling of tasks called jobs in several remote machines called as slaves in terms of Jenkins vocabulary. In this post, we are going to learn how to setup Jenkins from scratch for executing UFT automated test cases.
First, we need to understand, what is the need for running the automated tests with CI tools? It’s not just applicable for Jenkins, its applicable for any other CI tools or automation tools available in the market. Most of the people are going to tell you about agile and Smoke testing, those are correct and we will go back to this topic in other posts. First, we will look into more common issues which teams will face.
I was very happy with learning new languages and implementing new things for my project, and then one day we did some analysis and it seemed that we were spending most of our time in executing older regression test suites and we have only 20-30 % of our time left for development of new test cases, and it’s the problem which was growing day by day more severe.
So, the only solution for this problem is to assign the execution of the scripts to the actual owners. It seemed ideal, but in the real world that’s not as easy as we think, everybody has their own tasks to be completed, plus they didn’t have the knowledge for using UFT and you will be pushed back to ground zero, from where you started.
On the other side, tests we had were automated test cases, and hence by definition, they are supposed to be run automatically. But due to the manual interventions required due to the nature of scripts or application, it was close to impossible to execute the most of the scripts on a headless remote machine.
To resolve these issues we tried HP ALM’s Lab management tool, but I realized very soon that it can only execute the test cases so if you have to execute a bunch of scripts before or after execution of the scripts, you cannot. Also, there is no way to approve the results and a lot of time issues with the remote machine cause the test to fail.
There is another way of doing it using Automation Object model, but I really don’t want to take the headache of maintaining another piece of code which might change between versions of UFT.
I tried Jenkins and I am still using it, basically, it does everything, let’s see how…
Installation of Jenkins Server
If you don’t have Jenkins server, then follow this guide for the installation of the server. If you have installed Jenkins server using windows setup, you can access Jenkins server using the following URL
Check your Jenkins workspace path and update if required.
Create slave machine for the execution of the UFT scripts(Machine where UFT is installed), and install the following plug-ins
List of required plugins for the setup
- Copy Artifact Plugin
- Dashboard View Plugin
- Discard Old Build Plugin
- Email Extension Plugin
- Extra Columns Plugin
- Flexible Publish Plugin
- HP Application Automation Tools Plugin
- Test Result Analyzer Plugin
- Time Stamper Plugin
- Workspace Cleanup Plugin
Most of them are self-explanatory, but if you need any other details then, Google with the exact name of the plugin.
Creating Jenkins job
Once, you are able to open the Jenkins server, Click on “New Item”
Provide any Item name, select “freestyle project” and click on “OK” button
Once, you click on OK button, New Job configuration page will open. This is the main configuration page, where we will provide all of the configuration details about how the Job should execute.
Fill out the following information within the job configuration page
1) Description: Description about the job and the UFT test which is tied to this job.
2) Check on “Restrict where this project can be run” and provide the slave machine name with UFT installed. This is a very important step, because otherwise Jenkins will choose a slave machine randomly, and might choose Jenkins master machine (where Jenkins is installed) for execution. This will cause the job to fail.
3) Select appropriate build triggers, if you are using any source code control for storing UFT tests (Except ALM) e.g. GIT, SVN, TFS.
4) Under Build environment select “delete workspace before build starts”. This will help us in cleaning up the workspace and have a fresh start for each execution.
5) Click on Add build step button under build and select “Execute HP Tests from file system”. This will allow you to execute the tests from file system e.g. “C:\” or network drives.
6) Provide UFT tests or Solution path in the “Tests” text box. If you provide UFT’s solution file path then all tests under solution will be executed one after another. Leave Time out seconds blank.
7) Click on “Add post build action” under post build actions, and select “Publish HP test results”. Select “Always Archive HP test Reports”. This step will generate and archive Junit results for the execution on the Jenkins server, which can be used for analysis. we will discuss this in detail later.
8) If you have configured the email server then Add another post build step and “Editable email notification”. This option will send automated emails after the execution is over. We can attach any files in the email or we can send any files as the part of the email body.
After filling out the above information, Click on Save button and start the job by clicking on “Build now” button within job details page.
Once the execution is started, click on Build number, under build history, to check the current execution details
While the test is executing, you can click on Console output, to check the current execution status.
Once the execution is completed, you can see the test results on the Build page, under test results
This completes the basic configuration of the Jenkins Job.
How it works?
When you execute a UFT test from Jenkins, HP automation tools plugin will copy the following files to the Jenkins Job workspace directory
1) HpToolsLauncher.exe – This is the file used for actual execution of the UFT tests .when you take a closer look at the console output of the Jenkins build, you will see a line
"C:\Jenkins\workspace\JENKINS UFT TEST\HpToolsLauncher.exe" -paramfile props23032015132505096.txt
If you execute this line on the slave machine UFT test will execute.
2) LRAnalysisLauncher.exe – For load runner.
3) propsXXXXX.txt – This is the properties file for the HP Automation tools plugin. This will store different parameters required for the execution of UFT/Load tests.
4) ResultsXXXXX.xml – This is the JUnit results generated by the HpToolsLauncher.exe after the execution is completed. When we choose to create a post build step and choose to “Publish HP run results”. This Junit file will be uploaded back to the Jenkins and produce the test results shown in step 12 above.
General Issues faced during test execution
There are several issues you may encounter during the execution of UFT tests from Jenkins
1) If scripts have any “MsgBox” statement or any “InputBox” statement, scripts will directly fail.
2) If the machine is unlocked, then there is a chance for complex applications to fail.
3) We have to make sure that machine is cleaned up before and after the execution of tests(Closing all browsers and any other types of files like excel sheets or word documents).
4) We have to make sure that UFT scripts have a proper error handling mechanism or UFT will throw an error message and Jenkins Job will fail as soon as Popup appears on the page.
5) If we have a proper framework for our scripts, then most likely we will be executing multiple tests within one UFT test. Because Jenkins HP Automation tool can only count UFT tests and there is no way to indicate no. of tests, if you execute 2 UFT tests having 20 tests each, Jenkins dashboard will only display 2 tests executed, not 40 tests executed.
6) If UFT test report’s a failure in any of the tests, you will see the failure of Jenkins complete Job instead of a particular test case.
let’s see how to resolve each of these issues to create a perfect Jenkins job for executing automated UFT tests.
1) Replace any statement which requires human intervention
We must scan our scripts and replace any statements which require human intervention like MsgBox statements must be commented and should only be used during debugging. For any steps which use Input boxes for accepting inputs, use test parameters and action parameters. For receiving information back, use reporter.reportevent instead of msgbox statements.
2) Use Physical desktops, instead of virtual machines
If we use virtual machines as Jenkins slave machines, then we will have to make sure that the machines are up and running all the time, because virtual machine can go blank without any prior warning, if the user disconnects them.
3) Unlocking remote slave machine before executions
To overcome this issues, if possible change the machine setting, so that machine never locks automatically. But this option is not available in most of the companies because the machines running UFT will most probably have admin rights and it’s not advisable to leave the machine with admin rights unlocked.
First test your scripts, if they are able to run in the locked machine. If not then, you can use the following commands to unlock the Physical desktop machine from another computer (In case you are at home or other location).
For Unlocking the Remote Desktop(Physical machine)
Run on the command prompt
Query User
This command will provide the Session ID for your Login Session
Once you have the session id of your session execute the following command
Tscon <Session id from previous commapn> /DEST:Console
This command will unlock the desktop machine and you can execute your test normally.
4) Include Machine Cleanup Framework in UFT
Machine cleanup framework is explained here. This framework will use the VBScript classes for executing predefined cleanup code. This is required because, if the previous execution of code leaves some things open then, your subsequent executions might fail.
5) Include proper error handling mechanism in your scripts
Recovery scenarios cannot handle errors properly, for running tests from Jenkins you will need a way to handle all kinds of error, including UFT’s general run error’s. Because, if UFT throws a general run error, then your Jenkins execution will fail, while UFT will be waiting for someone to click on the error and close the test manually and this will create issues with the subsequent test executions. A detailed description of how to implement this framework is explained here.
Executing tests in Parallel
For execution multiple tests in parallel, you can choose to do one of these two methods
1) Open the tests and update the Label expression under “Restrict where this project can be run”, with the slave machine name. Once you update the two different tests with different machine names, you can execute the tests in parallel on those two machines (provided you have the proper licenses for executions).
2) Now, the previous method works, but that’s not the best way to execute parallel tests within Jenkins. To implement the second method you will have to figure out the machines with similar properties like “UFT MACHINE WITH IE 8”, so this is a specific machine having UFT installed and can be used for the apps which are only compatible with IE 8. Once you have all the machine characteristics identified you can then open the slave configurations for the machine and update these names in the Slave labels. Once this Logical grouping of the Machines is done then you can provide this logical name with “Restrict where this project can be run”, instead of machine names. Using this method, you wont have to actually remember about the machine names and go through the whole hard process to identifying which machine you should execute the test.
One more advantage of this method is if you have two machine with Label “UFT MACHINE WITH IE 8”, and when you execute the test, if the one of the machine is busy, then Jenkins will automatically Pickup the second machine and use it for execution.
Building test queue(sequential test execution)
Jenkins will automatically build the execution queue, once you select more number of jobs then the machines you have. These tests will be executed automatically when the machine will become free. To check this, Update the “Restrict where this project can be run” of three Jobs with the same slave name. Now, click on home page and Click on “Schedule a build” button for all three jobs, one after another and you will see the build queue, on the left side of the home page under “Build queue”.
Executing tests from ALM
For running tests from ALM, you have to make three changes to the current configuration explained above.
1) Add ALM Server
open Jenkins -> Manage Jenkins -> Configure System and find the sub section “Application Lifecycle Management”.
Click on “Add ALM Server” button. Provide the server name as Test or Prod or Dev and enter the server URL till qcbin. Click on save and apply.
You can add multiple ALM Servers.
2) Use “Execute HP Functional test from HP ALM” instead of “Execute HP tests from file system”.
Add the Build Step “Execute HP Functional test from file system”. Select ALM Server name from the “ALM Server” combobox. Provide Username, password, Domain, project and Test Set Path for the execution of the tests. Test set path is the actual physical path which you see in the ALM test lab tree structure starting with Root\
3) Check the advanced setting,
if you want to execute the tests on specific host defined in ALM.
You will see the following changes in the execution process
Modified Console Output
When you execute your tests from ALM Server, you will see one change in the Console output, you will start seeing all of the Reporter.reportevent statements on the Jenkins console log.
This feature can be used to check the current progress of the execution.
Automatic Execution update in ALM
Test execution status will be uploaded automatically to ALM Server.
Additional helpful setting\upgrades
1) Instead of using “Publish HP Run results option”, Create and upload your our own Junit results
When, one UFT test executes 100 test cases, HP Automation tools plugin will only return either one pass or one fail. To over come this create your own Junit results xml file at runtime after the UFT execution is completed. This will also serve as a checkpoint, if the UFT execution is completed successfully then only we should have a JUnit file generated.
Once you have this functionality implemented, you will start seeing awsome results on Jenkins and you will unlock the real power of Jenkins. JUnit results XML will look like this
<testsuites> <testsuite Name=“FirstAutomation” test=“2” failures=“0” time=““> <testcase name=“test1” classname=“FirstAutomation.TestCases” time=“27.68359” status=“pass”> <testcase name=“test2” classname=“FirstAutomation.TestCases” time=“13.33203” status=“pass”> </testsuite> </testsuites>
2) Upload test Artifacts to Jenkins
Jenkins will allow to upload the artifacts back to the Jenkins server. We have to use the copy artifacts plugin to do this. Within UFT you can create a function and call it within the Machine cleanup Framework’s class initialization(to make sure it executes) to setup the export of UFT datatable and UFT run results to the Jenkins workspace like this
Public Function SetQTPParameters() Dim App 'As Application Set App = CreateObject(“QuickTest.Application”) App.Options.Run.AutoExportReportConfig.AutoExportResults = True App.Options.Run.AutoExportReportConfig.StepDetailsReport = True App.Options.Run.AutoExportReportConfig.DataTableReport = True App.Options.Run.AutoExportReportConfig.LogTrackingReport = False App.Options.Run.AutoExportReportConfig.ScreenRecorderReport = False App.Options.Run.AutoExportReportConfig.SystemMonitorReport = False App.Options.Run.AutoExportReportConfig.ExportLocation = JenkinsWorkSpacePath App.Options.Run.AutoExportReportConfig.UserDefinedXSL = ““ App.Options.Run.AutoExportReportConfig.StepDetailsReportType = “HTML” App.Options.Run.AutoExportReportConfig.StepDetailsReportFormat = “Detailed” App.Options.Run.AutoExportReportConfig.ExportForFailedRunsOnly = False End Function
After creating this function, on the Jenkins configuration, add a build step “Archive the artifacts”. This is Upload the results back to the Jenkins server and will be visible for each execution. The default Archival path is Jenkins workspace, but we can also point it to the child folder inside jenkins workspace.
e.g. If Jenkins workspace path is “C:\Jenkins”(Create this in Slave Config -> Remote Root Directory) then the Job execution path will be “C:\Jenkins\JENKINS UFT TEST\”
to archive the artifacts from the child folder within workspace e.g. “C:\Jenkins\JENKINS UFT TEST\ExecutionResults”
use “ExecutionResults/*” in the files to archive textbox
In addition Upload any artifacts related to the test execution to Jenkins, if required.
3) Use Dashboard Plugin to create execution dashboards
Once, you have dashboard plugin installed in the Jenkins, create dashboard views and add test execution statistics Grid to it. This will pull the data from the Jobs latest JUnit files and publish the data on the Dashboard page Providing summary on the Dashboard home page.
Above is the snippet of the Test execution statistics Grid.
4) Use Flexible publish to produce the results
Use the flexible publish plugin to figure out things like, if the files are generated before actually passing the Jenkins job or failing the Jenkins Job. One example is check if the Junit results are generate before archiving them to Jenkins server. If the file is absent then Jenkins will throws an error and fail the job.
5) Close and open HP Remote agent during each run session
I have included a separate batch file for running the remote agent launcher during each run session and close it after the execution is over. This will make sure that all of the UFT related processes and connection are re-established during each run session.
6) Use “test result analyzer” for result analysis
You can use the test result analyzer for analyzing the results of two separate executions. To Open, Click on “Test Result Analyzer” on the job description page.
If, we have 100’s of tests, it might not be very easy to analyze the results between two executions, but if you have Custom JUnit reports generated, then you can use this feature for analyzing the results between any two executions
7) Use Batch File to Kill UFT
Create a batch file to kill UFT and add this to build step after the step for the execution of the UFT Test. This will make sure that UFT is not hanged in between and also if the Jenkins Job is stopped due to some reason you will have the machine ready for the next one.
Let me know, if you have any questions and i will try to resolve them.
Thanks
Automation Insights 
Console Output
Started by user admin
Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
Failed login to ALM Server URL: http://sm09520.dom1.e-ssi.net:8080/qcbin. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
Empty Results
Result Status: UNSTABLE
Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
Report not found
Build step ‘Publish HP tests result’ changed build result to FAILURE
Finished: FAILURE
LikeLike
Hi Sumit,
I am not able to put the screen shot, could you please help me , how can i upload the screen shot here?
Thanks,
Chandra
LikeLike
where are you giving the URL for ALM, in job configuration or Jenkins configuration??? –,—In Jenkin Configuration
Sumit, As i was not able to upload the screen shots here, i have sent you eamil, Could you please check it and look in to in…
Thank you very much for your support till date …
LikeLike
Jenkin Configuration:
ALM server name : HP ALM 11
ALM server URL : http://Server:8080/qcbin
Job Configuration:
Execute HP Test using HP AL LAb Management
ALM server : HP ALM 11
User name : my KID
Password : Windows PWD
Domain: ALM Domain
Project: Project
Run Type: Test Set
Test Set/Build Verification Suite ID : Root\Automation_POC\Jenkin_POC
Jenkin_POC is Test set in Test LAB
Timeslot Duration (in minutes, minimum = 30) – 30
Console Output
Started by user admin
Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
Failed login to ALM Server URL: http://sm09520.dom1.e-ssi.net:8080/qcbin. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
Empty Results
Result Status: UNSTABLE
Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
Report not found
Build step ‘Publish HP tests result’ changed build result to FAILURE
Finished: FAILURE
LikeLike
Jenkin Configuration:
ALM server name : HP ALM 11
ALM server URL : http://Server:8080/qcbin
Job Configuration:
Execute HP Test using HP AL LAb Management
ALM server : HP ALM 11
User name : my KID
Password : Windows PWD
Domain: ALM Domain
Project: Project
Run Type: Test Set
Test Set/Build Verification Suite ID : Root\Automation_POC\Jenkin_POC
Jenkin_POC is Test set in Test LAB
Timeslot Duration (in minutes, minimum = 30) – 30
LikeLike
Hi,
Which version of HP Tools Plugin are you using, I don’t see the options you are selecting.
Timeout is an optional field, you can remove the values from the timeout field.
I have never entered a value for the Environment configuration ID, so try clearing that field also and see.
Jenkins_POC is a testset name ?
I am sending you the screenshots for my configurations.
Also Let me know, what options you have for Run Type drop down..
Thanks,
Sumeet Singh Kushwah
LikeLike
Latest version of the Plugin is 4.0.1
LikeLike
Output
Started by user admin
Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
Failed login to ALM Server URL: http://sm09520.dom1.e-ssi.net:8080/qcbin. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
Empty Results
Result Status: UNSTABLE
Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
Report not found
Build step ‘Publish HP tests result’ changed build result to FAILURE
Finished: FAILURE
LikeLike
Hi,
Please find my comments as below…
Which version of HP Tools Plugin are you using, I don’t see the options you are selecting. —- I am confused here, where can i select this option & Which HP tools plugin ?
Jenkins_POC is a testset name ? ————– YES
Also Let me know, what options you have for Run Type drop down..–
Run Locally
Thanks,
Chandrakant Bhirud
LikeLike
Which version of HP Tools Plugin are you using, I don’t see the options you are selecting. …..
Please let me know the plug in name and where can i select it ?
LikeLike
I have downloaded and installed this plugin :
hp-application-automation-tools-plugin.hpi
LikeLike
Goto Jenkins home page -> Manage Jenkins ->Manage plugin So-> update tab select HP tools automation plugin and click download now and instal after restart..
LikeLike
To check the version click on installed tab and find HP tools automation plugin..
LikeLike
Hi Sumit,
I have downloaded and installed this plugin :
hp-application-automation-tools-plugin.hpi
LikeLike
About HP Application Automation Tools 4.0.1
LikeLike
Hi Sumit,
Now you know everything about the configuration and set up on my machine, could you please help me in understanding why i am getting the
error : Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
Report not found
Build step ‘Publish HP tests result’ changed build result to FAILURE
Finished: FAILURE
LikeLike
I am getting the following error:
Console Output
Started by user admin
Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
[Test_ALM] $ “C:\Program Files (x86)\Jenkins\workspace\Test_ALM\HpToolsLauncher.exe” -paramfile props24062016183419987.txt
“Started…”
Timeout is set to: -1
Run mode is set to: RUN_LOCAL
Server has been disconnected while performing GetServerSettings action
Error: “ALM server http://sm09520.dom1.e-ssi.net:8080/qcbin unreachable, check that server Url is correct”
Build step ‘Execute HP functional tests from HP ALM’ changed build result to FAILURE
Finished: FAILURE
LikeLike
Hi Sumit,
I have sent you the jenkin configurations for ALM integration through email, could you please check it and let me know, what i am missing.
Thanks for your help !!!
Regards,
Chandrakant
LikeLike
Hi Chandu,
Below is the analysis results,
1) I downloaded the code for the plugin and its failing to create the OTA connection object. Its unable to initialize the ALM application.
Below is the piece of code which is failing
///
///
///
///
public bool ConnectToProject(string QCServerURL, string QCLogin, string QCPass, string QCDomain, string QCProject)
{
if (string.IsNullOrWhiteSpace(QCServerURL)
|| string.IsNullOrWhiteSpace(QCLogin)
|| string.IsNullOrWhiteSpace(QCDomain)
|| string.IsNullOrWhiteSpace(QCProject))
{
ConsoleWriter.WriteLine(Resources.AlmRunnerConnParamEmpty);
return false;
}
try
{
TdConnection.InitConnectionEx(QCServerURL);
}
catch (Exception ex)
{
ConsoleWriter.WriteLine(ex.Message);
}
if (!TdConnection.Connected)
{
ConsoleWriter.WriteErrLine(string.Format(Resources.AlmRunnerServerUnreachable, QCServerURL));
return false;
}
try
{
TdConnection.Login(QCLogin, QCPass);
}
catch (Exception ex)
{
ConsoleWriter.WriteLine(ex.Message);
}
if (!TdConnection.LoggedIn)
{
ConsoleWriter.WriteErrLine(Resources.AlmRunnerErrorAuthorization);
return false;
}
try
{
TdConnection.Connect(QCDomain, QCProject);
}
catch (Exception ex)
{
}
if (!TdConnection.ProjectConnected)
{
ConsoleWriter.WriteErrLine(Resources.AlmRunnerErrorConnectToProj);
return false;
}
return true;
}
In the above code, following line of code is failing
TdConnection.InitConnectionEx(QCServerURL);
What you can try to do is, search for VBA to connect to ALM using OTA api, and try to add the reference to OTA client in excel macros and run that VBA. If you are getting the error at the same line then your installation of ALM is corrupted, for sure.
If you can execute the VBA, then we can raise this as a bug for the plugin.
In the meantime, I will try to execute one script from ALM to check, if its able to detect my ALM with the latest version of plugin, and I will update you the results.
Let me know, if cant find the VBA OTA code to connect to ALM, I will write it and send it to you.
Thanks,
Sumeet Singh Kushwah
LikeLike
Hi Sumit,
I have executed the following code using UFT & Its working .
Set TDConnection = CreateObject(“TDApiOle80.TDConnection”)
TDConnection.InitConnection “ALM URL” TDConnection.Login “UID”,”PWD”
TDConnection.Connect “ENERGY_WHOLESALE”,”QC_EGC_Neon” ‘
If TDConnection.Connected Then
MsgBox(“Connected to ” + chr (13) + “Server ” + TDConnection.ServerName + chr (13) +”Project ” + TDConnection.ProjectName )
Else
MsgBox(“Not Connected”)
End If
Thanks,
Chandu
LikeLike
I am using jenkins-1.624
before i installed the jenkins-2.10 but i was not getting any plugin installed, so i installed old version…
LikeLike
Hi Sumit,
Did you get any solution for this, do i need to install any other version of Jeniks? or what/…
please suggest…
Thanks,
Chandra
LikeLike
Hi,
Put this code in the excel macros and add the reference to OTA COM type library. Update your ALM Server name and provide your user name and password.
Sub c()
Dim OConn As New TDAPIOLELib.TDConnection
OConn.InitConnectionEx “[ALM SERVER URL GOES HERE]/qcbin”
If OConn Is Nothing Then
MsgBox “Cant initialize connection object”
Else
OConn.Login “[USER NAME GOES HERE]”, “[PASSWORD GOES HERE]”
If OConn.LoggedIn Then
MsgBox “Logged in”
OConn.Logout
End If
Set OConn = Nothing
End If
End Sub
Because the code you sent uses “InitConnection” which is only available for HP use and the code of the plugin uses “Initconnectionex” also plugin is trying to access TD Connection outside of HP environment. So lets try to do that once, I think you should get the error. I was busy recently so I have not tried executing tests from ALM using the plugin, but I will try to do that today and let you know the results.
Thanks,
sumeet singh kushwah
LikeLike
Hi Sumit
Today I am not feeling well but tomorrow definitely I will try this.
Thank you very much for your kind suport and all help even though you are busy in your work.
Really appreciate you kind support !!!
Regards,
Chandrakant
LikeLike
Hi Sumit,
Thank you very much for your kind support, I will execute and let you know the results.
Even though you are busy in your work but still you are very much helpful.
Really appreciate your kind support !!!
Regards,
Chandra
LikeLike
hi Sumit,
Today i executed the code in excem macros and its working fine, its goving the message box “logged In”
I am sharing the screen shot with you in email.
Thanks,
Chandra
LikeLike
Hi Sumit,
Could you please help me in executing the ALM script from Jenkins.
Thnaks,
Chandra
LikeLike
Hi sumit…if you know could you please help me out onthe same thing
LikeLike
Hi, I would like to run the same UFT tests in different environments. The UFT jobs are triggered from Jenkins pipeline and I would like to pass the environment from the Jenkins job to UFT. Is there a way I can achieve this ? Thanks
LikeLike
Have you tried using MTBX file for calling UFT tests from Jenkins.. I have written an article.. Passing UFT test parameters from Jenkins..
https://sumeetkushwah.com/2015/08/25/how-to-pass-uft-test-parameter-values-from-jenkins/
Let me know if you need anything else..
LikeLike
Hi Sumeet,
We use Jenkins ALM plugin to invoke our UFT scripts(both UI and API) in our CI/CD pipeline. We have a reusuable script concept where we call the same script in multiple places during execution and it is parametrized. We did this to avoid having multiple scripts(for easy maintenance). However, when the execution happens from Jenkins ( we are running tests locally on jenkins slave, we have installed UFT on slaves, because our jenkins is on Production box and ALM is in Test env and firewall restrictions doesn’t allow us to run tests on remote host or planned host), it takes lot of time to load the scripts and complete the execution. Is there any way to reduce the execution time. Just to give you an idea, the script when executed locally from ALM on our desktops takes 3 mins to complete execution where as from Jenkins it takes about 45 mins. Thanks for your help
LikeLike
So when you execute the same script
1) open Alm and click on run button there to run the script on desktop A, it takes 3 mins
2) when you provide the root folder path in Jenkins where Jenkins slave is installed on desktop A, and build the job it takes 45 mins..
Is it correct ??
Thanks,
Sumeet singh kushwah
LikeLike
Yes Sumeet, that is correct
LikeLike
Hi please help me out…i am getting the below error:
“Started…”
Timeout is set to: 60
Run mode is set to: RUN_LOCAL
Server has been disconnected while performing GetServerSettings action
Error: “ALM server https://—————/qcbin unreachable, check that server Url is correct”
Build step ‘Execute HP functional tests from HP ALM’ changed build result to FAILURE
Failed login to ALM Server URL: https://————–/qcbin/. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
Empty Results
Result Status: UNSTABLE
LikeLike
Could you please help me in executing the ALM script from Jenkins.
LikeLike
HI Kushwa,
We are trying to run UFT scripts form ALM using jenkins ,when the build runs the QTPRemoteagent.exe and uft.exe both proceses are sowing on VM but uft tool is not opening and not executing the test,when we ran similar test cases from ALM on same remote machine it is working file
LikeLike
When you execute UFT tests from Jenkins, UFT will run in hidden mode but your test execution should run fine.
Are you getting any errors in Jenkins console output ??
LikeLike
Hi Kushwah,
I am getting RPC server unavailbale and we did check the port 135 is opened and it is opened and also I unchecked the run hidden mode option in in Remoteagent settings
LikeLike
Ok. I don’t think that remote agent settings will impact the Jenkins executions.
1) how Jenkins is installed(windows service, Command prompt, Tomcat etc) and how slave is installed ?
2) can you try to execute the test on slave using a VBS file and let me know the results.
3) can you send me the console output and Jenkins job configurations.
I have not faced this issue, but I can try to replicate it.
LikeLike
Hi Kushwah,
Jenkins installed as windows service
I tried the test running using VBS files don’t see any issue
and also when running from ALM same testset executing fine
Run mode is set to: RUN_REMOTE
============================================================================
Starting test set execution
Test set name: Jenkins, Test set id: 2502
“Number of tests in set: “2
Test 1: [1]UFT_Jenkins will run on host: “XXXXX”
Test 2: [1]UFT_Jenkins_Copy_1 will run on host: “XXXX”
“Scheduler started at:20/09/2016 14:10:58
——————————————————————————————————-
20/09/2016 14:10:58 Running: [1]UFT_Jenkins
20/09/2016 14:10:58 Running test: [1]UFT_Jenkins, Test id: 1201, Test instance id: 3014
Test: [1]UFT_Jenkins, Id: 3014, Execution status: Running
Test: [1]UFT_Jenkins, Id: 3014, Execution status: Error, Message: The RPC server is unavailable
20/09/2016 14:11:52 Test complete: [1]UFT_Jenkins
——————————————————————————————————-
20/09/2016 14:11:52 Running: [1]UFT_Jenkins_Copy_1
20/09/2016 14:11:52 Running test: [1]UFT_Jenkins_Copy_1, Test id: 8428, Test instance id: 3015
Test: [1]UFT_Jenkins_Copy_1, Id: 3015, Execution status: Running
Test: [1]UFT_Jenkins_Copy_1, Id: 3015, Execution status: Error, Message: The RPC server is unavailable
20/09/2016 14:12:37 Test complete: [1]UFT_Jenkins_Copy_1
LikeLiked by 1 person
Ok so below are the few things you can try
1) install TD connect.exe on the slave machine.
2) change the option to Run locally in the job configuration.
3) set the option restric where this project can be run to the slave machine.
4) check allow other HP products to run tests and components.
5) make sure that you can login to ALM on the slave machine.
LikeLike
Hi Kushwah,
I am executing my script on remote machine reason why selected run on remote and I am giving the remote machine name I will try other options you mentioned
LikeLiked by 1 person
Hi Krish,
Were you able to resolve this issue because I am also getting the exact issue. Pointers would be much appreciated.
Regards,
Pooja
LikeLiked by 1 person
Hi Pooja,
What issue are u getting ?
Thanks,
Sumeet
LikeLike
How to run UFT failed test cases?
LikeLike
Hi Suraj,
I am marking failed test for executions in my Input sheet. Then I will just for re-execute the Job to execute the failed tests.
Thanks,
Sumeet Singh Kushwah
LikeLike
Hi Sumeet,
Means your writing failed test case in Excel sheet or some file and than Re-triggering the those test case manually??
Thanks,
Suraj Naik
LikeLike
Hi Suraj,
My main driver script reads the input excel sheet and execute the tests. If any test is failed it will be marked for execution. Now after the execution ends, my test will call the main driver script again, because failed tests are only marked for execution, main driver script will only run the failed tests. Main driver is called automatically twice by the script.
Thanks
Sumeet
LikeLike
Hi Sumeet,
I have added “proceed to next action iteration if any error comes and iam disabling the reporter filter(rfdiableall) in beginning but still Jenkin catches the error details and even test passes after recovery steps still it catching the failures and showing the fail in Jenkins job
Any work around for this?
LikeLike
Hey Ramu
Yes I had the same issue
1) add the post build step to force Jenkins to pass/success
2) create a junit file(which is explained in article). List the passed and failed tests in this junit file and upload that to Jenkins after the post build step.
3) uploading junit with test case status will post the write status on Jenkins.
4) remove the post build step to upload UFT run results on Jenkins. This will make sure that Jenkins is not getting results twice (from your junit and UFT results). Jenkins will still get the final error info if test fails.
The only way for Jenkins to pass the test is if no error is thrown and no reporter.reportevent micfail statements.
If you want to see how to avoid UFT from throwing errors then read the article below
https://sumeetkushwah.com/2014/08/12/implementing-try-catch-functionality-in-qtpuft/
Thanks,
sumeet Singh Kushwah
LikeLike
Hi Sumeeth,
Thanks for the Answer and will try and let you know the out come.
Could you please let me know how to run the test twice if fails immediately as you aware in QC we have option to run the test automatically ‘n’ number of times and jenkins will take the latest result which is good but when we are using MTBX ,i dont see any option running failed one immediatly and running whole bunch of test cases which included passed.
I have added naginator plugin but it is only running after execution completed and running all test cases again
Best Regards,
Ramu Gogurla
LikeLike
Hi Sumeeth, we have found solution for this, when scripts run in jenkins it will create 2 files param….txr and result….xml, from that we can get pass or fail
LikeLike
Hi Sumeet,
Very useful info in your forum. thanks.
I have tried the steps which you have mentioned above. but i encountered with the below error on execution.
“>>>> File/Folder not found: ‘X:\ATAF’
No valid tests were found. Fix the test paths.
Build step ‘Execute HP tests from file system’ changed build result to FAILURE”
Could you please provide more info on Test path. Am not sure what info needs to be provided there.
Bcos all the test scenarios and data are kept in svn itself and that is mapped in config.xml.
Please help me on this regard.
Regards,
Nandha
LikeLiked by 1 person
Hi Nandha,
If X: is shared drive mapped folder path, then it will not work. You must always provide the full folder path.
Thanks,
Sumeet
LikeLike
Hello,
I am usings Jenkins 2.19.2 in windows.
I have also installed plugin – HP Application Automation tools 4.5.0
but unable to build project using ALM12.01
I go to Manage Jenkins>Configure system>Enter ALM details.
Here when i enter the ALM url which i use for accessing via URL. I get following error – “Error openning a connection to the ALM server”
(I am able to open in IE ALM12 with the same URL as entered in Jenkins.But still i get this error)
Ignoring this error i configure my build with step “Execute functional tests from HP ALM” and enter all details
In test sets i have given the complete path of the test set i want to run
( eg: Root\Scheduler\SchedulerDummy1)
Finally when i build this i get following message:
Console Output:
Started by user admin
Building in workspace C:\Program Files (x86)\Jenkins\workspace\FirstProj
[FirstProj] $ “C:\Program Files (x86)\Jenkins\workspace\FirstProj\HpToolsLauncher.exe” -paramfile props03112016213435061.txt
“Started…”
Timeout is set to: -1
Run mode is set to: RUN_LOCAL
—————————————————————
And i get a round loading symbol which goes on and on….(i waited for 30 min …but nothing happened.Nor am i getting any error message.
Could anyone please help me resolve this?
If any other details required please let me know.
LikeLiked by 1 person
Hi Shibani,
What is the URL you are using in jenkins ?
Your should use below ALM url in jenkins
https:///qcbin
your URL should end at QCBin
Thanks,
Sumeet
LikeLike
Regarding my earlier post How to run UFT failed test cases?
Please give some example how i achieve this, i running daily Test set for example i have 20 tests are there in test set, 15 will pass out of 20 so i need run remaining 5 tests manually, how i can i do through Jenkins
LikeLiked by 1 person
Hi Suraj,
You can’t do this with Jenkins(No easy way), you will have to implement this in UFT.
But here is what I have implemented
I have an “executeTests” function which will loop through the input excel sheet with the tests.
This “executeTests” function has two for loops
1) The inner loop which will execute each test listed in the Input excel sheet.
2) The outer loop which runs twice so that the inner loop can be executed twice hence all of the tests will be executed twice.
The inner loop also keeps track of the test cases names which we have executed, so if we have already executed a test, then inner loop will only re-execute it if the status is failed for the test.
If you can send me the details about your framework structure, I can help you in implementing this concept for re-executing the failed tests automatically.
Thanks,
Sumeet Singh kushwah
LikeLike
Hi Sumit,
Following is my driver script
RunAction “OpenJavaApp”, oneIteration
RunAction “TestSomeApp”, oneIteration
RunAction “CloseTheApp”, oneIteration
Here 2nd RunAction may fail due to some Network connectivity or object is not visible if i ran same it will pass
Please can you help me how i can implement same
LikeLike
Hi Suraj,
Below is the updated code
Dim ArrayToStoreTestResults
”assign false for executing the test for the first time
ArrayToStoreTestResults = false
”’ This will make sure that the test is executed twice
For ExecutionCounter=0 to 1 Step 1
”’Store the test case execution results in the array ArrayToStoreTestResults
If ArrayToStoreTestResults = false then
RunAction “OpenJavaApp”, oneIteration
”’ Change ArrayToStoreTestResults to true if the test is passed, else change it to false
RunAction “TestSomeApp”, oneIteration
RunAction “CloseTheApp”, oneIteration
End IF
Next
Below is the image of the reports generated on jenkins for my executions
Let me know, if you need anything else
Thanks,
Sumeet
LikeLike
Thanks Sumeet
Every RunAction i have to update the excel sheet ? ro once completion of all test
LikeLike
how to run multiple QTP scripts from QC on different machines
Scenario: how to run multiple QTP scripts from QC on different machines, say for example 100 script in QC test lab, to run on 5 machines, each machine having 20 scripts, while running all at a time from QC, if one of the machine’s h/w got crashed, so those scripts should automatically distribute or run on any available machine.
any idea plz?
LikeLiked by 1 person
Hi,
Currently its implemented in our environment and I don’t think that it can be achieved by using ALM alone.
What We have implemented is below
1) All of our scripts are running from Jenkins.
2) We have created (nodes)machines in jenkins and assigned labels to the machines. So If we have 5 machines then the machine names will be machine 1, machine 2……, Machine 5. But we have assigned labels for controlling which machines will pick up the tests, so if for app1 we have 100 tests and we want them to run on first three machines, then in the Jenkins nodes(Machine) configuration we will add the label “APP1” for all three machines and in jenkins job configuration, we will add the machine name to execute the test as “APP1” instead of Machine 1…N. This will create a pool of three machines with label APP1. Now all three machines will run the tests as soon as we start the executions.
3) Jenkins will pick the machines on the go, So if one machine goes down then, it wont use that machine and executions will continue on the remaining two machine.
I have seen something called application pool in the ALM Lab Manager, but I don’t know, if that can be used in the similar way.
Please let me know, If you have ALM Lab manager available in your ALM, then I can try to provide you some details about it or if you need anything else.
Thanks,
Sumeet Singh Kushwah
LikeLike
UFT Headless execution for the most part does not work reliably.
LikeLiked by 2 people
There are so many variables involved like your framework, machine configs etc. but it does work fine. I have more then 2500 tests executing through Jenkins and we never had problems in executions.
LikeLiked by 1 person
Headless and running on a different machine ?
LikeLiked by 1 person
I had a defect open with HP and we could not solve it. It would loose video context and miss Ui control actions from time to time. It was not reliabile. HP stated we should keep a VM session active and open
before the automation was remotely kicked off.
Dave
LikeLiked by 1 person
Hi,
Below is the screenshot of UFT test execution from Jenkins
I am not doing it on VM’s I have physical machines. So, I don’t know if VM’s are causing the issue. But I have scripts to make sure that windows is logged in and not locked during the UFT executions.
All of my tests are running on different machines.
Thanks,
Sumeet
LikeLike
Hi Sumeet,
I have a new challenge. I am trying to execute UFT tests from Jenkins using GIT source code.
Now my challenge, how would be version control the action UFT script and xls files. We have already version controlled out function library and the object repo but not quite sure how to tackle the xls and uft test files we have associated with our frame work.
Have you come across something similar which I could look into?
Many thanks
Ajay
LikeLiked by 1 person
Hello Summet,
Currently i am running tests in VM but sometimes Jenkins through following error and Jobs will stop
Logs:
Unhandled Exception: System.NullReferenceException: Your Quality Center session has been disconnected. Contact your system administrator for more information
at Mercury.TD.Client.Ota.QC9.ITSTest.get_LastRun()
at HpToolsLauncher.AlmTestSetsRunner.GetTestRunId(ITSTest currentTest)
at HpToolsLauncher.AlmTestSetsRunner.RunTestSet(String tsFolderName, String tsName, Double timeout, QcRunMode runMode, String runHost)
at HpToolsLauncher.AlmTestSetsRunner.Run()
at HpToolsLauncher.Launcher.RunTests(IAssetRunner runner, String resultsFile)
at HpToolsLauncher.Launcher.Run()
at HpToolsLauncher.Program.Main(String[] args)
An error occured while disposing runner: Your Quality Center session has been disconnected. Contact your system administrator for more information.
Thanks,
Suraj NAIK
LikeLike
Pingback: UFT GIT Jenkins Integration | Automation Insights·
Hello Sumeet,
I was trying to see if there is anyway to run a single test case from a testset from HP ALM using Jenkins? If yes. please specify..
The goal is to run some custom vb script to upload screen shots to ALM after each test case ran and proceed to run other testcase …
LikeLike
Hi Summet,
We are trying to run the Load runner scripts integrating with Jenkins. We are executing the scenarios with the batch file by selecting the “Execute windows batch command” under the build section.
“C:\Program Files (x86)\HP\LoadRunner\bin\Wlrun.exe” -Run -TestPath “” -ResultName “”
We are able to view the results in the Results folder .However, we are getting the error at the Console Output “Build step ‘Execute Windows batch command’ marked build as failure”.
Please find below console output window message
———————————————————————-
Started by user
Building on master in workspace C:\Program Files (x86)\Jenkins\workspace\MyLoadRunner
[MyLoadRunner] $ cmd /c call C:\Windows\TEMP\hudson695684065292801928.bat
C:\Program Files (x86)\Jenkins\workspace\MyLoadRunner>C:\Batchfile\LR.bat
C:\Program Files (x86)\Jenkins\workspace\MyLoadRunner>”C:\Program Files (x86)\HP\LoadRunner\bin\Wlrun.exe” -Run -TestPath “C:\LRScenarios\Scenario1.lrs” -ResultName “C:\LRResults\Run6”
Build step ‘Execute Windows batch command’ marked build as failure
Finished: FAILURE
LikeLike
Hi Sumeet,
I am also facing an issue with ‘create scheduler’ which is mentioned earlier in the comments section
Error:
Could not create scheduler, please verify ALM client installation on run machine by downloading and in installing the add-in form:
But if I call manually command line seems to work (calling directly: HpToolsLauncher.exe -paramfile propsXXXXXXXXXXX.txt”), but when calling the task from Jenkins, it fails.
Could you please help me in this.
LikeLike
Are u using a windows 10 machine or windows 7 ??
LikeLike
Also how is your Jenkins slave configured ? As windows service ??
LikeLike
Executing this command inside IE opened as Admin resolved the same issue for me
http://AlmServer:8080/qcbin/start_a.jsp?common=true
Thanks,
Sumeet
LikeLike
Hi Summet,
I am also facing an issues which is mentioned in comments section earlier
Issue:
Could not create scheduler, please verify ALM client installation on run machine by downloading and in installing the add-in form: http://xxxxxxxx/qcbin/TDConnectivity_index.html
i have tried the solution which you mentioned but its still not working
but if we call manually the command line seems to work (calling directly: HpToolsLauncher.exe -paramfile propsXXXXXXXXXXX.txt”), but when calling the task from Jenkins, it fails.
Could you please help?
LikeLike
How is Jenkins slave configured ?? As a windows service ???
LikeLike
Executing this command inside IE opened as Admin resolved the same issue for me
http://AlmServer:8080/qcbin/start_a.jsp?common=true
Thanks,
Sumeet
LikeLike
Hi Sumeet,
I am running a Jenkins Free Style Project for UFT Windows based Application. I am using Hybrid Framework for Automation.
When I normally run Script from my local, it stores the Results in Run Results_Folder inside my Project Folder Structure. Also we can see it from Run Results Viewer.
My Concern is that I want to store theses reportswhich are handled in my Framework in Jenkins when Build is Run as a post Build Action.
Not Sure about the Plugin required. Can you please guide here
LikeLike
I am using archive the artifacts post build action to move the uft test results to Jenkins. My framework copies the results to Jenkins workspace and then Jenkins archives it
LikeLike
Hello,
can you forward a format/sample of the propsXXXXX.txt ?
Thank You
Dermot
LikeLike
Hi,
Thanks for the wonderful information. I have 1000 scripts from file system listed in the Build and 10 machines labelled. can you guide me on how to configure Jenkins to setup parallel execution. That is have Jenkins trigger each test case in the list on one of the available slaves
in parallel. currently the scripts execute in sequence though i have multiple slaves available.
Thanks
Vivek
LikeLike
Hi Vivek,
You need to assign the labels to the slave machine config. You can assign multiple labels to one machine. Labels can be separated using a space, so I can have labels like “facebook_machines linkedin_machines”.(don’t use double quotes). Assign same label names to all machine where u want to execute parallel tests.
Once you have the labels assigned, then u can use these label names in the jobs under the option to “restrict where this job can be executed”. Use one label name there like facebook_machines. Change this config for all the jobs you want to run in parallel.
After this if you trigger multiple jobs, Jenkins will automatically assign the machine out of one of the slaves where u have configured label names.
Let me know, if u need any other details.
LikeLike
Thanks for the quick reply. Will give it a try
LikeLike
Hello Sumeet,
I was able to assign same labels to 2 different machines and on job level i restricted it with those labels and now the job has 2 nodes to service the job.
However, I would like to know if one jenkins job has let’s say 10 test cases under build section. Will jenkins be able to distribute those 10 test cases to all the available slaves (2 in my case) lets say 5 on one slave and another 5 different and run them in parallel?
Regards,
Alankar
LikeLike
Hi Vivek,
Is your question regarding running multiple jenkins jobs in parallel or running multiple tests(within build section) in ONE jenkins job in parallel?
LikeLike
In my assignment I have no sorucecode repo like SVN and Maven, we have complete framework which run from driverscript, In that case how do I run my tests.?
LikeLike
Where is your driver script, in UFT to run the tests or it’s a vbs which runs different uft tests ??
LikeLike