Executing automated UFT\ALM tests from Jenkins

Get Trained and start executing UFT tests from Jenkins. Click Here for more details.

Hello Friends,

In this post, we will discuss integrating Jenkins CI tool with HP UFT.

Introduction

Jenkins is a CI tool which basically monitors external job executions and it’s mainly used by developers for building and deployment of code. It provides several functions like scheduling of tasks called jobs in several remote machines called as slaves in terms of Jenkins vocabulary. In this post, we are going to learn how to setup Jenkins from scratch for executing UFT automated test cases.

First, we need to understand, what is the need for running the automated tests with CI tools? It’s not just applicable for Jenkins, its applicable for any other CI tools or automation tools available in the market. Most of the people are going to tell you about agile and Smoke testing, those are correct and we will go back to this topic in other posts. First, we will look into more common issues which teams will face.

I was very happy with learning new languages and implementing new things for my project, and then one day we did some analysis and it seemed that we were spending most of our time in executing older regression test suites and we have only 20-30 % of our time left for development of new test cases, and it’s the problem which was growing day by day more severe.

So, the only solution for this problem is to assign the execution of the scripts to the actual owners. It seemed ideal, but in the real world that’s not as easy as we think, everybody has their own tasks to be completed, plus they didn’t have the knowledge for using UFT and you will be pushed back to ground zero, from where you started.

On the other side, tests we had were automated test cases, and hence by definition, they are supposed to be run automatically. But due to the manual interventions required due to the nature of scripts or application, it was close to impossible to execute the most of the scripts on a headless remote machine.

To resolve these issues we tried HP ALM’s Lab management tool, but I realized very soon that it can only execute the test cases so if you have to execute a bunch of scripts before or after execution of the scripts, you cannot. Also, there is no way to approve the results and a lot of time issues with the remote machine cause the test to fail.

There is another way of doing it using Automation Object model, but I really don’t want to take the headache of maintaining another piece of code which might change between versions of UFT.

I tried Jenkins and I am still using it, basically, it does everything, let’s see how…

Installation of Jenkins Server

If you don’t have Jenkins server, then follow this guide for the installation of the server. If you have installed Jenkins server using windows setup, you can access Jenkins server using the following URL

http://ServerName:8080

Check your Jenkins workspace path and update if required.

Create slave machine for the execution of the UFT scripts(Machine where UFT is installed), and install the following plug-ins

List of required plugins for the setup

  • Copy Artifact Plugin
  • Dashboard View Plugin
  • Discard Old Build Plugin
  • Email Extension Plugin
  • Extra Columns Plugin
  • Flexible Publish Plugin
  • HP Application Automation Tools Plugin
  • Test Result Analyzer Plugin
  • Time Stamper Plugin
  • Workspace Cleanup Plugin

Most of them are self-explanatory, but if you need any other details then, Google with the exact name of the plugin.

Creating Jenkins job

Once, you are able to open the Jenkins server, Click on “New Item”

Jenkins New Job

Provide any Item name, select “freestyle project” and click on “OK” button

Jenkins New Job Name

Once, you click on OK button, New Job configuration page will open. This is the main configuration page, where we will provide all of the configuration details about how the Job should execute.

Fill out the following information within the job configuration page

1) Description: Description about the job and the UFT test which is tied to this job.

2) Check on “Restrict where this project can be run” and provide the slave machine name with UFT installed. This is a very important step, because otherwise Jenkins will choose a slave machine randomly, and might choose Jenkins master machine (where Jenkins is installed) for execution. This will cause the job to fail.

3) Select appropriate build triggers, if you are using any source code control for storing UFT tests (Except ALM) e.g. GIT, SVN, TFS.

4) Under Build environment select “delete workspace before build starts”. This will help us in cleaning up the workspace and have a fresh start for each execution.

5) Click on Add build step button under build and select “Execute HP Tests from file system”. This will allow you to execute the tests from file system e.g. “C:\” or network drives.

Jenkins Execute HP tests From File System

6) Provide UFT tests or Solution path in the “Tests” text box. If you provide UFT’s solution file path then all tests under solution will be executed one after another. Leave Time out seconds blank.

7) Click on “Add post build action” under post build actions, and select “Publish HP test results”. Select “Always Archive HP test Reports”. This step will generate and archive Junit results for the execution on the Jenkins server, which can be used for analysis. we will discuss this in detail later.

8) If you have configured the email server then Add another post build step and “Editable email notification”. This option will send automated emails after the execution is over. We can attach any files in the email or we can send any files as the part of the email body.

After filling out the above information, Click on Save button and start the job by clicking on “Build now” button within job details page.

Jenkins Build Now

Once the execution is started, click on Build number, under build history, to check the current execution details

Jenkins Build Number

While the test is executing, you can click on Console output, to check the current execution status.

Jenkins Console Output

Once the execution is completed, you can see the test results on the Build page, under test results

Jenkins test output

This completes the basic configuration of the Jenkins Job.

How it works?

When you execute a UFT test from Jenkins, HP automation tools plugin will copy the following files to the Jenkins Job workspace directory

1) HpToolsLauncher.exe – This is the file used for actual execution of the UFT tests .when you take a closer look at the console output of the Jenkins build, you will see a line

"C:\Jenkins\workspace\JENKINS UFT TEST\HpToolsLauncher.exe" -paramfile props23032015132505096.txt

If you execute this line on the slave machine UFT test will execute.

2) LRAnalysisLauncher.exe – For load runner.

3) propsXXXXX.txt – This is the properties file for the HP Automation tools plugin. This will store different parameters required for the execution of UFT/Load tests.

4) ResultsXXXXX.xml – This is the JUnit results generated by the HpToolsLauncher.exe after the execution is completed. When we choose to create a post build step and choose to “Publish HP run results”. This Junit file will be uploaded back to the Jenkins and produce the test results shown in step 12 above.

General Issues faced during test execution

There are several issues you may encounter during the execution of UFT tests from Jenkins

1) If scripts have any “MsgBox” statement or any “InputBox” statement, scripts will directly fail.

2) If the machine is unlocked, then there is a chance for complex applications to fail.

3) We have to make sure that machine is cleaned up before and after the execution of tests(Closing all browsers and any other types of files like excel sheets or word documents).

4) We have to make sure that UFT scripts have a proper error handling mechanism or UFT will throw an error message and Jenkins Job will fail as soon as Popup appears on the page.

5) If we have a proper framework for our scripts, then most likely we will be executing multiple tests within one UFT test. Because Jenkins HP Automation tool can only count UFT tests and there is no way to indicate no. of tests, if you execute 2 UFT tests having 20 tests each, Jenkins dashboard will only display 2 tests executed, not 40 tests executed.

6) If UFT test report’s a failure in any of the tests, you will see the failure of Jenkins complete Job instead of a particular test case.

let’s see how to resolve each of these issues to create a perfect Jenkins job for executing automated UFT tests.

1) Replace any statement which requires human intervention

We must scan our scripts and replace any statements which require human intervention like MsgBox statements must be commented and should only be used during debugging. For any steps which use Input boxes for accepting inputs, use test parameters and action parameters. For receiving information back, use reporter.reportevent instead of msgbox statements.

2) Use Physical desktops, instead of virtual machines

If we use virtual machines as Jenkins slave machines, then we will have to make sure that the machines are up and running all the time, because virtual machine can go blank without any prior warning, if the user disconnects them.

3) Unlocking remote slave machine before executions

To overcome this issues, if possible change the machine setting, so that machine never locks automatically. But this option is not available in most of the companies because the machines running UFT will most probably have admin rights and it’s not advisable to leave the machine with admin rights unlocked.

First test your scripts, if they are able to run in the locked machine. If not then, you can use the following commands to unlock the Physical desktop machine from another computer (In case you are at home or other location).

For Unlocking the Remote Desktop(Physical machine)

Run on the command prompt

Query User

This command will provide the Session ID for your Login Session

JenkinsQueryUser

Once you have the session id of your session execute the following command

Tscon <Session id from previous commapn> /DEST:Console

This command will unlock the desktop machine and you can execute your test normally.

4) Include Machine Cleanup Framework in UFT

Machine cleanup framework is explained here. This framework will use the VBScript classes for executing predefined cleanup code. This is required because, if the previous execution of code leaves some things open then, your subsequent executions might fail.

5) Include proper error handling mechanism in your scripts

Recovery scenarios cannot handle errors properly, for running tests from Jenkins you will need a way to handle all kinds of error, including UFT’s general run error’s. Because, if UFT throws a general run error, then your Jenkins execution will fail, while UFT will be waiting for someone to click on the error and close the test manually and this will create issues with the subsequent test executions. A detailed description of how to implement this framework is explained here.

Executing tests in Parallel

For execution multiple tests in parallel, you can choose to do one of these two methods

1) Open the tests and update the Label expression under “Restrict where this project can be run”, with the slave machine name. Once you update the two different tests with different machine names, you can execute the tests in parallel on those two machines (provided you have the proper licenses for executions).

2) Now, the previous method works, but that’s not the best way to execute parallel tests within Jenkins. To implement the second method you will have to figure out the machines with similar properties like “UFT MACHINE WITH IE 8”, so this is a specific machine having UFT installed and can be used for the apps which are only compatible with IE 8. Once you have all the machine characteristics identified you can then open the slave configurations for the machine and update these names in the Slave labels. Once this Logical grouping of the Machines is done then you can provide this logical name with “Restrict where this project can be run”, instead of machine names. Using this method, you wont have to actually remember about the machine names and go through the whole hard process to identifying which machine you should execute the test.

JenkinsSalveConfig

JenkinsRestrictWhereProjectsCanRun

One more advantage of this method is if you have two machine with Label “UFT MACHINE WITH IE 8”, and when you execute the test, if the one of the machine is busy, then Jenkins will automatically Pickup the second machine and use it for execution.

Building test queue(sequential test execution)

Jenkins will automatically build the execution queue, once you select more number of jobs then the machines you have. These tests will be executed automatically when the machine will become free. To check this, Update the “Restrict where this project can be run” of three Jobs with the same slave name. Now, click on home page and Click on “Schedule a build” button for all three jobs, one after another and you will see the build queue, on the left side of the home page under “Build queue”.

Executing tests from ALM

For running tests from ALM, you have to make three changes to the current configuration explained above.

1) Add ALM Server

open Jenkins -> Manage Jenkins -> Configure System and find the sub section “Application Lifecycle Management”.

Click on “Add ALM Server” button. Provide the server name as Test or Prod or Dev and enter the server URL till qcbin. Click on save and apply.

You can add multiple ALM Servers.

2) Use “Execute HP Functional test from HP ALM” instead of “Execute HP tests from file system”.

Add the Build Step “Execute HP Functional test from file system”. Select ALM Server name from the “ALM Server” combobox. Provide Username, password, Domain, project and Test Set Path for the execution of the tests. Test set path is the actual physical path which you see in the ALM test lab tree structure starting with Root\

JenkinsExecuteTestFromALMServer

3) Check the advanced setting,

if you want to execute the tests on specific host defined in ALM.

You will see the following changes in the execution process

Modified Console Output

When you execute your tests from ALM Server, you will see one change in the Console output, you will start seeing all of the Reporter.reportevent statements on the Jenkins console log.

This feature can be used to check the current progress of the execution.

Automatic Execution update in ALM

Test execution status will be uploaded automatically to ALM Server.

Additional helpful setting\upgrades

1) Instead of using “Publish HP Run results option”, Create and upload your our own Junit results

When, one UFT test executes 100 test cases, HP Automation tools plugin will only return either one pass or one fail. To over come this create your own Junit results xml file at runtime after the UFT execution is completed. This will also serve as a checkpoint, if the UFT execution is completed successfully then only we should have a JUnit file generated.

Once you have this functionality implemented, you will start seeing awsome results on Jenkins and you will unlock the real power of Jenkins. JUnit results XML will look like this

<testsuites>
<testsuite Name=“FirstAutomation” test=“2” failures=“0” time=““>
<testcase name=“test1” classname=“FirstAutomation.TestCases” time=“27.68359” status=“pass”>
<testcase name=“test2” classname=“FirstAutomation.TestCases” time=“13.33203” status=“pass”>
</testsuite>
</testsuites>
When you publish this file after one UFT test execution you will see two passed test instead of one passed tests.
JenkinsJunitUploadedResults
In this above example, i have executed several tests and 17 of them failed. All of the tests were under one UFT test
To publish the JUnit results, add the post build step “Publish Junit test result report”. Provide the JUnit file path and enter 1.0 in the Health report amplification factor.

2) Upload test Artifacts to Jenkins

Jenkins will allow to upload the artifacts back to the Jenkins server. We have to use the copy artifacts plugin to do this. Within UFT you can create a function and call it within the Machine cleanup Framework’s class initialization(to make sure it executes) to setup the export of UFT datatable and UFT run results to the Jenkins workspace like this

Public Function SetQTPParameters()
Dim App 'As Application
Set App = CreateObject(“QuickTest.Application”)
App.Options.Run.AutoExportReportConfig.AutoExportResults = True
App.Options.Run.AutoExportReportConfig.StepDetailsReport = True
App.Options.Run.AutoExportReportConfig.DataTableReport = True
App.Options.Run.AutoExportReportConfig.LogTrackingReport = False
App.Options.Run.AutoExportReportConfig.ScreenRecorderReport = False
App.Options.Run.AutoExportReportConfig.SystemMonitorReport = False
App.Options.Run.AutoExportReportConfig.ExportLocation = JenkinsWorkSpacePath
App.Options.Run.AutoExportReportConfig.UserDefinedXSL = ““
App.Options.Run.AutoExportReportConfig.StepDetailsReportType = “HTML”
App.Options.Run.AutoExportReportConfig.StepDetailsReportFormat = “Detailed”
App.Options.Run.AutoExportReportConfig.ExportForFailedRunsOnly = False
End Function

After creating this function, on the Jenkins configuration, add a build step “Archive the artifacts”.  This is Upload the results back to the Jenkins server and will be visible for each execution. The default Archival path is Jenkins workspace, but we can also point it to the child folder inside jenkins workspace.

e.g. If Jenkins workspace path is “C:\Jenkins”(Create this in Slave Config -> Remote Root Directory) then the Job execution path will be “C:\Jenkins\JENKINS UFT TEST\”

to archive the artifacts from the child folder within workspace e.g. “C:\Jenkins\JENKINS UFT TEST\ExecutionResults”

use “ExecutionResults/*” in the files to archive textbox

In addition Upload any artifacts related to the test execution to Jenkins, if required.

3) Use Dashboard Plugin to create execution dashboards

Once, you have dashboard plugin installed in the Jenkins, create dashboard views and add test execution statistics Grid to it. This will pull the data from the Jobs latest JUnit files and publish the data on the Dashboard page Providing summary on the Dashboard home page.

JenkinsResultsDashboard

Above is the snippet of the Test execution statistics Grid.

4) Use Flexible publish to produce the results

Use the flexible publish plugin to figure out things like, if the files are generated before actually passing the Jenkins job or failing the Jenkins Job. One example is check if the Junit results are generate before archiving them to Jenkins server. If the file is absent then Jenkins will throws an error and fail the job.

5) Close and open HP Remote agent during each run session

I have included a separate batch file for running the remote agent launcher during each run session and close it after the execution is over. This will make sure that all of the UFT related processes and connection are re-established during each run session.

6) Use “test result analyzer” for result analysis

You can use the test result analyzer for analyzing the results of two separate executions. To Open, Click on “Test Result Analyzer” on the job description page.

Test Results Analyzer

If, we have 100’s of tests, it might not be very easy to analyze the results between two executions, but if you have Custom JUnit reports generated, then you can use this feature for analyzing the results between any two executions

7) Use Batch File to Kill UFT

Create a batch file to kill UFT and add this to build step after the step for the execution of the UFT Test. This will make sure that UFT is not hanged in between and also if the Jenkins Job is stopped due to some reason you will have the machine ready for the next one.

Let me know, if you have any questions and i will try to resolve them.

Thanks

276 responses to “Executing automated UFT\ALM tests from Jenkins

  1. Console Output
    Started by user admin
    Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
    Failed login to ALM Server URL: http://sm09520.dom1.e-ssi.net:8080/qcbin. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
    Empty Results
    Result Status: UNSTABLE
    Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
    Report not found
    Build step ‘Publish HP tests result’ changed build result to FAILURE
    Finished: FAILURE

    Like

  2. Hi Sumit,
    I am not able to put the screen shot, could you please help me , how can i upload the screen shot here?

    Thanks,
    Chandra

    Like

  3. where are you giving the URL for ALM, in job configuration or Jenkins configuration??? –,—In Jenkin Configuration

    Sumit, As i was not able to upload the screen shots here, i have sent you eamil, Could you please check it and look in to in…

    Thank you very much for your support till date …

    Like

  4. Jenkin Configuration:
    ALM server name : HP ALM 11
    ALM server URL : http://Server:8080/qcbin

    Job Configuration:
    Execute HP Test using HP AL LAb Management
    ALM server : HP ALM 11
    User name : my KID
    Password : Windows PWD
    Domain: ALM Domain
    Project: Project
    Run Type: Test Set
    Test Set/Build Verification Suite ID : Root\Automation_POC\Jenkin_POC
    Jenkin_POC is Test set in Test LAB

    Timeslot Duration (in minutes, minimum = 30) – 30

    Console Output
    Started by user admin
    Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
    Failed login to ALM Server URL: http://sm09520.dom1.e-ssi.net:8080/qcbin. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
    Empty Results
    Result Status: UNSTABLE
    Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
    Report not found
    Build step ‘Publish HP tests result’ changed build result to FAILURE
    Finished: FAILURE

    Like

  5. Jenkin Configuration:
    ALM server name : HP ALM 11
    ALM server URL : http://Server:8080/qcbin

    Job Configuration:
    Execute HP Test using HP AL LAb Management
    ALM server : HP ALM 11
    User name : my KID
    Password : Windows PWD
    Domain: ALM Domain
    Project: Project
    Run Type: Test Set
    Test Set/Build Verification Suite ID : Root\Automation_POC\Jenkin_POC
    Jenkin_POC is Test set in Test LAB

    Timeslot Duration (in minutes, minimum = 30) – 30

    Like

    • Hi,

      Which version of HP Tools Plugin are you using, I don’t see the options you are selecting.

      Timeout is an optional field, you can remove the values from the timeout field.

      I have never entered a value for the Environment configuration ID, so try clearing that field also and see.

      Jenkins_POC is a testset name ?

      I am sending you the screenshots for my configurations.

      Also Let me know, what options you have for Run Type drop down..

      Thanks,
      Sumeet Singh Kushwah

      Like

  6. Output
    Started by user admin
    Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
    Failed login to ALM Server URL: http://sm09520.dom1.e-ssi.net:8080/qcbin. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
    Empty Results
    Result Status: UNSTABLE
    Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
    Report not found
    Build step ‘Publish HP tests result’ changed build result to FAILURE
    Finished: FAILURE

    Like

  7. Hi,
    Please find my comments as below…

    Which version of HP Tools Plugin are you using, I don’t see the options you are selecting. —- I am confused here, where can i select this option & Which HP tools plugin ?

    Jenkins_POC is a testset name ? ————– YES

    Also Let me know, what options you have for Run Type drop down..–
    Run Locally

    Thanks,
    Chandrakant Bhirud

    Like

  8. Which version of HP Tools Plugin are you using, I don’t see the options you are selecting. …..
    Please let me know the plug in name and where can i select it ?

    Like

  9. Hi Sumit,
    I have downloaded and installed this plugin :

    hp-application-automation-tools-plugin.hpi

    Like

  10. Hi Sumit,

    Now you know everything about the configuration and set up on my machine, could you please help me in understanding why i am getting the

    error : Build step ‘Execute HP tests using HP ALM Lab Management’ changed build result to UNSTABLE
    Report not found
    Build step ‘Publish HP tests result’ changed build result to FAILURE
    Finished: FAILURE

    Like

  11. I am getting the following error:

    Console Output
    Started by user admin
    Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test_ALM
    [Test_ALM] $ “C:\Program Files (x86)\Jenkins\workspace\Test_ALM\HpToolsLauncher.exe” -paramfile props24062016183419987.txt
    “Started…”
    Timeout is set to: -1
    Run mode is set to: RUN_LOCAL
    Server has been disconnected while performing GetServerSettings action
    Error: “ALM server http://sm09520.dom1.e-ssi.net:8080/qcbin unreachable, check that server Url is correct”
    Build step ‘Execute HP functional tests from HP ALM’ changed build result to FAILURE
    Finished: FAILURE

    Like

  12. Hi Sumit,

    I have sent you the jenkin configurations for ALM integration through email, could you please check it and let me know, what i am missing.
    Thanks for your help !!!

    Regards,
    Chandrakant

    Like

    • Hi Chandu,

      Below is the analysis results,

      1) I downloaded the code for the plugin and its failing to create the OTA connection object. Its unable to initialize the ALM application.

      Below is the piece of code which is failing

      ///

      ///

      ///

      ///

      public bool ConnectToProject(string QCServerURL, string QCLogin, string QCPass, string QCDomain, string QCProject)

      {

      if (string.IsNullOrWhiteSpace(QCServerURL)

      || string.IsNullOrWhiteSpace(QCLogin)

      || string.IsNullOrWhiteSpace(QCDomain)

      || string.IsNullOrWhiteSpace(QCProject))

      {

      ConsoleWriter.WriteLine(Resources.AlmRunnerConnParamEmpty);

      return false;

      }

      try

      {

      TdConnection.InitConnectionEx(QCServerURL);

      }
      catch (Exception ex)

      {

      ConsoleWriter.WriteLine(ex.Message);

      }

      if (!TdConnection.Connected)

      {

      ConsoleWriter.WriteErrLine(string.Format(Resources.AlmRunnerServerUnreachable, QCServerURL));

      return false;

      }

      try

      {

      TdConnection.Login(QCLogin, QCPass);

      }
      catch (Exception ex)

      {

      ConsoleWriter.WriteLine(ex.Message);

      }

      if (!TdConnection.LoggedIn)

      {

      ConsoleWriter.WriteErrLine(Resources.AlmRunnerErrorAuthorization);

      return false;

      }

      try

      {

      TdConnection.Connect(QCDomain, QCProject);

      }

      catch (Exception ex)

      {

      }

      if (!TdConnection.ProjectConnected)

      {

      ConsoleWriter.WriteErrLine(Resources.AlmRunnerErrorConnectToProj);

      return false;

      }

      return true;

      }

      In the above code, following line of code is failing

      TdConnection.InitConnectionEx(QCServerURL);

      What you can try to do is, search for VBA to connect to ALM using OTA api, and try to add the reference to OTA client in excel macros and run that VBA. If you are getting the error at the same line then your installation of ALM is corrupted, for sure.

      If you can execute the VBA, then we can raise this as a bug for the plugin.

      In the meantime, I will try to execute one script from ALM to check, if its able to detect my ALM with the latest version of plugin, and I will update you the results.

      Let me know, if cant find the VBA OTA code to connect to ALM, I will write it and send it to you.

      Thanks,
      Sumeet Singh Kushwah

      Like

  13. Hi Sumit,
    I have executed the following code using UFT & Its working .

    Set TDConnection = CreateObject(“TDApiOle80.TDConnection”)

    TDConnection.InitConnection “ALM URL” TDConnection.Login “UID”,”PWD”
    TDConnection.Connect “ENERGY_WHOLESALE”,”QC_EGC_Neon” ‘

    If TDConnection.Connected Then
    MsgBox(“Connected to ” + chr (13) + “Server ” + TDConnection.ServerName + chr (13) +”Project ” + TDConnection.ProjectName )
    Else
    MsgBox(“Not Connected”)
    End If

    Thanks,
    Chandu

    Like

  14. I am using jenkins-1.624
    before i installed the jenkins-2.10 but i was not getting any plugin installed, so i installed old version…

    Like

  15. Hi Sumit,
    Did you get any solution for this, do i need to install any other version of Jeniks? or what/…
    please suggest…
    Thanks,
    Chandra

    Like

    • Hi,

      Put this code in the excel macros and add the reference to OTA COM type library. Update your ALM Server name and provide your user name and password.

      Sub c()

      Dim OConn As New TDAPIOLELib.TDConnection

      OConn.InitConnectionEx “[ALM SERVER URL GOES HERE]/qcbin”

      If OConn Is Nothing Then
      MsgBox “Cant initialize connection object”
      Else
      OConn.Login “[USER NAME GOES HERE]”, “[PASSWORD GOES HERE]”

      If OConn.LoggedIn Then
      MsgBox “Logged in”
      OConn.Logout
      End If

      Set OConn = Nothing

      End If

      End Sub

      Because the code you sent uses “InitConnection” which is only available for HP use and the code of the plugin uses “Initconnectionex” also plugin is trying to access TD Connection outside of HP environment. So lets try to do that once, I think you should get the error. I was busy recently so I have not tried executing tests from ALM using the plugin, but I will try to do that today and let you know the results.

      Thanks,
      sumeet singh kushwah

      Like

      • Hi Sumit

        Today I am not feeling well but tomorrow definitely I will try this.

        Thank you very much for your kind suport and all help even though you are busy in your work.

        Really appreciate you kind support !!!

        Regards,
        Chandrakant

        Like

  16. Hi Sumit,

    Thank you very much for your kind support, I will execute and let you know the results.

    Even though you are busy in your work but still you are very much helpful.

    Really appreciate your kind support !!!

    Regards,
    Chandra

    Like

  17. hi Sumit,

    Today i executed the code in excem macros and its working fine, its goving the message box “logged In”

    I am sharing the screen shot with you in email.

    Thanks,
    Chandra

    Like

  18. Hi Sumit,

    Could you please help me in executing the ALM script from Jenkins.

    Thnaks,
    Chandra

    Like

  19. Hi, I would like to run the same UFT tests in different environments. The UFT jobs are triggered from Jenkins pipeline and I would like to pass the environment from the Jenkins job to UFT. Is there a way I can achieve this ? Thanks

    Like

  20. Hi Sumeet,
    We use Jenkins ALM plugin to invoke our UFT scripts(both UI and API) in our CI/CD pipeline. We have a reusuable script concept where we call the same script in multiple places during execution and it is parametrized. We did this to avoid having multiple scripts(for easy maintenance). However, when the execution happens from Jenkins ( we are running tests locally on jenkins slave, we have installed UFT on slaves, because our jenkins is on Production box and ALM is in Test env and firewall restrictions doesn’t allow us to run tests on remote host or planned host), it takes lot of time to load the scripts and complete the execution. Is there any way to reduce the execution time. Just to give you an idea, the script when executed locally from ALM on our desktops takes 3 mins to complete execution where as from Jenkins it takes about 45 mins. Thanks for your help

    Like

    • So when you execute the same script

      1) open Alm and click on run button there to run the script on desktop A, it takes 3 mins

      2) when you provide the root folder path in Jenkins where Jenkins slave is installed on desktop A, and build the job it takes 45 mins..

      Is it correct ??

      Thanks,
      Sumeet singh kushwah

      Like

  21. Hi please help me out…i am getting the below error:
    “Started…”
    Timeout is set to: 60
    Run mode is set to: RUN_LOCAL
    Server has been disconnected while performing GetServerSettings action
    Error: “ALM server https://—————/qcbin unreachable, check that server Url is correct”
    Build step ‘Execute HP functional tests from HP ALM’ changed build result to FAILURE
    Failed login to ALM Server URL: https://————–/qcbin/. Exception: com.hp.application.automation.tools.common.SSEException: java.net.ConnectException: Connection timed out: connect
    Empty Results
    Result Status: UNSTABLE

    Like

  22. HI Kushwa,
    We are trying to run UFT scripts form ALM using jenkins ,when the build runs the QTPRemoteagent.exe and uft.exe both proceses are sowing on VM but uft tool is not opening and not executing the test,when we ran similar test cases from ALM on same remote machine it is working file

    Like

  23. Hi Kushwah,
    I am getting RPC server unavailbale and we did check the port 135 is opened and it is opened and also I unchecked the run hidden mode option in in Remoteagent settings

    Like

    • Ok. I don’t think that remote agent settings will impact the Jenkins executions.

      1) how Jenkins is installed(windows service, Command prompt, Tomcat etc) and how slave is installed ?
      2) can you try to execute the test on slave using a VBS file and let me know the results.
      3) can you send me the console output and Jenkins job configurations.

      I have not faced this issue, but I can try to replicate it.

      Like

  24. Hi Kushwah,
    Jenkins installed as windows service
    I tried the test running using VBS files don’t see any issue
    and also when running from ALM same testset executing fine

    Run mode is set to: RUN_REMOTE
    ============================================================================
    Starting test set execution
    Test set name: Jenkins, Test set id: 2502
    “Number of tests in set: “2
    Test 1: [1]UFT_Jenkins will run on host: “XXXXX”
    Test 2: [1]UFT_Jenkins_Copy_1 will run on host: “XXXX”
    “Scheduler started at:20/09/2016 14:10:58
    ——————————————————————————————————-
    20/09/2016 14:10:58 Running: [1]UFT_Jenkins
    20/09/2016 14:10:58 Running test: [1]UFT_Jenkins, Test id: 1201, Test instance id: 3014
    Test: [1]UFT_Jenkins, Id: 3014, Execution status: Running
    Test: [1]UFT_Jenkins, Id: 3014, Execution status: Error, Message: The RPC server is unavailable
    20/09/2016 14:11:52 Test complete: [1]UFT_Jenkins
    ——————————————————————————————————-
    20/09/2016 14:11:52 Running: [1]UFT_Jenkins_Copy_1
    20/09/2016 14:11:52 Running test: [1]UFT_Jenkins_Copy_1, Test id: 8428, Test instance id: 3015
    Test: [1]UFT_Jenkins_Copy_1, Id: 3015, Execution status: Running
    Test: [1]UFT_Jenkins_Copy_1, Id: 3015, Execution status: Error, Message: The RPC server is unavailable
    20/09/2016 14:12:37 Test complete: [1]UFT_Jenkins_Copy_1

    Liked by 1 person

    • Ok so below are the few things you can try

      1) install TD connect.exe on the slave machine.
      2) change the option to Run locally in the job configuration.
      3) set the option restric where this project can be run to the slave machine.
      4) check allow other HP products to run tests and components.
      5) make sure that you can login to ALM on the slave machine.

      Like

  25. Hi Kushwah,
    I am executing my script on remote machine reason why selected run on remote and I am giving the remote machine name I will try other options you mentioned

    Liked by 1 person

      • Hi Sumeet,

        Means your writing failed test case in Excel sheet or some file and than Re-triggering the those test case manually??

        Thanks,
        Suraj Naik

        Like

      • Hi Suraj,

        My main driver script reads the input excel sheet and execute the tests. If any test is failed it will be marked for execution. Now after the execution ends, my test will call the main driver script again, because failed tests are only marked for execution, main driver script will only run the failed tests. Main driver is called automatically twice by the script.

        Thanks
        Sumeet

        Like

  26. Hi Sumeet,
    I have added “proceed to next action iteration if any error comes and iam disabling the reporter filter(rfdiableall) in beginning but still Jenkin catches the error details and even test passes after recovery steps still it catching the failures and showing the fail in Jenkins job
    Any work around for this?

    Like

    • Hey Ramu

      Yes I had the same issue

      1) add the post build step to force Jenkins to pass/success
      2) create a junit file(which is explained in article). List the passed and failed tests in this junit file and upload that to Jenkins after the post build step.
      3) uploading junit with test case status will post the write status on Jenkins.
      4) remove the post build step to upload UFT run results on Jenkins. This will make sure that Jenkins is not getting results twice (from your junit and UFT results). Jenkins will still get the final error info if test fails.

      The only way for Jenkins to pass the test is if no error is thrown and no reporter.reportevent micfail statements.

      If you want to see how to avoid UFT from throwing errors then read the article below

      https://sumeetkushwah.com/2014/08/12/implementing-try-catch-functionality-in-qtpuft/

      Thanks,
      sumeet Singh Kushwah

      Like

      • Hi Sumeeth,
        Thanks for the Answer and will try and let you know the out come.
        Could you please let me know how to run the test twice if fails immediately as you aware in QC we have option to run the test automatically ‘n’ number of times and jenkins will take the latest result which is good but when we are using MTBX ,i dont see any option running failed one immediatly and running whole bunch of test cases which included passed.

        I have added naginator plugin but it is only running after execution completed and running all test cases again

        Best Regards,
        Ramu Gogurla

        Like

  27. Hi Sumeeth, we have found solution for this, when scripts run in jenkins it will create 2 files param….txr and result….xml, from that we can get pass or fail

    Like

  28. Hi Sumeet,

    Very useful info in your forum. thanks.
    I have tried the steps which you have mentioned above. but i encountered with the below error on execution.
    “>>>> File/Folder not found: ‘X:\ATAF’
    No valid tests were found. Fix the test paths.
    Build step ‘Execute HP tests from file system’ changed build result to FAILURE”

    Could you please provide more info on Test path. Am not sure what info needs to be provided there.
    Bcos all the test scenarios and data are kept in svn itself and that is mapped in config.xml.
    Please help me on this regard.

    Regards,
    Nandha

    Liked by 1 person

  29. Hello,
    I am usings Jenkins 2.19.2 in windows.
    I have also installed plugin – HP Application Automation tools 4.5.0
    but unable to build project using ALM12.01

    I go to Manage Jenkins>Configure system>Enter ALM details.
    Here when i enter the ALM url which i use for accessing via URL. I get following error – “Error openning a connection to the ALM server”
    (I am able to open in IE ALM12 with the same URL as entered in Jenkins.But still i get this error)

    Ignoring this error i configure my build with step “Execute functional tests from HP ALM” and enter all details
    In test sets i have given the complete path of the test set i want to run
    ( eg: Root\Scheduler\SchedulerDummy1)

    Finally when i build this i get following message:

    Console Output:
    Started by user admin
    Building in workspace C:\Program Files (x86)\Jenkins\workspace\FirstProj
    [FirstProj] $ “C:\Program Files (x86)\Jenkins\workspace\FirstProj\HpToolsLauncher.exe” -paramfile props03112016213435061.txt
    “Started…”
    Timeout is set to: -1
    Run mode is set to: RUN_LOCAL
    —————————————————————
    And i get a round loading symbol which goes on and on….(i waited for 30 min …but nothing happened.Nor am i getting any error message.

    Could anyone please help me resolve this?

    If any other details required please let me know.

    Liked by 1 person

  30. Regarding my earlier post How to run UFT failed test cases?

    Please give some example how i achieve this, i running daily Test set for example i have 20 tests are there in test set, 15 will pass out of 20 so i need run remaining 5 tests manually, how i can i do through Jenkins

    Liked by 1 person

    • Hi Suraj,

      You can’t do this with Jenkins(No easy way), you will have to implement this in UFT.

      But here is what I have implemented

      I have an “executeTests” function which will loop through the input excel sheet with the tests.

      This “executeTests” function has two for loops

      1) The inner loop which will execute each test listed in the Input excel sheet.
      2) The outer loop which runs twice so that the inner loop can be executed twice hence all of the tests will be executed twice.

      The inner loop also keeps track of the test cases names which we have executed, so if we have already executed a test, then inner loop will only re-execute it if the status is failed for the test.

      If you can send me the details about your framework structure, I can help you in implementing this concept for re-executing the failed tests automatically.

      Thanks,
      Sumeet Singh kushwah

      Like

  31. Hi Sumit,

    Following is my driver script
    RunAction “OpenJavaApp”, oneIteration
    RunAction “TestSomeApp”, oneIteration
    RunAction “CloseTheApp”, oneIteration

    Here 2nd RunAction may fail due to some Network connectivity or object is not visible if i ran same it will pass

    Please can you help me how i can implement same

    Like

    • Hi Suraj,

      Below is the updated code

      Dim ArrayToStoreTestResults

      ”assign false for executing the test for the first time
      ArrayToStoreTestResults = false

      ”’ This will make sure that the test is executed twice
      For ExecutionCounter=0 to 1 Step 1
      ”’Store the test case execution results in the array ArrayToStoreTestResults
      If ArrayToStoreTestResults = false then
      RunAction “OpenJavaApp”, oneIteration
      ”’ Change ArrayToStoreTestResults to true if the test is passed, else change it to false
      RunAction “TestSomeApp”, oneIteration
      RunAction “CloseTheApp”, oneIteration
      End IF
      Next

      Below is the image of the reports generated on jenkins for my executions

      Let me know, if you need anything else

      Thanks,
      Sumeet

      Like

  32. how to run multiple QTP scripts from QC on different machines
    Scenario: how to run multiple QTP scripts from QC on different machines, say for example 100 script in QC test lab, to run on 5 machines, each machine having 20 scripts, while running all at a time from QC, if one of the machine’s h/w got crashed, so those scripts should automatically distribute or run on any available machine.

    any idea plz?

    Liked by 1 person

    • Hi,

      Currently its implemented in our environment and I don’t think that it can be achieved by using ALM alone.

      What We have implemented is below

      1) All of our scripts are running from Jenkins.
      2) We have created (nodes)machines in jenkins and assigned labels to the machines. So If we have 5 machines then the machine names will be machine 1, machine 2……, Machine 5. But we have assigned labels for controlling which machines will pick up the tests, so if for app1 we have 100 tests and we want them to run on first three machines, then in the Jenkins nodes(Machine) configuration we will add the label “APP1” for all three machines and in jenkins job configuration, we will add the machine name to execute the test as “APP1” instead of Machine 1…N. This will create a pool of three machines with label APP1. Now all three machines will run the tests as soon as we start the executions.
      3) Jenkins will pick the machines on the go, So if one machine goes down then, it wont use that machine and executions will continue on the remaining two machine.

      I have seen something called application pool in the ALM Lab Manager, but I don’t know, if that can be used in the similar way.

      Please let me know, If you have ALM Lab manager available in your ALM, then I can try to provide you some details about it or if you need anything else.

      Thanks,
      Sumeet Singh Kushwah

      Like

  33. I had a defect open with HP and we could not solve it. It would loose video context and miss Ui control actions from time to time. It was not reliabile. HP stated we should keep a VM session active and open
    before the automation was remotely kicked off.

    Dave

    Liked by 1 person

    • Hi,

      Below is the screenshot of UFT test execution from Jenkins

      UFT Jenkins test execution results

      I am not doing it on VM’s I have physical machines. So, I don’t know if VM’s are causing the issue. But I have scripts to make sure that windows is logged in and not locked during the UFT executions.

      All of my tests are running on different machines.

      Thanks,
      Sumeet

      Like

  34. Hi Sumeet,

    I have a new challenge. I am trying to execute UFT tests from Jenkins using GIT source code.

    Now my challenge, how would be version control the action UFT script and xls files. We have already version controlled out function library and the object repo but not quite sure how to tackle the xls and uft test files we have associated with our frame work.

    Have you come across something similar which I could look into?

    Many thanks

    Ajay

    Liked by 1 person

  35. Hello Summet,

    Currently i am running tests in VM but sometimes Jenkins through following error and Jobs will stop

    Logs:

    Unhandled Exception: System.NullReferenceException: Your Quality Center session has been disconnected. Contact your system administrator for more information
    at Mercury.TD.Client.Ota.QC9.ITSTest.get_LastRun()
    at HpToolsLauncher.AlmTestSetsRunner.GetTestRunId(ITSTest currentTest)
    at HpToolsLauncher.AlmTestSetsRunner.RunTestSet(String tsFolderName, String tsName, Double timeout, QcRunMode runMode, String runHost)
    at HpToolsLauncher.AlmTestSetsRunner.Run()
    at HpToolsLauncher.Launcher.RunTests(IAssetRunner runner, String resultsFile)
    at HpToolsLauncher.Launcher.Run()
    at HpToolsLauncher.Program.Main(String[] args)
    An error occured while disposing runner: Your Quality Center session has been disconnected. Contact your system administrator for more information.

    Thanks,
    Suraj NAIK

    Like

  36. Pingback: UFT GIT Jenkins Integration | Automation Insights·

  37. Hello Sumeet,

    I was trying to see if there is anyway to run a single test case from a testset from HP ALM using Jenkins? If yes. please specify..

    The goal is to run some custom vb script to upload screen shots to ALM after each test case ran and proceed to run other testcase …

    Like

  38. Hi Summet,
    We are trying to run the Load runner scripts integrating with Jenkins. We are executing the scenarios with the batch file by selecting the “Execute windows batch command” under the build section.

    “C:\Program Files (x86)\HP\LoadRunner\bin\Wlrun.exe” -Run -TestPath “” -ResultName “”

    We are able to view the results in the Results folder .However, we are getting the error at the Console Output “Build step ‘Execute Windows batch command’ marked build as failure”.

    Please find below console output window message
    ———————————————————————-
    Started by user
    Building on master in workspace C:\Program Files (x86)\Jenkins\workspace\MyLoadRunner
    [MyLoadRunner] $ cmd /c call C:\Windows\TEMP\hudson695684065292801928.bat

    C:\Program Files (x86)\Jenkins\workspace\MyLoadRunner>C:\Batchfile\LR.bat

    C:\Program Files (x86)\Jenkins\workspace\MyLoadRunner>”C:\Program Files (x86)\HP\LoadRunner\bin\Wlrun.exe” -Run -TestPath “C:\LRScenarios\Scenario1.lrs” -ResultName “C:\LRResults\Run6”
    Build step ‘Execute Windows batch command’ marked build as failure
    Finished: FAILURE

    Like

  39. Hi Sumeet,

    I am also facing an issue with ‘create scheduler’ which is mentioned earlier in the comments section
    Error:
    Could not create scheduler, please verify ALM client installation on run machine by downloading and in installing the add-in form:

    But if I call manually command line seems to work (calling directly: HpToolsLauncher.exe -paramfile propsXXXXXXXXXXX.txt”), but when calling the task from Jenkins, it fails.

    Could you please help me in this.

    Like

  40. Hi Summet,
    I am also facing an issues which is mentioned in comments section earlier
    Issue:
    Could not create scheduler, please verify ALM client installation on run machine by downloading and in installing the add-in form: http://xxxxxxxx/qcbin/TDConnectivity_index.html

    i have tried the solution which you mentioned but its still not working

    but if we call manually the command line seems to work (calling directly: HpToolsLauncher.exe -paramfile propsXXXXXXXXXXX.txt”), but when calling the task from Jenkins, it fails.

    Could you please help?

    Like

  41. Hi Sumeet,
    I am running a Jenkins Free Style Project for UFT Windows based Application. I am using Hybrid Framework for Automation.
    When I normally run Script from my local, it stores the Results in Run Results_Folder inside my Project Folder Structure. Also we can see it from Run Results Viewer.
    My Concern is that I want to store theses reportswhich are handled in my Framework in Jenkins when Build is Run as a post Build Action.
    Not Sure about the Plugin required. Can you please guide here

    Like

  42. Hi,
    Thanks for the wonderful information. I have 1000 scripts from file system listed in the Build and 10 machines labelled. can you guide me on how to configure Jenkins to setup parallel execution. That is have Jenkins trigger each test case in the list on one of the available slaves
    in parallel. currently the scripts execute in sequence though i have multiple slaves available.

    Thanks
    Vivek

    Like

    • Hi Vivek,

      You need to assign the labels to the slave machine config. You can assign multiple labels to one machine. Labels can be separated using a space, so I can have labels like “facebook_machines linkedin_machines”.(don’t use double quotes). Assign same label names to all machine where u want to execute parallel tests.

      Once you have the labels assigned, then u can use these label names in the jobs under the option to “restrict where this job can be executed”. Use one label name there like facebook_machines. Change this config for all the jobs you want to run in parallel.

      After this if you trigger multiple jobs, Jenkins will automatically assign the machine out of one of the slaves where u have configured label names.

      Let me know, if u need any other details.

      Like

      • Hello Sumeet,

        I was able to assign same labels to 2 different machines and on job level i restricted it with those labels and now the job has 2 nodes to service the job.
        However, I would like to know if one jenkins job has let’s say 10 test cases under build section. Will jenkins be able to distribute those 10 test cases to all the available slaves (2 in my case) lets say 5 on one slave and another 5 different and run them in parallel?

        Regards,
        Alankar

        Like

    • Hi Vivek,
      Is your question regarding running multiple jenkins jobs in parallel or running multiple tests(within build section) in ONE jenkins job in parallel?

      Like

  43. In my assignment I have no sorucecode repo like SVN and Maven, we have complete framework which run from driverscript, In that case how do I run my tests.?

    Like

Leave a Reply to Priya Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s