Overview
Jenkins is a continuous integration tool primarily used in software development flow to build and test projects continuously and monitoring executions of externally-run jobs. Its rich plugin ecosystem supports a wide variety of source control, build, test, analysis and deployment systems.
IncrediBuild is a build acceleration technology that allows builds, tests, and other development processes to execute in parallel over a distributed network.
IncrediBuild for Jenkins allows customer to scale their Jenkins nodes to seamlessly distribute compute processes across machines in the local network or public cloud. By using the aggregated idle CPU cycles across your network, IncrediBuild effectively transforms any Jenkins node to become a super computer node that can scale to use hundreds of cores you already own instead of only its local cores, thus highly accelerating time-consuming jobs such as compilations, unit tests, QA scripts, code analysis, packaging, assets creation and more.
In order to unleash the full benefit of IncrediBuild, it is recommended to install IncrediBuild on every Jenkins node and any other machine that can contribute its idle CPU cycles to the IncrediBuild distributed platform.
The remote machines that contribute their idle CPU cycles to the IncrediBuild infrastructure only require to have an IncrediBuild Agent installed on them. There is no need to copy your source code, tool chain or any other file to these remote machines.
Once a process will be distributed by IncrediBuild to be executed on a remote machine, all the data that will be required for the process to run successfully will be copied on demand to a special sandbox by IncrediBuild’s process virtualization mechanism.
For further information on the IncrediBuild distributed platform please refer to the IncrediBuild user manual.
Setting up Jenkins to Run with IncrediBuild
Prerequisites
The IncrediBuild plugin for Jenkins has been tested to work on Jenkins 1.580.1 and later versions. However, it may work on earlier versions as well. The plugin has no dependencies on other plugins.
An IncrediBuild infrastructure is required to be set in your environment in order to be able to accelerate your Jenkins builds.
| Info |
|---|
In order to use the IncrediBuild Jenkins plugin, an Enterprise Edition of IncrediBuild is required. |
Installation
To install the IncrediBuild plugin for Jenkins:
- Download the IncrediBuild Jenkins plugin by clicking here.
- Copy the hpi file to: /JENKINS_HOME/plugins
- Restart the Jenkins service
For further information on installing plugins in Jenkins please refer to the Jenkins manual, “How to install plugins” section.
Plugin Configuration
The IncrediBuild plugin requires no configurations.
Jenkins Slave Nodes Configuration
On each Jenkins node where an IncrediBuild build step may execute, you need to make sure that an IncrediBuild Agent is properly installed and configured. Additionally, make sure that the PATH variable contains the IncrediBuild binaries folder when accessed by Jenkins.
IncrediBuild Build Steps
Using IncrediBuild Build Steps
The IncrediBuild for Jenkins plugin provides wrappers around the IncrediBuild’s BuildConsole and IBConsole commands as an alternative to running the commands in a shell or batch scripts.
IncrediBuild Jenkins plugin build steps
Adding IncrediBuild Build Steps to a Jenkins Job
The IncrediBuild Jenkins plugin adds four new build steps, which are shown in the drop-down menu.
To add an IncrediBuild build step to a Jenkins job:
- Navigate to the Jenkins job you want to configure to execute an IncrediBuild command.
- Click Configure in the upper-left area.
- In the build steps section of the page click Add build step in the dropdown menu and select one of the available IncrediBuild build steps.
- Once a build step is added, you will be able to configure the step’s parameters just below the step's title (see screenshots in the following sections). For each parameter, clicking the help icon will open a tooltip with relevant information for the selected parameter.
- See the following sections on how to configure each of the IncrediBuild build steps.
- Click Save on the configuration page.
BuildConsole (Visual Studio)
This build step provides a wrapper around the IncrediBuild BuildConsole command, for distributing and accelerating compilations of Visual Studio solutions or projects.
For more information on using IncrediBuild to accelerate Visual Studio compilations, see IncrediBuild for Visual Studio in the IncrediBuild online manual.
Required Configurations
Step configuration requires setting the target solution to build together with the solution configuration or preset file. The plugin will validate that there are no contradictory options selected.
Optional Configurations
Other BuildConsole optional parameters are grouped as; Target and Configuration, Distribution Control, Output and Logging, Miscellaneous and Advanced.
IncrediBuild Jenkins plugin - Visual Studio build step
IncrediBuild for Make and Other Build Tools
This build step provides a wrapper around the IncrediBuild "BuildConsole \Command" command line, geared for accelerating the build process of projects based on Make, MSBuild, WAF, ninja, scons, CMake or other build systems that are supported by default by IncrediBuild.
For a full list of pre-defined supported build tools and more information on using IncrediBuild to accelerate Make and other build tools, see IncrediBuild for Make and Other Build Tools in the IncrediBuild online manual.
Required Configurations
Step configuration requires setting the command to execute (e.g. make all).
Note that by default, IncrediBuild automatically adds a parallel processes switch (in Make, this switch is named -j; in MSBuild it is /m:). If a parallel processes switch is already part of the command line, IncrediBuild will change its value to a larger value in order to execute (and distribute) much more processes in parallel.
In MSBuild for example, IncrediBuild will change the value of an existing /m switch to a /m64, in the Make build system the -J flag will be changed to -j200. If the executed command is given in a Batch file (i.e. /command = Batchfile), and the MSBuild command that runs the build is inside the Batch file, IncrediBuild won’t be able to determine how to change the /m value, and therefore won’t change it. In this case, it is up to the user to replace or add a /m command with a high value to it (the recommendation is more than 40 for MSBuild and more than 100 to Make build system variants).
Optional Configurations
Optional BuildConsole parameters are grouped as; Distribution Control, Output and Logging and Miscellaneous.
IncrediBuild Jenkins plugin - Make & other build tools build step
IncrediBuild for Dev Tools
This build step provides a wrapper around the IncrediBuild "IBConsole" command line, geared for accelerating various multi-process custom executions, such as unit tests, QA scripts, packaging, code analysis, asset builds, simulations and others.
For more information on using IncrediBuild to accelerate various development executions such as: tests, code analysis, asset creation, or other custom executions, see IncrediBuild for DevTools in the IncrediBuild online manual.
Required Configurations
Step configuration requires setting the command to execute (“MyUnitTests.bat”, for example).
Optional Configurations
Other BuildConsole optional parameters are conveniently grouped to Distribution control, Output and Logging and Miscellaneous.
IncrediBuild Jenkins plugin - DevTools build step
IB Console (Linux)
The IncrediBuild for Jenkins plugin is cross platform and provides users with the ability to accelerate Jenkins builds executed on top of Linux operating systems. In order to accelerate builds initiated by Jenkins Linux build agents, a Linux version of IncrediBuild has to be properly installed and configured.
For accelerating Jeknins Linux builds, please contact us at support@IncrediBuild.com for further instructions.
Required Configurations
Step configuration requires setting the Target Solution File.
Optional Configurations
You can see the other ib_console optional parameters in the following screenshot:
IncrediBuild Jenkins plugin - Linux build step
Build Steps Configuration Fields Table
This section provides detailed information on each of the configuration fields available in the vaious IncrediBuild for Jenkins build steps.
...
Field name
...
Description
...
Target Solution File
...
The solution file to be built.
...
Build Type
...
The type of build to perform.
...
Configuration
...
Determines the solution configuration to be built (e.g. Debug|Win32). Several comma-separated configuration names can be specified (e.g. Debug|Win32,Release|Win32).
...
Custom Configuration Preset
...
Determines the projects/configurations that will be built using a Preset previously saved in the Batch Build dialog. This option is useful when a complex combination of projects and configurations is required. The /Preset command option replaces both the /cfg and the /prj options.
...
Project
...
Determines a specific project to be built. By default, the dependencies of the project will be built as well.
...
Do Not Build Dependencies for the Project
...
Builds only the specified project configurations, ignoring any dependencies these configurations may have. When used in conjunction with Clean, this cleans only the specified configurations. When used in conjunction with /Rebuild, this rebuilds only the specified configurations.
...
Build all Projects Regardless of Errors
...
Makes a solution build continue regardless of specific projects failing to build. Normally, IncrediBuild stops the build when a project fails to build its output file. This option instructs IncrediBuild to continue building the remaining project configurations, regardless of previous projects failing to build.
...
UseEnv
...
Use PATH, INCLUDE, LIBPATH, and LIB environment variables instead of MS Visual C++ IDE paths.
...
Add Parameters to cl.exe
...
Allows additional compiler options to be added to all calls to cl.exe. Enter one or more commands separated by a space character, as they would appear in the cl.exe command line. e.g. /O2 /FA.
...
Remove Parameters from cl.exe
...
Specifies compiler options that will be removed from all calls to cl.exe. Enter one or more commands separated by a space character, as they would appear in the cl.exe command line. e.g. /Z7 /Zd /Zi /ZI.
...
Force Visual Studio Version
...
Forces a specific version of Visual Studio toolset to be used. This is especially helpful when working with upgraded projects, since Visual Studio sometimes uses the incorrect version to build a solution after it was upgraded.
...
Use MSBuild Instead of Devenv
...
Sets Msbuild as the top build manager instead of devenv. Use this only if you are experiencing issues with the default Devenv builds, or if you need to pass arguments to msbuild manually using the MsbuildArgs option.
...
Arguments for Msbuild
...
Passes additional arguments to MSBuild when using the BuildConsole command's UseMsbuild switch.
...
Profile.xml
...
The Profile XML file is used to define how Automatic Interception Interface should handle the various processes involved in a distributed job. It is not necessary for Visual Studio or Make and Build tools builds, but can be used to provide configuration options if those builds use additional processes that are not included in those packages. You are required to configure distributable processes in Dev Tools builds.
...
Avoid Task Execution on Local Machine When Possible
...
Overrides the Agent Settings dialog Avoid task execution on local machine when possible option. This allows you to free more resources on the initiator machine and could be beneficial to distribution in scenarios where the initiating machine is bottlenecking the build with High CPU usage.
...
Maximum CPUs to Utilize in the Build
...
Specifies the maximum amount of remote cores to utilize in the build. Overrides global max. CPUs/Cores in build setting.
...
Newest Allowed Helper Machine OS
...
Specifies the newest operating system installed on a helper machine to be allowed to participate as a helper in the build.
...
Oldest Allowed Helper Machine OS
...
Specifies the oldest operating system installed on a helper machine to be allowed to participate as a helper in the build.
...
Build Title
...
Specifies a custom header line which will be displayed in the beginning of the build output text. This title will also be used for the Build History and Build Monitor displays.
...
Save IncrediBuild Monitor File
...
Writes a copy of the build progress (.ib_mon) file to the specified location. If only a folder name is given, IncrediBuild generates a GUID for the file name. A message containing the location of the saved .ib_mon file is added to the end of the build output.
...
Suppress STDOUT
...
Does not write anything to the standard output.
...
Output Log File
...
Writes build output to a file.
...
Show Commands in Output
...
Shows, for each file built, the command-line used by IncrediBuild to build the file.
...
Show Agents in Output
...
Shows the Agent used to build each file.
...
Show Time in Output
...
Shows the Start and Finish time for each file built.
...
Hide IncrediBuild Header in Output
...
Suppresses the IncrediBuild header in the build output.
...
Internal IncrediBuild Logging Level
...
Overrides the internal IncrediBuild logging level for this build. Does not affect output or any user accessible logging. Used mainly to troubleshoot issues with the help of IncrediBuild support.
...
Set an Environment Variable
...
Sets or overrides environment variables for the context of the build.
...
Stop On Errors
...
When specified, the execution will stop as soon as an error is encountered. This is the default behavior in Visual Studio builds, but not the default for Make and Build tools or Dev Tools builds.
...
Additional Arguments
...
Add additional BuildConsole arguments manually. The value of this field will be concatenated to the final BuildConsole command line.
...
Nice Value
...
