Image Removed

...

Distributing an Entire Custom Step or Build Event

This method can be used in Visual Studio 2005 and higher, and allows the distribution of the whole build steps to remote machines:

...

The IncrediBuild_AllowRemote Directive

In order to indicate that a custom build step or build event is distributable, enter the following line anywhere within the step's command line sequence:

rem IncrediBuild_AllowRemote

This indicates that IncrediBuild may distribute the step's execution to a remote machine.

Examples of custom steps that you may want to execute in a remote machine are:

• Code generation
• Rendering
• Simulations
• Calculations that are part of the code building

The option of running tasks in remote often goes hand-in-hand with the IncrediBuild_AllowOverlap directive that enables the parallelization of tasks.

If the distributed task creates output files, it is important that these files be specified explicitly using either the custom build step's "Output Files" field (custom build steps only), or one or more of the Output File Definition Directives.

Important! For Visual Studio 2008 and earlier, distributed steps that are missing output file specifications will not create any output files! For Visual Studio 2010 and later, it is recommended to specify the output files. However, they will synchronize even if they are not specified.

The IncrediBuild_AllowOverlap Directive

The IncrediBuild_AllowOverlap directive instructs IncrediBuild to execute custom steps, events and tools in parallel to other tasks that would otherwise be waiting for the end of the execution of the custom steps events or tools, before they would execute.

Consider the following scenario:

Project A is a static library project with a post-link step that deletes some files and writes text to a log file. Project A is defined as a dependency (sub-project) of project B, because project B links with its output. Visual Studio (and thus IncrediBuild's) default behavior is to wait for the completion of all project-level steps in a project's sub-projects before starting that project's compilation. As a result, project B does not start to compile until project A's post-link step has completed, slowing down the build.

In some cases, when a step generates files that are necessary for the next project to compile, this behavior is required. The IncrediBuild_AllowOverlap directive should be used when this is not the case, and output files generated by a project-level step are not referenced by source files in any dependent project.

To declare that the output files of a project-level custom build step or build event (post-build, pre-link, etc.) are not referenced by source files in any dependant project, enter the following line as one of the step's commands:

rem IncrediBuild_AllowOverlap

This will not affect the step's operation, but will advise IncrediBuild to start compiling dependent projects as soon as possible, regardless of that step.

In many cases this directive could also be used to speed up the execution of a single project, since Visual Studio runs tasks within a project in a sequential manner when it comes to custom build steps, build events, build tools and build rules, and in many cases this is redundant.

Note: Custom Build Tools and custom build rules can be distributed and parallelized using IncrediBuild_AllowRemote and IncrediBuild_AllowOverlap in Visual Studio 2005 and 2008, however if you would like to distribute and parallelize Custom Build tools in Visual Studio versions 2010 and higher, please see method two below.

Breaking Custom Steps and Build Event to Multiple Tasks

Sometimes a custom build step, build event or a build tool can be further broken down to smaller tasks, and parallelized to achieve further acceleration.

This method can be used in Visual Studio versions 2010 and higher, and allows the distribution and parallelization of the contents of the build steps, on the process level.

This feature is only available in scenarios where:
1. The IncrediBuild predicted feature (on by default) is not turned off.
2. The IncrediBuild UseMSBuild directive is not used.
3. The build is not executed with "build regardless of errors".

Example 1: If within a custom step a user executes a complete Make command that builds a complete project, you can instruct IncrediBuild to highly accelerate your Make command execution by using the IncrediBuild Make and Build Tools package. Instead of treating the Make command as a single task, IncrediBuild will instead “break” this command to its sub-tasks and will execute the compilation in a parallel and distributed manner.

Example 2: In the case that you have a custom step, which has a tool that runs many rendering commands, you can use Profile XML not only to distribute the rendering but also to intercept the tool that spawns the rendering processes. In this case, you can expand IncrediBuild’s Profile XML, as described below, to control and also accelerate and distribute custom steps, rules, and events.

You can integrate Profile XML file into Visual Studio 2010 and later, provided that you have the appropriate package (Dev Tools (see IncrediBuild for Dev Tools) or Make and Other Build Tools (see IncrediBuild for Make and Other Build Tools)). Integrating Profile XML enables you to have complete control on the way IncrediBuild treats the custom tools.

By default, custom steps, exec tasks, and special build tools are not distributed to remote agents.
In order to allow IncrediBuild to distribute and manage these tasks, an XML file should be used based on IncrediBuild's Automatic Interception interface.
There are two ways in which a profile.xml file can be used in order to manage such tasks:

1. In the IncrediBuild BuildConsole command, use the /profile directive to specify a profile for IncrediBuild to use: /profile=[profile_file_name].xml (without the square brackets characters).
2. If you are building from within the Visual Studio IDE or would like your solution to have a default profile to use (per solution), create [solution_folder]\[solution_name].ib_profile.xml profile xml file. This profile will be automatically loaded by IncrediBuild in case that no profile file was specified with the /profile=[profile_file_name].xml switch.

...

Note: If your Custom Build Steps and Build Events appear as a single task (single bar in the build monitor labeled “IBCustomStep” ) after adding the profile.xml file, please change the registry value CustomStepVs10Support from the default “1” to “0” under the HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Xoreax\IncrediBuild\Builder registry key.

Parallelization of Custom Build tools and custom build rules using the .ib_profile.xml

Visual Studio 2010 and newer versions use the MSBuild engine which executes all custom build tools and custom build rules within the context of a single project in a sequential  manner – on a single CPU core

For example, in the following scenario: A custom build tool running moc.exe (as in the GUI SDK QT) will be executed in a sequential manner in project A for each file that needs to be generated.

Specifying in your IncrediBuild profile a tool name (the process name – moc.exe) with the AllowPredictedBatch option will instruct IncrediBuild to execute these moc tools in parallel resulting in a significant performance boost.

Since the tool will run in a parallel manner, it can also be configured to be distributed to remote machines (please see XML sample below).

Note that the tools that can be safely accelerated with this feature must have the following characteristics:

1)      Two Instances of the executable will not attempt to write to the same file (e.g. output file, log file)

2)      None of the instances of the executable should be dependent on anything that the other instances do. (e.g one instance does not depend on the output file of another instance, none of the instances deletes any files or folders the other instances use).

Parallelization will be applied on the defined tools up to the point where a different executable will be invoked (CL, LINK or anything else).

To use the feature add AllowPredictedBatch="true" tool tag to your solution’s .ib_profile.xml

Usage Example:

If your solution name file is my_solution.sln, and you would like to parallelize and distribute moc.exe execution, add the file my_solution.ib_profile.xml to the folder where my_solution.sln resides – this will apply the profile to any build of this solution ran from within the IDE. Alternatively, if building from the command line, you can use the “/profile” switch to specify the profile file:

Buildconsole my_solution.sln /rebuild /cfg=”debug|win32” /profile=”my_solution.ib_profile.xml”

Example contents of my_solution.ib_profile.xml:

<?xml version="1.0" encoding="UTF-8" standalone="no" ?>

<Profile FormatVersion="1">

  <Tools>

    <Tool Filename="moc" AllowRemote="true" AllowPredictedBatch="true"/>

  </Tools>

</Profile>

...

...

Visual Studio Reference:

IncrediBuild Solutions:

User Manual:

...

Image Added