IncrediBuild Terminology Overview

IncrediBuild Terminology Overview

Jobs and Tasks

Jobs – A Job is the entire process tree being executed by IncrediBuild. The Job at the Initiator is broken up into Tasks.

• Tasks are executed at the OS-level within a process tree.
• Tasks can also invoke subtasks, both tasks and sub-tasks can be distributed to run on remote machines.
• Tasks represent the process to be executed either by an Initiator or Helper Agent. 

Example: In Visual Studio, the tasks that are being executed remotely by IncrediBuild are usually compilation tasks. In C++ compilations, the cl.exe which is a compilation process, will be distributed by IncrediBuild to be executed remotely. As there are many compilation tasks when performing a full rebuild, IncrediBuild can potentially distribute hundreds of compilation tasks to be executed concurrently on remote machines (Helpers), thus, highly accelerating the entire compilation execution.

IncrediBuild Coordinator and Backup Coordinator

The coordinator is the component which manages resource availability in an IncrediBuild pool. It communicates with the Agents to identify the resources that are available or unavailable and manages them accordingly.

The Coordinator performs the following functions:

• Subscribes and unsubscribes Agents from participating within the resource pool.
• Maintains up-to-date Agent information through Status Reports received from Agents.
• When tasks are waiting to be distributed, the Coordinator provides the Initiator with the necessary Helper agents.
• The Coordinator is responsible for allocating IncrediBuild solutions and core packages to Agents.

Backup Coordinator: Setting a Backup Coordinator is optional. A backup Coordinator kicks in when the main Coordinator fails for some reason (network disconnection, for example). You hopefully won't need it, but as with any professional working environment, having a safe guard can always be beneficial and will make sure your IncrediBuild infrastructure is always up and running.

To summarize the relationship between an Initiator Agent, Coordinator, and Helper Agent:

1)The Initiator calculates how many additional remote cores it requires.

2) The Initiator sends a request to the Coordinator for necessary number of Helper Agents.

3) The Coordinator scans the network, checks where resources are available, and assigns them for the Initiator to use.

For example, if the Initiator wants to execute 10 concurrent processes in a distributed manner and requests for 10 Helper cores, it may receive all or some of them from the Coordinator. After completing the execution of these 10 processes, if the Initiator needs to distribute 8 additional processes, as it already has 10 Helper cores assigned to it, it will not request additional remote cores from the Coordinator but will instead release two remote cores back to the resource pool.

As mentioned above, when a process is distributed to be executed on a Helper machine, IncrediBuild's virtualization takes care of everything for you behind the scene as though the remote code is a local core. You don't need to install anything besides an IncrediBuild Agent on this remote machine, nor copy your source files or executables. Also, any file that will be generated by the distributed process will automatically be copied by IncrediBuild to its designated path on the Initiator machine. This unique IncrediBuild technology is named "Process Virtualization".

If you would like to dive deeper into the intricacies of how this technology works, there is an article that explains Process Virtualization in-depth called What is Process Virtualization?