Page Comparison

Image Removed

...

The following chapter includes instructions on how to configure and run IncrediBuild for Unit Tests to run in Assembly-level mode or Test-level mode with the NUnit framework. The setting that specifies the Assembly or Test level modes for NUnit is specified within the IBTestConsole command, as described below.

Supported Frameworks:

IBTestConsole supports the following frameworks:

• NUnit 2
• NUnit 3
• Google Test (Gtest)

IBtestConsole will automatically detect the executed framework. However, if you want to force it to run a specific framework, you can use the /test=NUnit3 or /test=NUnit2 CLI command line options.

Running the IBTestConsole

To configure and run the IBTestConsole, you should first run the IBTestConosle.exe file, located in the IncrediBuild installation folder. This is a console application that initializes IncrediBuild for Unit Tests in various modes of operation, according to its various parameters.

IBTestConsole Commands Usage

The IBTestConsole commands follow the following syntax:

IBTestConsole [command] [options]

• Command - stands for the original command of your Unit tests framework. The command may include flags (described below).
• Options – one of the options described in the section below.

...

The following framework command flags are supported by IncrediBuild for Unit Tests. Each framework version includes different flags as described below. To learn more about each flag, please refer to the following resources:

Nunit 3 CLI reference: https://github.com/nunit/docs/wiki/Console-Command-Line

Nunit2 CLI reference: https://nunit.org/docs/2.6.2/consoleCommandLine.html

NUnit 2 Supported Flags

• /run

• /runlist

• /include

• /exclude

• /result, /xml

• /noresult, /noxml

• /framework

• /timeout

NUnit 3 Command Flags

• @FILE

• --testlist=FILE

• --where=EXPRESSION

• --config=NAME

• --framework=FRAMEWORK

• --timeout=MILLISECONDS

• --seed=SEED

• --skipnontestassemblies

• --result=SPEC

• --noresult

• --shadowcopy

• --noheader, --noh

• --params | p=PARAMETER

• --dispose-runners

GTest Command Flags

• --gtest_output

• --gtest_filter

...

The following options can be added at the end of the IBTestConsole command.

...

Option

...

Mandatory

...

Description

...

Notes

...

/OpenMonitor

...

Opens the Execution Monitor window, which shows the execution progress.

...

/TargetDir

...

Specifies the working directory on which the test framework operated.

...

This command should be used in case the IBTestConsole was not initiated from the same directory as the test assemblies.

...

/test={framework}

...

Forces IBTestConsole to handle the executed command as a specific framework.

...

By default, IBTestConsole will try to detect the target framework executed. Use this option to force IBTestConsole to handle the executed command as a specific framework.

Supported frameworks:

• NUnit2
• NUnit3

...

/TestLevel=N

...

Specifies the following

• Assembly/test level – /testlevel=default will use assembly level distribution.
• Number of cores – replace the N with a number to specify the number of cores to be used in the test level distribution. IncrediBuild will distribute all the tests evenly between the cores (e.g. if there are 1000 tests and N=200, each of the 200 cores will run 5 tests).
• Test per core – the /testlevel=deep or /testlevel=1 will use all the available cores in the pool, while striving to run each test in a different core.

...

The number of cores to be used may determine the efficiency of the parallelization. In general, you should set the /testlevel=N command option so that N should be around x2.5 of the number of cores that you have on your local machine. To learn more about test level optimization, see Advanced Test Level Optimization.

...

/Help

...

Prints the help on the screen

...

/LogFile

...

Specifies the log file name

...

/LogLevel

...

Sets logging level. Default: Error level

Return Codes

IBTestConsole return the following return codes:

...

Code

...

Description

...

0

...

Successfully exited. This means that no internal error occurred during any of the following stages:

• user cmdline was correct
• discovery stage finished fine
• all tests were successfully executed
• The results were successfully merged (even with test failures)
• Console output was displayed to the user (if applicable)

...

1

...

Internal error during execution for any of stages described above

...

-1

...

Unexpected internal error. in real shouldn't ever occur

...

...

Visual Studio Solution:

IncrediBuild Solutions:

User Manual:

...

Image Added