Running Tests
The most visible part of Nette Tester is the command-line test runner. It is extremely fast and robust because it automatically runs all tests as separate processes in parallel using multiple threads. It can also run itself in watch mode.
The test runner is invoked from the command line. Pass the directory containing the tests as a parameter. For the current directory, simply enter a dot:
vendor/bin/tester .
The test runner scans the specified directory and all its subdirectories, looking for tests, which are files ending in
*.phpt
or *Test.php
. It also reads and evaluates their annotations to determine which tests to run and how.
It then executes the tests. During execution, it prints characters to the terminal to indicate the progress:
.
– test passeds
– test has been skippedF
– test failed
The output may look like this:
_____ ___ ___ _____ ___ ___
|_ _/ __)( __/_ _/ __)| _ )
|_| \___ /___) |_| \___ |_|_\ v2.5.2
Note: No php.ini is used.
PHP 8.3.2 (cli) | php -n | 8 threads
........s..........................
OK (35 tests, 1 skipped, 1.7 seconds)
When run again, it first executes the tests that failed in the previous run, so you immediately know if you have fixed the error.
The Tester's exit code is zero if no test fails. Otherwise, it is non-zero.
The Tester runs PHP processes without php.ini
. More details in the Own
php.ini section.
Command-Line Options
You can get an overview of command-line options by running Tester without parameters or with the -h
option:
_____ ___ ___ _____ ___ ___ |_ _/ __)( __/_ _/ __)| _ ) |_| \___ /___) |_| \___ |_|_\ v2.5.2 Usage: tester [options] [<test file> | <directory>]... Options: -p <path> Specify PHP interpreter to run (default: php). -c <path> Look for php.ini file (or look in directory) <path>. -C Use system-wide php.ini. -d <key=value>... Define INI entry 'key' with value 'value'. -s Show information about skipped tests. --stop-on-fail Stop execution upon the first failure. -j <num> Run <num> jobs in parallel (default: 8). -o <console|console-lines|tap|junit|log|none> (e.g. -o junit:output.xml) Specify one or more output formats with optional file name. -w | --watch <path> Watch directory. -i | --info Show tests environment info and exit. --setup <path> Script for runner setup. --temp <path> Path to temporary directory. Default by sys_get_temp_dir(). --colors [1|0] Enable or disable colors. --coverage <path> Generate code coverage report to file. --coverage-src <path> Path to source code. -h | --help This help.
-p <path>
Specifies the PHP binary that will be used to run tests. By default, it is php
.
tester -p /home/user/php-7.2.0-beta/php-cgi tests
-c <path>
Specifies which php.ini
will be used when running tests. By default, no php.ini is used. See Own php.ini for more information.
-C
A system-wide php.ini
is used. So on UNIX platform, all the /etc/php/{sapi}/conf.d/*.ini
files too.
See Own php.ini section.
-d <key=value>
Sets the value of a PHP configuration directive for the tests. This parameter can be used multiple times.
tester -d max_execution_time=20
-s
Displays information about skipped tests.
--stop-on-fail
Tester stops testing upon the first failing test.
-j <num>
Specifies the number of parallel processes to run tests in. The default value is 8. To run tests sequentially, use the value 1.
-o <console|console-lines|tap|junit|log|none>
Sets the output format. The default is the console format. You can specify the name of the file into which the output will be
written (e.g., -o junit:output.xml
). The -o
option can be repeated multiple times to generate multiple
formats at once.
console
: same as the default format, but the ASCII logo is not printed in this caseconsole-lines
: similar to console, but the result of each test is listed on a separate line with additional informationtap
: TAP format suitable for machine processingjunit
: JUnit XML format, also suitable for machine processinglog
: Outputs the testing progress. Includes all failed, skipped, and also successful testsnone
: nothing is printed
-w | --watch <path>
After completing the tests, Tester does not exit but continues to run and watch PHP files in the specified directory. When a file changes, it runs the tests again. This parameter can be used multiple times if you want to watch multiple directories.
Useful when refactoring a library or debugging tests.
tester --watch src tests
-i | --info
Shows information about the runtime environment for tests. For example:
tester -p /usr/bin/php7.1 -c tests/php.ini --info PHP binary: /usr/bin/php7.1 PHP version: 7.1.7-1+0~20170711133844.5+jessie~1.gbp5284f4 (cli) Code coverage engines: (not available) Loaded php.ini files: /var/www/dev/demo/tests/php.ini PHP temporary directory: /tmp Loaded extensions: Core, ctype, date, dom, ereg, fileinfo, filter, hash, ...
--setup <path>
Tester loads the specified PHP script at startup. The variable Tester\Runner\Runner $runner
is available within
this script. Assume a file tests/runner-setup.php
with the following content:
$runner->outputHandlers[] = new MyOutputHandler;
We run Tester with:
tester --setup tests/runner-setup.php tests
--temp <path>
Sets the path to the directory for Tester's temporary files. The default value is returned by sys_get_temp_dir()
.
You will be notified if the default value is not valid.
If you are unsure which directory is being used, run Tester with the --info
parameter.
--colors 1|0
By default, Tester detects if the terminal supports colors and colorizes its output accordingly. This option overrides the
auto-detection. You can set coloring globally using the NETTE_TESTER_COLORS
environment variable.
--coverage <path>
Tester generates a report showing how much of the source code is covered by tests. This option requires the Xdebug or PCOV PHP extension to be installed, or PHP 7+ with the PHPDBG SAPI, which is faster. The extension of the target file determines its format: HTML or Clover XML.
tester tests --coverage coverage.html # HTML report tester tests --coverage coverage.xml # Clover XML report
The priority for selecting the coverage engine is as follows:
- PCOV
- PHPDBG
- Xdebug
When using PHPDBG, extensive tests might fail due to memory exhaustion. Collecting code coverage information is
memory-intensive. In this case, calling Tester\CodeCoverage\Collector::flush()
within your test can help. It writes
the collected data to disk and frees up memory. If data collection is not running or if Xdebug is used, the call has no
effect.
See an `example HTML report with code coverage.
--coverage-src <path>
Used in conjunction with the --coverage
option. <path>
is the path to the source code for which
the report is generated. Can be used multiple times.
Own php.ini
Tester runs PHP processes with the -n
option, which means that no php.ini
is loaded (not even those
from /etc/php/conf.d/*.ini
on UNIX systems). This ensures a consistent environment for running tests, but it also
disables all external PHP extensions normally loaded by the system's PHP.
To preserve the loading of system php.ini files, use the -C
parameter.
If you need specific extensions or special INI settings for your tests, we recommend creating your own php.ini
file and distributing it with your tests. Then, run Tester with the -c
option, for example,
tester -c tests/php.ini tests
. The INI file might look like this:
[PHP]
extension=php_pdo_mysql.dll
extension=php_pdo_pgsql.dll
memory_limit=512M
Running Tester on UNIX with a system php.ini
, like tester -c /etc/php/cli/php.ini
, does not load
other INIs from /etc/php/conf.d/*.ini
. This is a PHP behavior, not specific to Tester.