Running Tests

Nette Tester is run from the command line. We can try it, without any arguments it will only show a help summary.

vendor/bin/tester            # installation by Composer in UNIX
vendor\bin\tester            # installation by Composer in Windows
php tester/src/tester.php    # manual installation

For simplification we only use the tester command further in this document. But one of the ways mentioned above with a full path is required to run it.

As a parameter we will pass the test directory. For the current directory just enter a dot:

tester .

The Tester walks through the passed directory and subdirectories. It creates a list of found tests and it runs them one by one. It runs every test as a new process, every test runs completely isolated from others.

The Tester searches for *.phpt and *Test.php files. The previously failing tests run as first.

The Tester runs PHP processes with -n option, so without php.ini. More details in the Own php.ini chapter.

Command line options

We obtain command line options overview by running the Tester without parameters or with option -h.

-p <path>

The Tester uses this PHP binary for running the tests:

tester -p /home/user/php-7.2.0-beta/php-cgi tests

-c <path>

The option is the same as for PHP binary. Passed php.ini is used for tests. No php.ini is used in default. Example when we distribute php.ini along with tests:

tester -c tests/php.ini tests

-C

A system-wide php.ini is used. So on UNIX platform, all the /etc/php/{sapi}/conf.d/*.ini files too.

-l | –log <path>

Testing progress is written into file. All failed, skipped and also successful tests:

tester --log /var/log/tests.log tests

-d <key=value>

The option is the same as for PHP binary. It defines INI value for tests. The option can be used multiple times.

-s

Information about skipped tests will be shown.

--stop-on-fail

Tester stops testing upon the first failing test.

-j <num>

Tests run in a <num> parallel precesses. Default value is 8. If we wish to run tests in series, we use value 1.

-o <console|tap|junit|none>

Output format. Default is the console format.

  • console: the same as default, but the ASCII logo is not printed in this case
  • tap: TAP format appropriate for machine processing
  • junit: JUnit XML format, appropriate for machine processing too
  • none: nothing is printed

-w | –watch <path>

The Tester doesn't end after tests are completed but it keeps running and watching given directory. It runs tests again whenever directory changes. Parameter can be used multiple times. It is handy during tests writing and debugging.

-i | –info

It shows information about a test running environment. For example:

tester -p /usr/bin/php7.1 -c tests/php.ini --info


PHP binary:
/usr/bin/php7.1

PHP version:
7.1.7-1+0~20170711133844.5+jessie~1.gbp5284f4 (cli)

Loaded php.ini files:
/var/www/dev/demo/tests/php.ini

PHP temporary directory:
/tmp

Loaded extensions:
Core, ctype, date, dom, ereg, fileinfo, filter, hash, ...

--setup <path>

The Tester loads given PHP script on start. Variable Tester\Runner\Runner $runner is available in it. Let's assume file tests/runner-setup.php:

$runner->outputHandlers[] = new MyOutputHandler;

and we run the Tester:

tester --setup tests/runner-setup.php tests

--temp <path>

Sets a path to directory for temporary files of Tester. Default value is by sys_get_temp_dir(). When default value is not valid, you will be noticed.

If we are not sure which directory is used, we can run Tester with the --info parameter.

--colors 1|0

The Tester detects a colour-able terminal in default and colorizes its output. This option is over the autodetection. We can set colouring globally by a system environment variable NETTE_TESTER_COLORS.

--coverage <path>

Tester will generate a report with overview how much is the source code coveraged by tests. This option requires PHP extension Xdebug enabled, or PHP 7 with the PHPDBG SAPI, which is faster. The destination file extension determines the contents format. HTML or Clover XML.

tester tests --coverage coverage.html  # HTML report
tester tests --coverage coverage.xml   # Clover XML report

Extensive tests may fails during run by PHPDBG due to memory exhaustion. Coverage data collecting is memory consuming operation. In that case, calling the Tester\CodeCoverage\Collector::flush() inside a test may help. It will flush collected data into file and frees memory. When data collection is not in progress, or Xdebug is used, calling has no effect.

An example of HTML report with code coverage.

--coverage-src <path>

We use it with option --coverage simultaneously. The <path> is a path to source code for which we generate the report. Can be used repeatedly.

Own php.ini

The Tester runs PHP processes with -n option, so no php.ini is loaded (not even that from /etc/php/conf.d/*.ini in UNIX). It ensures the most clean environment for the tests run, but it also deactivates all the external PHP extensions commonly loaded by system PHP.

If you want to keep system configuration, use the -C parameter.

If you need some extensions or some special INI settings, we recommend to create own php.ini file and distribute it among the tests. Then we run Tester with -c option, e.g. tester -c tests/php.ini. The INI file may look like:

[PHP]

extension=php_pdo_mysql.dll
extension=php_pdo_pgsql.dll

memory_limit=512M

Running the Tester with a system php.ini in UNIX, e.g. tester -c /etc/php/cgi/php.ini, does not load other INI from /etc/php/conf.d/*.ini. That's the PHP behaviour, not the Tester's.