Test Annotations

Annotations determine how tests will be handled by command-line test runner. They are written at the beginning of the test file.

Annotations are case insensitive. They also have no effect if the test is run manually as a regular PHP script.

Example:

/**
 * TEST: Basic database query test.
 *
 * @dataProvider files/databases.ini
 * @exitCode 56
 * @phpVersion < 5.5
 */

require __DIR__ . '/../bootstrap.php';

TEST

It is not an annotation actually. It only sets the test title which is printed on fail or into logs.

@skip

Test is skipped. It is handy for temporary test deactivation.

@phpVersion

Test is skipped if is not run by the corresponding PHP version. We write annotation as @phpVersion [operator] version. We can leave out the operator, default is >=. Examples:

/**
 * @phpVersion 5.3.3
 * @phpVersion < 5.5
 * @phpVersion != 5.4.5
 */

@phpExtension

Test is skipped if all mentioned PHP extension are not loaded. Multiple extensions can be written in a single annotation, or we can use the annotation multiple times.

/**
 * @phpExtension pdo, pdo_pgsql, pdo_mysql
 * @phpExtension json
 */

@dataProvider

This annotation suits when we want to run the test multiple times but with different data. (Not to be confused with the annotation of the same name for TestCase.)

We write annotation as @dataProvider file.ini. INI file path is relative to the test file. Test runs as many times as number of sections contained in the INI file. Let's assume the INI file databases.ini:

[mysql]
dsn = "mysql:host=127.0.0.1"
user = root
password = ******

[postgresql]
dsn = "pgsql:host=127.0.0.1;dbname=test"
user = postgres
password = ******

[sqlite]
dsn = "sqlite::memory:"

and the file database.phpt in the same directory:

/**
 * @dataProvider databases.ini
 */

$args = Tester\Environment::loadData();

The test runs three times and $args will contain values from sections mysql, postgresql or sqlite.

There is one more variation when we write annotations with a question mark as @dataProvider? file.ini. In this case, test is skipped if the INI file doesn't exist.

Annotation possibilities have not been mentioned all yet. We can write conditions after the INI file. Test runs for the given section only if all conditions match. Let's extend the INI file:

[mysql]
dsn = "mysql:host=127.0.0.1"
user = root
password = ******

[postgresql 8.4]
dsn = "pgsql:host=127.0.0.1;dbname=test"
user = postgres
password = ******

[postgresql 9.1]
dsn = "pgsql:host=127.0.0.1;dbname=test;port=5433"
user = postgres
password = ******

[sqlite]
dsn = "sqlite::memory:"

and we will use annotation with condition:

/**
 * @dataProvider  databases.ini  postgresql, >=9.0
 */

The test runs only once for section postgresql 9.1. Other sections don’t match the conditions.

Similarly, we can pass path to a PHP script instead of INI. It must return array or Traversable. File databases.php:

return [
	'postgresql 8.4' => [
		'dsn' => '...',
		'user' => '...',
	],

	'postgresql 9.1' => [
		'dsn' => '...',
		'user' => '...',
	],
];

@multiple

We write it as @multiple N where N is an integer. Test runs exactly N-times.

@testCase

Annotation has no parameters. We use it when we write a test as TestCase classes. In this case, the command-line test runner will run the individual methods in separate processes and in parallel in multiple threads. This can significantly speed up the entire testing process.

@exitCode

We write it as @exitCode N where N is the exit code of the test. For example if exit(10) is called in the test, we write annotation as @exitCode 10. It is considered to fail if the test ends with a different code. Exit code 0 (zero) is verified if we leave out the annotation

@httpCode

Annotation is evaluated only if PHP binary is CGI. It is ignored otherwise. We write it as @httpCode NNN where NNN is expected HTTP code. HTTP code 200 is verified if we leave out the annotation. If we write NNN as a string evaluated as zero, for example any, HTTP code is not checked at all.

@outputMatch a @outputMatchFile

The behavior of annotations is consistent with Assert::match() and Assert::matchFile() assertions. But pattern is found in test’s standard output. A suitable use case is when we assume the test to end by fatal error and we need to verify its output.

@phpIni

It sets INI configuration values for test. For example we write it as @phpIni precision=20 and it works in the same way as if we passed value from the command line by parameter -d precision=20.