Even good programmers make mistakes. The difference between a good programmer and a bad one is that a good one detects it sooner by using automated tests.

  • “One who doesn't test is doomed to repeat his or her own mistakes.” (wise proverb)
  • “When we get rid of one error, another one appears.” (Murphy's Law)
  • “Do test, do test, do test” (Martin Iljič Fowler)

Have you ever written the following code in PHP?

$obj = new MyClass;
$result = $obj->process($input);

var_dump($result);

So, have you ever dumped a function call result just to check by eye that it returns what it should return? You surely do it many times per day. If everything works, do you delete this code and expect that the class will not be broken in the future? Murphy's Law guarantees the opposite :-)

In fact, we wrote the test. And if we didn't delete it we could run it any time in the future to verify that everything still works as it should. You may create a large amount of these tests over time, so it would be nice if we were able to run them automatically. It would be useful to slightly modify test not to require our inspection, simply to be able to check itself.

And Nette Tester helps exactly with that.

Installation and requirements

Minimal required PHP version by Tester is 7.1. For more details, see the supported PHP versions table.

Preferred way of installation is by Composer and every following example assumes that. But the Tester can be used without it. It will be also shown below.

Installation by Composer

Let's assume you have the Composer up and running and an application named demo with the following structure:

demo/
├── src/           # application code we want to test
├── tests/         # tests we are writing
├── vendor/
└── composer.json

Navigate in CLI terminal to the application directory and add Tester as a dependency using Composer:

cd demo
composer require --dev nette/tester

Manual installation

Download Tester from GitHub and extract it or clone its repository by git clone https://github.com/nette/tester.git. Directory structure of our demo application is now as follows:

demo/
├── src/           # application code we want to test
├── tester/        # source of downloaded Tester
│   ├── src/
│   ├── tests/
│   ├── ...
│   └── readme.md
│
└── tests/         # tests we are writing

Running the Tester

Nette Tester is run from the command line. We can try it, without any arguments it will only show a help summary.

cd demo
vendor/bin/tester            # installation by Composer in UNIX
vendor\bin\tester            # installation by Composer in Windows
php tester/src/tester.php    # manual installation

For simplification we only use the tester command further in this document. But one of the ways mentioned above with a full path is required to run it.

Writing the tests

Our application has no tests yet. We create a simple class to be tested and we save it to file src/Greeting.php

<?php

class Greeting
{
    public function say($name)
    {
        if (!$name) {
            throw new InvalidArgumentException('Invalid name');
        }
        return "Hello $name";
    }
}

Let's write a test now. We save it into file tests/greeting.phpt. Don't be discouraged by its length, it is shown later how to simplify it.

<?php

use Tester\Assert;

# Load Tester library
require __DIR__ . '/../vendor/autoload.php';       # installation by Composer
require __DIR__ . '/../tester/src/bootstrap.php';  # manual installation

# Load the tested class. Composer or your autoloader surely takes
# care of that in practice.
require __DIR__ . '/../src/Greeting.php';


# Adjust PHP behaviour and enable some Tester features (described later)
Tester\Environment::setup();


$o = new Greeting;

Assert::same('Hello John', $o->say('John'));  # we expect the same

Assert::exception(function () use ($o) {       # we expect an exception
    $o->say('');
}, InvalidArgumentException::class, 'Invalid name');

The test is written, we can run it from command line for the first time:

cd tests
php greeting.phpt

We have run the test as an ordinary PHP script for the first time. Even it seems common, big potential is hidden in it. We can step through the test in IDE or load it via web browser.

The first run discloses syntactic errors and if we didn't make a typo, the test ends without error report. Let's change an assertion in the test to Assert::same('Hi John', $o->say('John')); and let's watch what happens when run.

As our application grows, number of tests grows with it. It would not be practical to run tests one by one. We use the Tester for running:

cd demo
tester tests/greeting.phpt  # we run a single test
tester tests                # we run all tests in directory

To see how more tests running looks like, let's try to run all tests which test the Tester itself. They are part of its installation:

tester vendor/nette/tester/tests  # installation by Composer
tester tester/tests               # manual installation

How the Tester runs

The Tester walks through the passed directories. It creates a list of found tests and it runs them one by one. It runs every test as a new process, every test runs completely isolated from others.

The Tester searches for *.phpt and *Test.php files. The previously failing tests run as first.

The Tester runs PHP processes with -n option, so without php.ini. More details in the running-tests chapter.

Test results evaluation

The Tester prints test results continuously during testing:

  • . (dot) – test passed
  • s – test has been skipped
  • F – test failed

Output may look like:

 _____ ___  ___ _____ ___  ___
|_   _/ __)( __/_   _/ __)| _ )
  |_| \___ /___) |_| \___ |_|_\  v2.0.0

PHP 7.1.3 (cli) | php -n | 8 threads

........s................F.........

-- FAILED: tests/greeting.phpt
   Failed: 'Hello John' should be
       ... 'Hi John'

   in src/Framework/Assert.php(370)
   in src/Framework/Assert.php(52) Tester\Assert::fail()
   in tests/greeting.phpt(6) Tester\Assert::same()


FAILURES! (35 tests, 1 failures, 1 skipped, 1.7 seconds)

35 tests were run, one failed, one was skipped. Tester's exit code is zero if no one fails. Non-zero otherwise.

Environment::setup()

There has been mentioned a little bit cryptic Tester\Environment::setup() call in tests. What does it do?

  • improves error dump readability (coloring included), otherwise, default PHP stack trace is printed
  • enables check that assertions has been called in test, otherwise, tests without (e.g. forgotten) assertions pass too
  • automatically starts code coverage collector when --coverage is used (described later)

Using is optional but recommended.

Directories structure

It may seem prematurely to talk about it, but well-structured directories with tests saves a lot of work. We separate tests into subdirectories by namespace of tested classes:

demo/
└── tests/
    ├── NamespaceOne/
    │   ├── MyClass.getUsers.phpt
    │   ├── MyClass.setUsers.phpt
    │   └── ...
    │
    ├── NamespaceTwo/
    │   ├── MyClass.creating.phpt
    │   ├── MyClass.dropping.phpt
    │   └── ...
    │
    ├── test.one.phpt
    ├── test.two.phpt
    ├── ...
    └── bootstrap.php

and we create a file bootstrap.php. It contains a common code for all tests. For example classes autoloading, environment configuration, temporary directory creation, helpers and similar. Every test loads the bootstrap and pays attention to testing only. The bootstrap can look like:

require __DIR__ . '/../vendor/autoload.php';

Tester\Environment::setup();
date_default_timezone_set('Europe/Prague');

define('TMP_DIR', '/tmp/demo-app-tests');

There is no difference to run the tests with changed directory structure. The Tester finds all *.phpt and *Test.php tests recursively and runs them:

cd demo
tester tests

But we can run tests for a single namespace easily:

tester tests/NamespaceOne

Supported PHP versions

Tester \ PHP 5.3 5.4 5.5 5.6 7.0 7.1 7.2 7.3
0.9.0 – 1.7.1 Yes Yes Yes Yes Yes Yes no no
1.7.2 Yes Yes Yes Yes Yes Yes Yes Yes
2.0.* no no no Yes Yes Yes Yes Yes
2.1.* no no no no no Yes Yes Yes
dev (master) no no no no no Yes Yes Yes

Till version 1.7 Tester supported HHVM 3.3.0 or newer. Support has been dropped since Tester 2.0. Usage was simple:

tester -p hhvm

Continuous integration

If you’re not familiar with continuous integration, this guide will show you what it is all about.