Nette Tester 2.0.0 is out

Compatibility

New Tester requires PHP in version 5.6 or newer and supports PHP 7.2. Simultaneously, support for HHVM has been dropped. There have been small opinion poll on this topic.

Changes in tests runner

Tester's default interpreter (when called without the -p option) is CLI now. It used to be CGI. You will see the notice in the console about it.

When you pass a directory as a tester argument, it will find all *.phpt files as usual, but newly it will find all *Test.php files additionally.

The Tester used to run tests in order as they have been found. Now, it will run last time failing tests as first. For this purpose, the Tester uses a temporary directory. Default is got by sys_get_temp_dir() and you can change it by explicit --temp <path> option. The --info call will help you find out currently used directory. Try to think about this new feature in relation to --stop-on-fail or --watch options. It may improve your workflow.

As mentioned in the documentation, every test is run by PHP in a separated process with -n option. So without php.ini file. Sometimes it is painful to tune up own php.ini file for tests. A new option -C is introduced, it causes that -n is not used and as a consequence, tests run with system-wide PHP settings.

Sometimes is good to know whether test is running by tester myTest.phpt or as an ordinary PHP script by php myTest.phpt. An environment variable will help you to recognize it:

if (getenv(Tester\Environment::RUNNER)) {
	# by Tester
} else {
	# another way
}

And when talking about environment variables, there is another one. When tests run in parallel, by default in 8 threads, you can be interested in a current thread number.

$threadNumber = getenv(Tester\Environment::THREAD);

Changes in testing framework

New modifiers for Assert::match() have been added. The %w% matches alphanumeric chars 0-9, a-z, A-Z and _ (underscore). The %% matches one % char.

The behavior of FileMock has been fixed, mainly in an append mode.

Another change concerns the TestCase. Now, the TestCase::tearDown() run always, even aften an error occurred in a testing method. Small BC break is that the TestCase::run() method does not accept arguments anymore. If you need to run a single testing method, use the TestCase::runTest($method, array $args = []) method.

New test file annotation @phpExtension has been added. When some of the given extensions are not loaded, the test will be skipped. You can use one annotation with multiple extensions or split it into multiple annotations. For example:

/**
 * @phpExtension pdo, pdo_pgsql
 * @phpExtension json
 */

Completely new helper FileMutator has been added. Its purpose is to adjust the source code of loaded files. The first real use case is removing the final keyword from classes and class methods. Usage is simple. Call the Tester\Environment::bypassFinals() on the very top of a test file, it will setup FileMutator for you.

require 'bootstrap.php';
Tester\Environment::bypassFinals();
# Now, you can extend final classes or overload final class methods.

Changes in output formating

When other than the expected exception is thrown in the Assert::exception(), its stack trace will be printed. It helps to find out, what is wrong.

If you are using Assert::match() with modifiers (%a%, %d% and so on), you will be pleasured by a new feature. In case of fail, modifiers are replaced by their matching values in output and dumps. An example will say more. Imagine test such:

$tested = "My name is Milo and I'm -65535 years old.";
Assert::match("My name is %S% and I'm %d% years old.", $tested);

# An output before the Tester 2
	Failed: 'My name is Milo and I'm -65535 years old.' should match
		... 'My name is %S% and I'm %d% years old.'

# An output with new Tester
	Failed: 'My name is Milo and I'm -65535 years old.' should match
		... 'My name is Milo and I'm %d% years old.'

Do you see that small difference? The %S% modifier is now replaced by a matching value and you don't need to decrypt where the mismatch occurred. Very handy when matching long strings.

If you use code coverage functionality with a PHPDBG, some tests may fail due to memory exhaustion. Coverage data collecting is a memory consuming operation. You can increase memory limits, or you can call Tester\CodeCoverage\Collector::flush(). It will write collected data into the file and release occupied memory. Calling has no effect with Xdebug or when collecting is not running at all.

The HTML output of code coverage has been facelifted. Check out an example.

And here we come with the biggest BC break in the Tester 2. Changes in the OutputHandler interface. Before I'll describe it, I have to make a short introduction about how tests are handled internally by the Tester.

Inside the Tester, a single test is represented by Test class instance. In a short, it contains test file path, test title, result code, captured standard output and so on. Check out the source code for details. These instances are passed to output handlers in the Tester 2. Refactored OutputHandler now looks as follows:

namespace Tester\Runner;

interface OutputHandler
{
	# Before testing, something like init phase.
	function begin();

	# When a new test has been found. Called for every test before testing starts.
	function prepare(Test $test);

	# Test achives some result.
	function finish(Test $test);

	# Called on a testing end. Not necessarily all tests have been run.
	function end();
}

Some other minor improvements have been done, like printing code coverage percent value into the console, or dumping arrays in a short syntax.

In conclusion

The Tester's internal code went through more changes, but they are not important for end-users. For example, Tester's codebase now uses a new Nette coding style with small true, false, null constants and uses a new code checker. Details can be found in commits history.

Documentation is up to date and is still in the same place as usual.

Many thanks to all contributors! Enjoy the new release!