How to run Behat scenarios and functional tests from a Symfony bundle in isolation of a project?

When working on a reusable bundle it's beneficial to run its test suite in isolation of a project. This way the test suite is not dependent on project's configuration or enabled bundles. It is also much easier to run it on a continuos integration server like Travis.

AppKernel and configuration

Providing an AppKernel is the main task we need to do to run both Behat scenarios and functional tests from our bundle without installing it in a Symfony2 project.

Within an AppKernel we're able to register bundles and container configuration (registerBundles() registerContainerConfiguration() methods). Next to the bundle we're working on, we should enable and configure bundles required for it to work.

Configuration is typically done in config/config_test.yml file (scenarios and tests are usually run in a test environment). Most of the time we'll also need a routing file (typically config/routing_test.yml).

Last but not least it's good to change the default location of cache and logs to a temporary directory (getCacheDir() and getLogDir() methods).

<?php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    /**
     * @return array
     */
    public function registerBundles()
    {
        return array(
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
            new Symfony\Bundle\MonologBundle\MonologBundle(),
            new Sensio\Bundle\FrameworkExtraBundle\SensioFrameworkExtraBundle(),
            new Zalas\Bundle\DemoBundle\ZalasDemoBundle()
        );
    }

    /**
     * @return null
     */
    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        $loader->load(__DIR__.'/config/config_'.$this->getEnvironment().'.yml');
    }

    /**
     * @return string
     */
    public function getCacheDir()
    {
        return sys_get_temp_dir().'/ZalasDemoBundle/cache';
    }

    /**
     * @return string
     */
    public function getLogDir()
    {
        return sys_get_temp_dir().'/ZalasDemoBundle/logs';
    }
}

I usually put the kernel in Features/Fixtures/Project/app/AppKernel.php (for Behat) or Tests/Functional/app/AppKernel.php (for functional tests) but the location doesn't matter.

For more flexibility look at the FrameworkBundle or JMSPaymentCoreBundle. Both bundles have good examples of parametrized kernel configurations.

Autoloader

Using composer solves most of our autoloading problems. We only need to remember of setting target-dir in our composer.json (big thanks to Adrien Brault for pointing it out):

{
    "autoload": {
        "psr-4": { "Zalas\\Bundle\\DemoBundle\\": "" }
    }
}

We'll also need to register an autoloader for annotations (if we use annotations).

Following example of bootstrap.php is a simple implementation of an autoloader for functional tests. To use it with Behat path to the vendor/autoload.php needs to be updated.

<?php

use Doctrine\Common\Annotations\AnnotationRegistry;

if (!file_exists($file = __DIR__.'/../vendor/autoload.php')) {
    throw new \RuntimeException('Install the dependencies to run the test suite.');
}

$loader = require $file;
AnnotationRegistry::registerLoader(array($loader, 'loadClass'));

Behat

Once we prepared the AppKernel and set up the autoloading we can move to Behat configuration. In particular, we need to define:

• path to the Features folder
• list of contexts we need to use
• path to the AppKernel (kernel.path for the Symfony2 extension)
• path to the bootstrap file for the Symfony2 extension (kernel.bootsrap for the Symfony2 extension)

default:
  formatters:
    progress: true
  suites:
    demo:
      paths:
        features: Features
      contexts: [Zalas\Bundle\DemoBundle\Features\Context\FeatureContext]
  extensions:
    Behat\Symfony2Extension:
      kernel:
        env: test
        debug: true
        path: Features/Fixtures/Project/app/AppKernel.php
        bootstrap: Features/Fixtures/Project/app/bootstrap.php
    Behat\MinkExtension:
      base_url: 'http://www.acme.dev/app_test.php/'
      sessions:
        default:
          symfony2: ~

Now we can run our Behat scenarios without installing the bundle in a Symfony project:

./vendor/bin/behat

Symfony2 functional tests

Configuration for Symfony2 functional tests is done in a standard PHPUnit file (typically phpunit.xml.dist). We need to provide a path to the AppKernel as an environment variable (KERNEL_DIR).

<?xml version="1.0" encoding="UTF-8"?>

<phpunit bootstrap="./Tests/bootstrap.php" color="true">
  <testsuites>
    <testsuite name="ZalasDemoBundle test suite">
      <directory suffix="Test.php">./Tests</directory>
    </testsuite>
  </testsuites>

  <php>
    <strong> <server name="KERNEL_DIR" value="./Tests/Functional/app" /></strong>
  </php>

  <filter>
    <whitelist>
      <directory>./</directory>
      <exclude>
        <directory>./Resources</directory>
        <directory>./Tests</directory>
        <directory>./vendor</directory>
      </exclude>
    </whitelist>
  </filter>
</phpunit>

Now we can run our functional tests without installing the bundle in a Symfony project:

phpunit

Travis CI

With such a setup running our bundle's test suite on an integration server becomes very simple. Here's an example .travis.yml file:

language: php

php:
  - 5.3
  - 5.4

before_script:
  - curl -s http://getcomposer.org/installer | php
  - php composer.phar --dev install

script:
  - 'phpunit --coverage-text && ./vendor/bin/behat'

Need to use a database? Just create it before running the script (don't forget to remove it afterwards):

language: php

php:
  - 5.3
  - 5.4

before_script:
  - curl -s http://getcomposer.org/installer | php
  - php composer.phar --dev install
  - mysql -e 'CREATE DATABASE zalas_demo_test;'

script:
  - 'phpunit --coverage-text && ./vendor/bin/behat'

after_script:
  - mysql -e 'DROP DATABASE zalas_demo_test;'

Demo

To demonstrate this approach I prepared a DemoBundle. You can clone it from github and test how it works yourself, or you can see how it's run on Travis.