Beware of PHPUnit data providers with heavy setup methods

Data providers can be a perfect fit to assert a lot of expectations without writing a full test for each.

This makes it cheap and easy to add more test cases. For an in-depth introduction to data providers, I recommend this excellent article on the Tighten blog.

class AddTest extends TestCase
{
    /** @dataProvider values */
    public function it_adds_values(int $a, int $b, int $result): void
    {
        $this->assertEquals($result, add($a, $b));
    }

    public function values(): array
    {
        return [
            [1, 1, 2],
            [1, 2, 3],
            [5, 5, 10],
            // …
        ];
    }
}

Data providers run setUp for each value. If you need a clean slate for every test, there’s no way around this. If you don’t, data providers make your tests slower than they need to be.

Consider a heavier integration test that migrate & seeds the database in setUp().

class MyTest extends TestCase
{
    public function setUp(): void
    {
        parent::setUp();

        // Migrate and seed database values…
    }

    /** @dataProvider values */
    public function it_does_something(string $value): void
    {
      // …
    }

    public function values(): array
    {
        return [ /* … */ ];
    }
}

Each case in values() will re-seed the database. If this isn’t needed, you’re better off looping over the values yourself.

class MyTest extends TestCase
{
    public function setUp(): void
    {
        parent::setUp();

        // Do some heavy lifting…
    }

    public function it_does_something(): void
    {
      $values = [ /* … */ ];

      foreach ($this->value() as $value) {
          // …
      }
    }

    public function values(): array
    {
        return [ /* … */ ];
    }
}

We recently updated a few tests in a project we’re working on, and it almost doubled the speed!

With a data provider:

Time: 00:08.517, Memory: 92.50 MB
OK (43 tests, 43 assertions)

In a loop:

Time: 00:04.448, Memory: 70.14 MB
OK (1 test, 43 assertions)