Magento Cloud code testing

In a Magento Cloud Docker development environment, you can use the integrated test suite in each Magento Commerce Cloud Suite package for Magento code testing. The test suites, which use the Codeception testing framework for PHP, provide acceptance tests to validate code intended for contribution to Magento Cloud package repositories.

Before you run tests, you must prepare your Docker environment and update the test configuration file. (See Prepare the test environment.)

Test resources

Use the following table to review the test configuration files and available tests for each package.

For Magento application testing, use the Magento Application Testing (MFTF) framework to run functional tests. See Magento application testing.

Prepare the Docker environment for testing

To set up and configure the test environment:

1. Clone the GitHub repository for the package to test.

   1
   git clone git@github.com:magento/<cloud-suite-package-repository>.git
   

   For example:

   1
   git clone git@github.com:magento/ece-tools.git
   

2. Stop all services that use the following ports:

   • 80—varnish
   • 443—web, tls
   • 3306—apache, mysql

3. Add the following hostname to your /etc/hosts file.

   1
   127.0.0.1 magento2.docker
   

   Alternatively, you can run the following command to add it to the file:

   1
   echo "127.0.0.1 magento2.docker" | sudo tee -a /etc/hosts
   

4. Add required project dependencies if you are testing the following packages.

   • For the magento/magento-cloud-patches package:

     1
     composer require "magento/magento-cloud-docker:^1.0.0" --no-update
     

   • For the magento/magento-cloud-components package:

     1
     2
     3
     4
     composer require "magento/magento-cloud-docker:^1.0.0" --no-update
     composer require "magento/framework:*" --no-update
     composer require "magento/module-store:*" --no-update
     composer require "magento/module-url-rewrite:*" --no-update
     

5. Switch to the preferred PHP version for running tests.

6. Update the project dependencies.

   1
   composer update
   

Add credentials

Add credentials to your Docker environment using any of the following methods:

• Use environment variables
• Load credentials from the environment configuration file
• Add variables directly to the test configuration file

To add credentials using environment variables:

1. Create environment variables for your GitHub authentication keys.

   1
   export REPO_USERNAME=your_public_key
   

   1
   export REPO_PASSWORD=your_private_key
   

2. Some packages require a GitHub authentication during installation. Create an environment variable with your GitHub token that can be used to install these packages.

   1
   `export GITHUB_TOKEN=your_github_token
   

To load credentials from the environment configuration file:

1. Run the following commands to write credentials to the ./.env file.

   1
   echo "REPO_USERNAME=your_public_key" >> ./.env
   

   1
   echo "REPO_PASSWORD=your_private_key" >> ./.env
   

2. Edit the codeception.dist.yml file.

   1
   2
   3
   4
   params:
       - tests/functional/configuration.dist.yml
       - env
       - .env
   

3. Add a dependency for the vlucas/phpdotenv package required to load the environment variables.

   1
   composer require "vlucas/phpdotenv": "^3.0"
   

To add credentials directly to the test configuration file:

1. Open the codeception.dist.yml file in an editor.

2. Replace the %REPO_USERNAME%, %REPO_PASSWORD%, and %GITHUB_TOKEN% placeholder values with your credentials:

   1
   2
   3
   4
   5
   6
   7
   8
   modules:
     config:
       Magento\CloudDocker\Test\Functional\Codeception\TestInfrastructure:
       ...
       composer_magento_username: "%REPO_USERNAME%"
       composer_magento_password: "%REPO_PASSWORD%"
       composer_github_token: "%GITHUB_TOKEN%"
         ...
   

Run tests

By default, functional tests produce a short output. You can receive more detailed output by editing the codeception.dist.yml test configuration file to set the printOutput: property to true.

1
2
3
4
5
6
7
8
9
10
modules:
  config:
    Magento\CloudDocker\Test\Functional\Codeception\TestInfrastructure:
      ...
      printOutput: true
      ...
    Magento\CloudDocker\Test\Functional\Codeception\Docker:
      ...
      printOutput: true
      ...

You can locate the test configuration file in the root directory for each package. (See Test resources).

Run a specific test

Use the following command format to run a specific functional test:

1
./vendor/bin/codecept run Acceptance <TestName>Cest

For example, the following test for magento/ece-tools code verifies that the post-deploy task runs successfully.

1
./vendor/bin/codecept run Acceptance PostDeployCest

Sample response:

1
2
3
4
5
6
7
8
9
10
11
12
Codeception PHP Testing Framework v2.5.6
Powered by PHPUnit 6.5.14 by Sebastian Bergmann and contributors.
Running with seed:
Acceptance Tests (1) -----------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------
PostDeployCest: Test post deploy | {"ADMIN_EMAIL":"admin@example.com"}
 [Magento\MagentoCloud\Test\Functional\Robo\Tasks\GenerateDockerCompose] Running ./bin/ece-docker build:compose
 --mode=functional --php=7.2
...
...
✔ PostDeployCest: Test post deploy | {"ADMIN_EMAIL":"admin@example.com"} (210.41s)

You can see the available tests for each package in the package source files. See Test resources.

Run all tests

Use the following commands to run all available tests for each PHP versions:

• PHP 7.1

  1
  ./vendor/bin/codecept run -g php71 --steps
  

• PHP 7.2

  1
  ./vendor/bin/codecept run -g php72 --steps
  

• PHP 7.3

  1
  ./vendor/bin/codecept run -g php73 --steps
  

  If you are testing magento/ece-tools code, use the following command:

  1
  ./vendor/bin/codecept run -x php71 -x php72 --steps
  

For help with Codeception command options, use the .vendor/bin/codecept run --help command, or see the Codeception CLI documentation for the installed version.