How To Automatically Deploy Your PHP Apps

A typical developing environment includes versioning the application with Git, so how does one get the project into production without logging into the server every time to pull the new code, refresh the cache, compile the assets, and so on?! This seems like a bad idea to do every time you want to deploy something into production. In this article, we’re going to talk about a tool called PHP Deployer, which allows us to create a script that will do all that, and more, for us.

What Does Deploying Mean?

Deploying refers to the process of moving your application from local or development environments to the production server(s). In the past, people used tools like FTP or rsync to simply upload the files onto the server and be done. However, nowadays, we’re using more advanced tools, such as Git, caching, JS / CSS compilers, dependency managers like Composer, NPM, Bower, Grunt, and all of these need a way to interact with the production server to run specific commands.

By the way, if you don’t know how to use Git, here’s a tutorial on it.

How do we Automise this Process?

The way I see it, there are quite a few options:

1. Creating a shell script to run when we want to do a deploy.
2. Creating a script with the commands and when to run with the help of a tool written in your language of choice. This is basically the same approach as the first, but it simplifies the process by creating a wrapper over the shell script commands to connect to the server, run commands on it, and so on.
3. Using a Continuous Integration (CI) tool, like Travis CI or Scrutinizer CI, or AWS, which allows you to define these commands inside their configuration. However, this option is handy when we’re using AWS or the CI tool in other sites of the app. We can’t really use only the deployment part that easily.
4. Using tools like Laravel Forge.

There probably are other ways of doing this, but this article is about small projects that only need one server to operate, hence we don’t need the full power of AWS, Laravel Forge or a CI tool. We’re going to focus on the second option, using a tool to create a deployment script.

PHP Deployer

PHP Deployer allows us to set up servers and tasks which we can schedule to run in any order we want on any server we want.

Installation

To install PHP deployer, simply download it from the official website with the following command:

curl -L https://deployer.org/deployer.phar -o dep

Save it to your project’s root directory.

Give it execution permissions:

chmod +x dep

And we’re all set. Now we can use ./dep to run the tool. If you run that in the console, you’ll get the help screen of the tool.

You can also move the tool to some place like /usr/local/bin/dep to be able to use it globally throughout the system, but if the tool comes out with a new version and changes some functions, you’ll have to update all of the deployment scripts in order for it to work again. Setting it up on a per-project basis, like we just did, we’re safer.

Setting Up a New Deployment Script

PHP Deployer has a few pre-defined ‘recipes’ for the top PHP frameworks, which we can use to initialize our deployment script. To use one of the recipes, simply run:

./dep init

And choose your recipe.

However, for the purpose of learning how each of the functions in a recipe works, we’re going to create a deployment script from scratch. That being said, let’s create a new file called deploy.php with a single line, <?php.

Configuring a Server

In order to deploy something, we need to connect to a server first. PHP Deployer uses the host() function to create a server. A typical setup looks like this:

host('1.3.3.7')
    ->stage('production')
    ->user('www-data')
    ->identityFile('~/.ssh/id_rsa')
    ->set('deploy_path', '/var/www/codepicky.com');

We start with the IP or hostname that points to our server, then, we name it with the ->stage() function to know which server that is, because we might also want to deploy to another staging server in order to test the changes first, before deploying to the production server.

The ->user() function defines the user that we want to connect as on that server.

->identityFile() allows us to specify the SSH key that we want to use to connect to the server. This key needs to be installed on the server first.

->set() allows us to create host-based variables that we can use later in the deployment script. In the above example, we set deploy_path to /var/www/codepicky.com, because that’s where we want to deploy the app.

You can follow this pattern to define as many hosts as you like. Just remember to name them differently with the ->stage() function. We’re going to use that name when we want to run the script.

You can find all the supported functions of host(), here.

Setting Up a Task

A task allows us to perform a specific action on a host. We will schedule it later when to actually run, but for now, the only thing to understand is that a task would run something on the server. For example, if we wanted to log a message stating that we’re doing a new deployment, it’d look like this:

task('log-new-deploy', function () {
    run('echo "[' . date('Y-m-d H:i:s') . '] Deploying a new version of the app." >> /my/log/file');
});

That append something similar to:

[2018-09-05 12:00:00] Deploying a new version of the app.

To the file located at /my/log/file, on the specified host. So, PHP deployer connects to the host, then begins to execute the tasks.

The run() function simply runs the command on the host. There are other functions available, and you can find them here.

Variables

PHP Deployer allows us to set variables and use them through its functions, like run().

Setting a Variable

To set a global variable, i.e., that applies to all hosts, set it directly before the hosts’ definitions. For example, if we wanted to specify the branch name of the repository that we want to pull the code from, we’d do:

set('branch', 'master');

Using a Variable

If we want to use that variable inside run, we can, using the {{branch}} syntax. PHP deployer knows that you’re referring to a variable if you’re using {{ }}, so it will replace it with the variable’s content. E.g.:

write('We\'re going to pull the code from branch: {{branch}}');

This would write to the console, when we deploy our script, the following:

We're going to pull the code from branch: master

write() is just another PHP deployer function, like run(), that can be used to output messages to the console while executing the deployment script.

Setting a Variable on a Specific Host

We can assign or change the value of a variable if we use the ->set() function on a host() object. To understand how this would be useful, consider the following example:

set('branch', 'master');

host('1.3.3.7')
    ->stage('production')
    // ...
    ;

host('1.3.3.8')
    ->stage('staging')
    // ...
    ->set('branch', 'develop');

We’re setting the branch to master initially, but on the host called “staging”, we’re changing the value to develop.

PHP Deployer will detect, based on the host where we’re deploying the new code, which value to use.

Setting Up an Inline Task

If we have a task that only has a single run() command inside, there’s a shortcut for it, we can use:

task('log-new-deploy', 'echo "[' . date('Y-m-d H:i:s') . '] Deploying a new version of the app." >> /my/log/file');

This is equivalent with our task from before but without the callable function, however, because this is actually a string and the previous example was a callback, the date() output will vary.

Setting Up a Chain of Tasks

Because of how PHP Deployer works, we could, in theory, write our entire set of commands from a single task. But that would make the script messy, harder to understand and harder to debug. Instead, what we should do is create a task for each set of instructions of the same kind.

For example, if we want to change the chmod of some files, we should do it in a task. If then we want to compile the assets of our projects, we should do it in another.

This keeps the overall script a little bit more modular and it allows us to reuse parts of it on other projects. Not to mention that if a task fails during deployment, we can see which one failed, making our debug process easier.

To setup a chain of tasks, we setup another task which runs the others in the specified order, like so:

task('create-new-release', function () {
    // ...
});

task('create-symlinks', function () {
    // ...
});

task('compile-assets', function () {
    // ...
});

task('deploy', [
    'create-new-release',
    'create-symlinks',
    'compile-assets',
]);

Notice the deploy task which calls the other into action.

Deploying

To actually deploy something, i.e., execute the tasks on a host, after we’ve setup our script, we have the following options.

Run a Single Task on All Hosts

dep compile-assets

This would connect to all the hosts, one by one, and run the compile-assets task.

Run a Single Task on a Specific Host

dep compile-assets production

This assumes that we have a host defined with the name of production. It will connect to that host and run the task.

Running Multiple Tasks

To run multiple tasks, you need to create a task that calls other tasks, like we did above when we’ve set up a chain of tasks.

task('deploy', [
    'create-new-release',
    'create-symlinks',
    'compile-assets',
]);

Then, you simply run that as you would any other task, either on all hosts or on a specific one:

dep deploy
# OR
dep deploy staging

PHP Deployer Documentation

The official documentation can be found here, and I encourage you to go check it out and look at all the functions available.

Real World Example

Now that we’ve seen how to create hosts and tasks, let’s see how a real world example would look like. I have one of my own setups from a Laravel project. Let’s see how it all looks and then we’ll examine each of the tasks.

<?php

$startTime = microtime(true);

host('1.3.3.7')
    ->stage('production')
    ->user('www-data')
    ->identityFile('~/.ssh/id_rsa')
    ->set('branch', 'master')
    ->set('deploy_path', '/var/www/site.com');

host('1.3.3.8')
    ->stage('staging')
    ->user('deployer')
    ->identityFile('~/.ssh/id_rsa')
    ->set('branch', 'develop')
    ->set('deploy_path', '/home/sites/site.com');

set('releases_list', function () {
    return explode("\n", run('ls -dt {{deploy_path}}/releases/*'));
});

set('repository', '[email protected]:username/site.com.git');
set('keep_releases', 10);

task('confirm', function () {
    if (! askConfirmation('Are you sure you want to deploy to production?')) {
        write('Ok, quitting.');
        die;
    }
})->onStage('production');

task('create:release', function () {
    $i = 0;

    do {
        $releasePath = '{{deploy_path}}/releases/' . date('m_d_H_i_') . $i++;
    } while (run("if [ -d $releasePath ]; then echo exists; fi;") == 'exists');

    run("mkdir $releasePath");
    set('release_path', $releasePath);

    writeln("Release path: $releasePath");
});

task('update:code', function () {
    run("git clone -b {{branch}} -q --depth 1 {{repository}} {{release_path}}");
});

task('create:symlinks', function () {
    // Link .env.
    run("ln -nfs {{deploy_path}}/static/.env {{release_path}}");

    // Link storage.
    run("ln -nfs {{deploy_path}}/static/storage {{release_path}}");

    // Link vendor.
    run("ln -nfs {{deploy_path}}/static/vendor {{release_path}}");
});

task('update:vendors', function () {
    cd('{{release_path}}');
    writeln('<info>  Updating npm</info>');
    run('npm-cache install npm --no-dev');

    writeln('<info>  Updating composer</info>');
    run('composer install --no-dev');
});

task('update:permissions', function () {
    run('chmod -R a+w {{release_path}}/bootstrap/cache');
    run('chown -R {{user}}:{{user}} {{release_path}} -h');
});

task('compile:assets', function () {
    cd('{{release_path}}');
    run('npm run prod');
    run('rm -rf {{release_path}}/node_modules');
});

task('optimize', function () {
    run('php {{release_path}}/artisan cache:clear');
    run('php {{release_path}}/artisan view:clear');
    run('php {{release_path}}/artisan config:clear');
    run('php {{release_path}}/artisan config:cache');
});

task('site:down', function () {
    writeln(sprintf('<info>%s</info>', run('php {{release_path}}/artisan down')));
});

task('migrate:db', function () {
    writeln(sprintf('  <info>%s</info>', run('php {{release_path}}/artisan migrate --force --no-interaction')));
});

task('update:release_symlink', function () {
    run('cd {{deploy_path}} && if [ -e live ]; then rm live; fi');
    run('cd {{deploy_path}} && if [ -h live ]; then rm live; fi');

    run('ln -nfs {{release_path}} {{deploy_path}}/live');
});

task('site:up', function () {
    writeln(sprintf('  <info>%s</info>', run('php {{deploy_path}}/live/artisan up')));
});

task('clear:opcache', function(){
    run('cachetool opcache:reset --fcgi=/var/run/php/php7.2-fpm.sock');
});

task('cleanup', function () {
    $releases = get('releases_list');
    $keep = get('keep_releases');

    while ($keep-- > 0) {
        array_shift($releases);
    }

    foreach ($releases as $release) {
        run("rm -rf $release");
    }
});

task('notify:done', function () use ($startTime) {
    $seconds = intval(microtime(true) - $startTime);
    $minutes = substr('0' . intval($seconds / 60), -2);
    $seconds %= 60;
    $seconds = substr('0' . $seconds, -2);

    shell_exec("osascript -e 'display notification \"It took: $minutes:$seconds\" with title \"Deploy Finished\"'");
    shell_exec('say deployment finished');
});

task('rollback', function () {
    $releases = get('releases_list');

    if (isset($releases[1])) {
        writeln(sprintf('<error>%s</error>', run('php {{deploy_path}}/live/artisan down')));

        $releaseDir = $releases[1];
        run("ln -nfs $releaseDir {{deploy_path}}/live");
        run("rm -rf {$releases[0]}");

        writeln("Rollback to `{$releases[1]}` release was successful.");
        writeln(sprintf('  <error>%s</error>', run("php {{deploy_path}}/live/artisan up")));
    } else {
        writeln('  <comment>No more releases you can revert to.</comment>');
    }
});

task('deploy', [
    'confirm',
    'create:release',
    'update:code',
    'create:symlinks',
    'update:vendors',
    'update:permissions',
    'compile:assets',
    'optimize',
    'site:down',
    'migrate:db',
    'update:release_symlink',
    'site:up',
    'clear:opcache',
    'cleanup',
    'notify:done'
]);

The $startTime variable is used to track how much the deployment took to execute, this is not a PHP Deployer feature, it’s just something I like to include because I like to know how much a deployment takes. We’re going to use this variable again after all the tasks finished executing.

The following two blocks define the hosts that I can deploy to.

host('1.3.3.7')
    ->stage('production')
    ->user('www-data')
    ->identityFile('~/.ssh/id_rsa')
    ->set('branch', 'master')
    ->set('deploy_path', '/var/www/site.com');

host('1.3.3.8')
    ->stage('staging')
    ->user('deployer')
    ->identityFile('~/.ssh/id_rsa')
    ->set('branch', 'develop')
    ->set('deploy_path', '/home/sites/site.com');

The first one is the production environment, and the second, the staging environment, where I deploy my develop branch to test out new features. These are features that either need to be manually tested or whom I need to see how they behave with the staging database that mirrors the production one. This is for those rare cases where we just can mirror something locally which exists in production.

Notice that for each host I’m setting up different values for the branch and deploy_path variables. That’s why those variables aren’t globally set.

The next block:

set('releases_list', function () {
    return explode("\n", run('ls -dt {{deploy_path}}/releases/*'));
});

Defines a more complex variable, containing all the previous releases which I did. A release is a deployment, and I’m keeping multiple because I want to be able to roll back to an older one, if anything goes bad after a specific deployment. This variable is also used to count the releases, in order to delete older ones, if the total number of releases exceeds the amount that I want to keep. I’m also ordering them by the time they were created, to keep only the most recent ones.

Next, I’m setting 2 global variables with how many releases I want to keep and the repo’s SSH address.

set('repository', '[email protected]:username/site.com.git');
set('keep_releases', 10);

After that, I’m creating a task that would kill the script if I answer negatively to the given question.

task('confirm', function () {
    if (! askConfirmation('Are you sure you want to deploy to production?')) {
        write('Ok, quitting.');
        die;
    }
})->onStage('production');

This is applied only on the production host, and it’s added because I want to make sure that I really want to deploy to production by adding this second layer of protection against deploying by mistake.

This is how it works:

$ ./dep deploy production
➤ Executing task confirm
Are you sure you want to deploy to production? [y/N]   <I pressed Enter here>
Ok, quitting.

Because I didn’t confirm, the script stopped before even touching the production server.

The next task creates a new release location.

task('create:release', function () {
    $i = 0;

    do {
        $releasePath = '{{deploy_path}}/releases/' . date('m_d_H_i_') . $i++;
    } while (run("if [ -d $releasePath ]; then echo exists; fi;") == 'exists');

    run("mkdir $releasePath");
    set('release_path', $releasePath);

    writeln("Release path: $releasePath");
});

First, it tries to come up with a directory name that is unique, then it creates the directory and sets a global variable called release_path to the directory’s value.

Notice that we can set up global variables within other tasks, if they are dynamic like in my example.

The last line writes this variable’s value to the console, because I want to know where the new release is going to, just in case I need that information for debugging, if the deployment fails.

The next task is cloning the new code on the new release location.

task('update:code', function () {
    run("git clone -b {{branch}} -q --depth 1 {{repository}} {{release_path}}");
});

Next, I’m creating the necessary symlinks to some static files that belong only to the host I’m deploying on.

task('create:symlinks', function () {
    // Link .env.
    run("ln -nfs {{deploy_path}}/static/.env {{release_path}}");

    // Link storage.
    run("ln -nfs {{deploy_path}}/static/storage {{release_path}}");

    // Link vendor.
    run("ln -nfs {{deploy_path}}/static/vendor {{release_path}}");
});

The .env file contains all the variables such as the database connection values, Redis connection, caching method, app environment, and so on. For security reasons, these things are not to be included in the Git versioning, so I’m creating a symlink to them from a pre-defined static location. Also, these variables may vary from environment to environment, and this is another reason not the include them in the repo.

The storage directory is a Laravel specific directory that holds dynamic files like cache, framework or logs files. This is again something that’s going to be environment-specific, so I’m keeping it in a predefined location and creating a symlink every time I do a new deploy, because the app relies on it and it’s not in the repo.

The vendor directory is use by Composer to autoload all the project dependencies, including Laravel itself, and because it’s managed by Composer, I can keep it in a static location and simply install the dependencies based on the composer.lock file.

Note: Creating symlinks doesn’t only allow you to hide configuration files from the repo, like the .env file, but it also allows you to keep things like the vendor directory from the repo, saving space.

The next task install the dependencies based on the lock files.

task('update:vendors', function () {
    cd('{{release_path}}');

    writeln('<info>  Updating npm</info>');
    run('npm-cache install npm --no-dev');

    writeln('<info>  Updating composer</info>');
    run('composer install --no-dev');
});

The lock files are changed only when I update dependencies through composer or npm. And to keep the same version of dependencies on all hosts, I only use the update commands locally to generate new lock files. Then I use the install commands on the hosts, which will install the same versions of dependencies on the hosts like I have them locally.

This makes sure that I don’t use different versions of dependencies on production or staging than I do locally. It helps me stay away from debugging nightmares, like a library or package not working on staging / production but working locally.

The next task updates some permissions on some files.

task('update:permissions', function () {
    run('chmod -R a+w {{release_path}}/bootstrap/cache');
    run('chown -R {{user}}:{{user}} {{release_path}} -h');
});

Note: The {{user}} variable is not configured using set();, rather it’s a variable set by PHP Deployer when you set the ->user() on a host.

The next task compiles all the JS and CSS assets, and then it removes node_modules because it’s not required anymore, in order to save space.

task('compile:assets', function () {
    cd('{{release_path}}');
    run('npm run prod');
    run('rm -rf {{release_path}}/node_modules');
});

The next task is using some Laravel specific tools to optimize (cache) some of the configuration files and views.

task('optimize', function () {
    run('php {{release_path}}/artisan cache:clear');
    run('php {{release_path}}/artisan view:clear');
    run('php {{release_path}}/artisan config:clear');
    run('php {{release_path}}/artisan config:cache');
});

The site:down task is putting the app in maintenance mode.

task('site:up', function () {
    writeln(sprintf('  <info>%s</info>', run('php {{deploy_path}}/live/artisan up')));
});

I’m using it because I want to update the database with the new migrations (alterations), if any.

The next task is applying the database changes using Laravel’s tool.

task('migrate:db', function () {
    writeln(sprintf('  <info>%s</info>', run('php {{release_path}}/artisan migrate --force --no-interaction')));
});

It’s also writing the output to the console, because some times those fail and I want to see the error message.

Next, I’m updating the live symlink to the new release path.

task('update:release_symlink', function () {
    run('cd {{deploy_path}} && if [ -e live ]; then rm live; fi');
    run('cd {{deploy_path}} && if [ -h live ]; then rm live; fi');

    run('ln -nfs {{release_path}} {{deploy_path}}/live');
});

Note that until this stage, no task has messed with the application code that’s actually live and being used even while I’m deploying a new version of the code. This gives me the flexibility to update all my code, compile all the assets, reset the caches, restart queues, if I have any, and so on, without actually touching the code that’s being used. The live symlink is the location that my VHost points at.

This is the power of using multiple release directories. When I’m done setting everything up, I’m just changing the symlink and I have a new release up and running.

The next task removes the maintenance mode message and restores it live.

task('site:up', function () {
    writeln(sprintf('  <info>%s</info>', run('php {{deploy_path}}/live/artisan up')));
});

This is used after I’ve switched my live symlink.

The next task clears OPCache.

task('clear:opcache', function(){
    run('cachetool opcache:reset --fcgi=/var/run/php/php7.2-fpm.sock');
});

This allows the new code to be picked up by new visitors.

Next, I’m removing old releases that I don’t want to keep anymore.

task('cleanup', function () {
    $releases = get('releases_list');
    $keep = get('keep_releases');

    while ($keep-- > 0) {
        array_shift($releases);
    }

    foreach ($releases as $release) {
        run("rm -rf $release");
    }
});

The notify:done task simply calculates the time that took for all tasks to execute and displays a macOS notification with the time.

task('notify:done', function () use ($startTime) {
    $seconds = intval(microtime(true) - $startTime);
    $minutes = substr('0' . intval($seconds / 60), -2);
    $seconds %= 60;
    $seconds = substr('0' . $seconds, -2);

    shell_exec("osascript -e 'display notification \"It took: $minutes:$seconds\" with title \"Deploy Finished\"'");
    shell_exec('say deployment finished');
});

Note that shell_exec executes a command like run(), but on my local computer, where I run ./dep, and not on the host, that’s why I don’t use run() here.

The rollback task is designed to switch back to an older release. I can use this task if a deployment fails. This way I don’t have to check what went wrong and have the site down for an undetermined amount of time, I can just run one command to rollback, and then I can focus on debugging, and the app stays live, and continues to be able to be used in the meantime.

task('rollback', function () {
    $releases = get('releases_list');

    if (isset($releases[1])) {
        writeln(sprintf('<error>%s</error>', run('php {{deploy_path}}/live/artisan down')));

        $releaseDir = $releases[1];
        run("ln -nfs $releaseDir {{deploy_path}}/live");
        run("rm -rf {$releases[0]}");

        writeln("Rollback to `{$releases[1]}` release was successful.");
        writeln(sprintf('  <error>%s</error>', run("php {{deploy_path}}/live/artisan up")));
    } else {
        writeln('  <comment>No more releases you can revert to.</comment>');
    }
});

And finally, there’s the task that ties all of the deployment tasks together, in the order that makes sense.

task('deploy', [
    'confirm',
    'create:release',
    'update:code',
    'create:symlinks',
    'update:vendors',
    'update:permissions',
    'compile:assets',
    'optimize',
    'site:down',
    'migrate:db',
    'update:release_symlink',
    'site:up',
    'clear:opcache',
    'cleanup',
    'notify:done'
]);

The tasks will be executed in the defined order and the output, if successful, would look like this:

$ ./dep deploy production
➤ Executing task confirm
Are you sure you want to deploy to production? [y/N] y
✔ Ok
➤ Executing task create:release
Release path: /home/sites/site.com/releases/09_05_11_01_0
✔ Ok
✔ Executing task update:code
✔ Executing task create:symlinks
➤ Executing task update:vendors
  Updating npm
  Updating composer
✔ Ok
✔ Executing task update:permissions
✔ Executing task compile:assets
➤ Executing task site:down
Application is now in maintenance mode.
✔ Ok
✔ Executing task optimize
➤ Executing task migrate:db
  Nothing to migrate.
✔ Ok
✔ Executing task update:release_symlink
➤ Executing task site:up
  Application is now live.
✔ Ok
✔ Executing task clear:opcache
✔ Executing task cleanup
✔ Executing task notify:done

Conclusion

Now that we have the script. Every time we push something to the repository, we can run ./dep deploy production and push the code to production with a single line of code. Hopefully, you can see how helpful this is when you have a complicated deployment experience like the one in my real world example.