Composer packaging

This article has been flagged for not conforming to the wiki guidelines. Please help Gentoo out by starting fixing things.

Introduction

What is Composer?

Composer is a bundling dependency manager for PHP projects. It's similar to Bundler (for Ruby) or npm (for Javascript). It automates the process of downloading the needed project's dependencies. Recently it has become extremely popular and is used by major projects like PHPUnit and Drush.

The main repository for Composer packages is Packagist.

How does it work?

Each project that can be installed with Composer has a composer.json file that lists some information about it: the project name, its description, its dependencies, and so on. When Composer is invoked, it goes through the list of dependencies and downloads them all, storing them into the project's vendor directory. Afterwards, it generates a PHP class autoloader vendor/autoload.php that will automatically load any of the classes in the project or its dependencies.

Packaging hurdles

Most Composer packages use the autoloader (generated by Composer) to find and include() their class files. As a result, the require() statements typically used to include one PHP file into another are often missing. In that case, the package will not work out-of-the-box -- or in fact at all -- until an autoloader is created for it.

Composer's build system is incompatible with the way Gentoo's package managers work, so Composer cannot be used to generate the necessary autoloaders. For a long time, this obstacle prevented Gentoo system from packaging anything that relied heavily on Composer. Fortunately, there is now workaround this issue.

Autoloading

The autoloaders that Composer generates support the PSR-0, PSR-4, and classmap formats. The generation of those autoloaders is the main obstacle to packaging Composer projects, and the creation of the Fedora Autoloader is the breakthrough that makes it possible.

The Fedora autoloader library provides a few functions -- addPsr4, addPsr0, and addClassMap -- that can generate class autoloaders for a directory tree. Moreover, it can build one big autoloader from a bunch of existing ones, allowing you to create an autoloader that works for an application and all of its dependencies (that's how Composer works). For example, the following file creates an autoloader for the dev-php/psr-log package. The psr-log-autoload.php file is meant to be locates in the same directory as the library code, so we tell the autoloader to look in the current directory __DIR__ and to prefix the class names therein with Psr\Log:

FILE psr-log-autoload.php

<?php
if (!class_exists('Fedora\\Autoloader\\Autoload', false)) {
  require('/usr/share/php/Fedora/Autoloader/autoload.php');
}

\Fedora\Autoloader\Autoload::addPsr4('Psr\\Log\\', __DIR__);

Now, most people won't be using that package directly; instead, it will appear as a dependency of other PHP packages in the tree. In fact, it is used by dev-php/composer itself. Here's a simplified version of the composer-autoload.php file that not only generates an autoloader for the Composer namespace, but also pulls in the Psr\Log autoloader as well:

FILE composer-autoload.php

<?php
if (!class_exists('Fedora\\Autoloader\\Autoload', false)) {
    require('/usr/share/php/Fedora/Autoloader/autoload.php');
}

\Fedora\Autoloader\Autoload::addPsr4('Composer\\', __DIR__);

// Dependencies, there are more in the real autoloader.
\Fedora\Autoloader\Dependencies::required(array(
  '/usr/share/php/Psr/Log/autoload.php'
));

Naming conventions

Ebuilds

Gentoo's naming conventions and the ones for Composer/Packagist don't quite agree. When choosing a Gentoo name for a Packagist project, you should try to choose the simplest name that is unambiguous and agrees with the Composer/Packagist names. Any slashes in the name must be converted to hyphens, and most packages should wind up in the dev-php category, although that's not required. The idea is best illustrated with some examples.

The psr/log package, once the slash is replaced by a hyphen, becomes dev-php/psr-log. This closely matches what upstream uses, and what users will expect. A search for "psr" or "log" will turn up the correct package. The name dev-php/log would not be appropriate because it is too general.
The fedora/autoloader package becomes dev-php/fedora-autoloader, because dev-php/autoloader is too general.
The composer/composer package becomes simply dev-php/composer, because there is no ambiguity in the name, and -- let's face it -- dev-php/composer-composer would be stupid.

Namespacing

Since we choose where to install everything, the resulting class namespaces are technically at our mercy. However, whenever possible, you should retain the upstream namespaces. As an example, consider the dev-php/symfony-process package. Symfony is a large framework that contains a number of components, and the Process component is merely one piece of the puzzle. As a result, the upstream namespace used for it is Symfony\Component\Process, which does not quite agree with the "symfony-process" name in the ebuild. Nevertheless, if a user wants to use dev-php/symfony-process directly, the upstream documentation will instruct him to use their namespaces:

FILE symfony-process-example.php

<?php
use Symfony\Component\Process\Process;

$process = new Process('ls -lsa');
$process->run();

// etc.

We don't want to invalidate all of the existing documentation without a very good reason.

Dependencies

All Composer packages should have a composer.json file that lists its dependencies. The following snippets are taken from dev-php/composer itself. First, the require field lists all of the usual runtime requirements for the package. Most entries refer to other Composer/Packagist packages, but there's some special support for specifying the version of php itself that is required. You'll need to know Composer's version syntax in order to translate these into ebuild dependencies.

FILE composer.json

  "require": {
    "php": "^5.3.2 || ^7.0",
    "justinrainbow/json-schema": "^1.6 || ^2.0 || ^3.0 || ^4.0",
    "composer/ca-bundle": "^1.0",
    "composer/semver": "^1.0",
    "composer/spdx-licenses": "^1.0",
    "seld/jsonlint": "^1.4",
    "symfony/console": "^2.7 || ^3.0",
    "symfony/finder": "^2.7 || ^3.0",
    "symfony/process": "^2.7 || ^3.0",
    "symfony/filesystem": "^2.7 || ^3.0",
    "seld/phar-utils": "^1.0",
    "seld/cli-prompt": "^1.0",
    "psr/log": "^1.0"
  },

Another field, require-dev specifies the "developer" dependencies. These often include the things needed to build documentation or run tests. Often they'll be hidden behind USE=doc or USE=test in your ebuild.

FILE composer.json

  "require-dev": {
    "phpunit/phpunit": "^4.5 || ^5.0.5",
    "phpunit/phpunit-mock-objects": "^2.3 || ^3.0"
  },

The version bounds listed in the composer.json files are often too strict. Unless you have a good reason to believe them, a good general rule is to respect upstream's lower bounds, but to ignore any upper bounds.

Tests

The test suites for Composer packages present another problem. Before a package is installed, it has no autoloader, but both the main class hierarchy and the test class hierarchy usually need autoloaders to be able to run.

Often you can fix this by copying the package's autoload.php file into both the current and test directories. This works because that autoloader is based on "the current directory," i.e. wherever you stick the file. Below you'll find an example from dev-php/symfony-yaml. (If this doesn't work, you may have to get clever.)

FILE symfony-yaml-2.1.0.ebuild

src_prepare() {
    default
    if use test; then
        cp "${FILESDIR}"/autoload.php "${S}"/autoload-test.php || die
    fi
}

src_test() {
    phpunit --bootstrap "${S}"/autoload-test.php || die "test suite failed"
}