User:Leo3418/Hamcrest 2.x migration

This page explains the proposal to migrate packages in the Gentoo repository that use Hamcrest 1.x to Hamcrest 2.x.

Problem statement

Hamcrest released version 2.1 to provide the org.hamcrest JPMS module (referred to as "module" below), which other Java projects can use to integrate with the JPMS.

When another Java project depends on the org.hamcrest module, building the project requires Hamcrest 2.x, and only 2.x, to be in the class path.

Using Hamcrest 1.x in place of 2.x would trigger an error like the following:

assertj-core/src/main/java9/module-info.java:44: error: module not found: org.hamcrest
  requires static org.hamcrest;

Hamcrest 1.x must not be in the class path. Having both 1.x and 2.x in the class path would result in the following errors:

error: the unnamed module reads package org.hamcrest.internal from both hamcrest.core and org.hamcrest
assertj-core/src/main/java9/module-info.java:13: error: module org.assertj.core reads package org.hamcrest.internal from both org.hamcrest and hamcrest.core
module org.assertj.core {
^

Under the context of Gentoo Java packages, it is easy to ensure that Hamcrest 2.x is in the class path when a package needs it, but it is difficult to avoid having both 1.x and 2.x in the class path. A real example is AssertJ Core 3.24.2, which depends on both JUnit 4.13.2 and Hamcrest 2.2. JUnit 4.13.2 depends on Hamcrest 1.3. Therefore, the class path will contain both version 1.3 and version 2.2, leading to the aforementioned errors.

To resolve this problem, packages that depend on Hamcrest 1.x should be migrated to 2.x to satisfy the requirement that 2.x must be the exclusive Hamcrest version in the class path.

Feasibility

According to the Hamcrest upstream, the API of Hamcrest 2.x "has not changed since Hamcrest 1.3"^[1], so Hamcrest 2.x is supposed to be a compatible drop-in replacement for 1.3, or maybe even 1.x.

One exception, as the upstream noted, is the org.hamcrest.Factory annotation's removal. The upstream said that the annotation "should not be used in client code"^[1], so ideally, this should not be a problem. In reality, at least one known example of a Java project that uses the annotation exists, which is JUnit 4.13.2. Compiling JUnit 4.13.2 with Hamcrest 2.2 fails due to these errors:

user $

javac -source 1.8 -target 1.8 -d target/classes -encoding ISO-8859-1 -classpath /usr/share/hamcrest/lib/hamcrest.jar:/usr/share/hamcrest/lib/hamcrest.jar @sources.lst

junit4-r4.13.2/src/main/java/org/junit/internal/matchers/ThrowableCauseMatcher.java:4: error: cannot find symbol
import org.hamcrest.Factory;
                   ^
  symbol:   class Factory
  location: package org.hamcrest
junit4-r4.13.2/src/main/java/org/junit/internal/matchers/ThrowableMessageMatcher.java:4: error: cannot find symbol
import org.hamcrest.Factory;
                   ^
  symbol:   class Factory
  location: package org.hamcrest
junit4-r4.13.2/src/main/java/org/junit/internal/matchers/StacktracePrintingMatcher.java:4: error: cannot find symbol
import org.hamcrest.Factory;
                   ^
  symbol:   class Factory
  location: package org.hamcrest
junit4-r4.13.2/src/main/java/org/junit/internal/matchers/ThrowableCauseMatcher.java:46: error: cannot find symbol
    @Factory
     ^
  symbol:   class Factory
  location: class ThrowableCauseMatcher<T>
  where T is a type-variable:
    T extends Throwable declared in class ThrowableCauseMatcher
junit4-r4.13.2/src/main/java/org/junit/internal/matchers/ThrowableMessageMatcher.java:33: error: cannot find symbol
    @Factory
     ^
  symbol:   class Factory
  location: class ThrowableMessageMatcher<T>
  where T is a type-variable:
    T extends Throwable declared in class ThrowableMessageMatcher
junit4-r4.13.2/src/main/java/org/junit/internal/matchers/StacktracePrintingMatcher.java:42: error: cannot find symbol
    @Factory
     ^
  symbol:   class Factory
  location: class StacktracePrintingMatcher<T>
  where T is a type-variable:
    T extends Throwable declared in class StacktracePrintingMatcher
junit4-r4.13.2/src/main/java/org/junit/internal/matchers/StacktracePrintingMatcher.java:48: error: cannot find symbol
    @Factory
     ^
  symbol:   class Factory
  location: class StacktracePrintingMatcher<T>
  where T is a type-variable:
    T extends Throwable declared in class StacktracePrintingMatcher
junit4-r4.13.2/src/main/java/org/junit/matchers/JUnitMatchers.java:60: error: incompatible types: no instance(s) of type variable(s) U exist so that Matcher<Iterable<? extends U>> conforms to Matcher<Iterable<T>>
        return CoreMatchers.everyItem(elementMatcher);
                                     ^
  where U,T are type-variables:
    U extends Object declared in method <U>everyItem(Matcher<U>)
    T extends Object declared in method <T>everyItem(Matcher<T>)
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
8 errors

Therefore, the migration would require some Gentoo downstream changes to either Hamcrest 2.x or Java projects that depend on Hamcrest 1.x, so the issues pertaining to org.hamcrest.Factory can be resolved.

Other parties' reactions

This section covers the following things, which potentially affect whether the migration should be performed, and how the migration should be executed:

Whether the Hamcrest upstream has any plans to reinstate the org.hamcrest.Factory annotation
Whether those Java projects that depend on Hamcrest 1.x plan to upgrade to 2.x
Other distributions' resolution to this problem

Hamcrest upstream

The Hamcrest upstream consciously removed the org.hamcrest.Factory annotation before releasing 2.1 despite a related user inquiry, so they are unlikely to add the annotation back. If someone had requested Hamcrest to add the annotation back, it is reasonable to infer that the upstream would disregard the request and respond that the Java packages that use Hamcrest was the party that is responsible to resolve this issue and should eliminate any uses of the annotation.

https://github.com/hamcrest/JavaHamcrest/issues/224#issuecomment-446984317: A user commented that org.hamcrest.Factory was no longer available.
https://github.com/hamcrest/JavaHamcrest/issues/224#issuecomment-446989079: A Hamcrest project member replied with regards to the user's comment, saying "you shouldn't need it".

With regards to JUnit 4.13.2, the Hamcrest upstream provided an upgrade guide for Gradle and Maven projects that use JUnit 4.13.2. The guide is not applicable to Gentoo as Gentoo Java packages typically do not use Gradle or Maven to build a project, even if the project's build system is Gradle or Maven.

JUnit upstream

The JUnit 4 project issue tracker has a ticket for Hamcrest 2.2 upgrade, but the JUnit upstream has not made any visible progress on resolving the ticket yet.

Debian

Debian builds JUnit 4.13.2 with Hamcrest 2.2. They patch both JUnit and Hamcrest to fix build issues. In particular, the Hamcrest patch adds back the org.hamcrest.Factory annotation to Hamcrest 2.2.

Possible solutions

This section presents and evaluates different solutions to this problem that are applicable to Gentoo.

Add Auto-Module-Name to dev-java/hamcrest-core

Based on my experiment, defining Automatic-Module-Name: org.hamcrest in a Hamcrest 1.x JAR file is sufficient to fix the problem. The Automatic-Module-Name allows the JAR file to co-exist with Hamcrest 2.x in the class path without triggering the "the unnamed module reads package from both hamcrest.core and org.hamcrest" error.

To define the Automatic-Module-Name, add this line to the dev-java/hamcrest-core ebuild:

CODE Proposed addition to dev-java/hamcrest-core

# Although not added by upstream, this helps eliminate
# "module reads package from both hamcrest.core and org.hamcrest" errors when
# both this package and dev-java/hamcrest are in the compile-time classpath
JAVA_AUTOMATIC_MODULE_NAME="org.hamcrest"

Packages that depend on Hamcrest 1.x can still be built with Hamcrest 1.x. Therefore, this solution does not truly migrate those packages to Hamcrest 2.x; however, it is still sufficient for resolving the problem raised by Hamcrest 2.x.

The benefit of this solution is that it just requires the minimal change and does not apply any patches to any packages, so:

It should not require changing other Java packages that depend on Hamcrest 1.x.
It does not create extra maintenance burden due to any additional patches.

These are the possible sources of risk in this solution, though the likelihood and potential negative impact of each risk are still to be studied:

Setting Automatic-Module-Name for a JAR file, while the upstream does not set it
Having two JAR files that both bear the same Automatic-Module-Name and contain a lot of classes with the same name in the class path

Create a standalone package for org.hamcrest.Factory

This solution involves creating an ebuild that installs a JAR file with only the org.hamcrest.Factory annotation. When a Java package that needs the annotation is migrated to Hamcrest 2.x, it will depend on both Hamcrest 2.x and this ebuild.

FILE hamcrest-Factory-1.3.ebuildAn ebuild that installs a JAR file with only org.hamcrest.Factory

# Copyright 1999-2023 Gentoo Authors
# Distributed under the terms of the GNU General Public License v2

EAPI=8

JAVA_PKG_IUSE="doc source"
MAVEN_ID="org.hamcrest:hamcrest-core:1.3"

inherit java-pkg-2 java-pkg-simple

DESCRIPTION="Standalone package for the org.hamcrest.Factory annotation"
HOMEPAGE="https://wiki.g.o/wiki/No_homepage"
SRC_URI="https://github.com/hamcrest/JavaHamcrest/archive/hamcrest-java-${PV}.tar.gz"

LICENSE="BSD-2"
SLOT="0"
KEYWORDS="~amd64"

DEPEND=">=virtual/jdk-1.8:*"
RDEPEND=">=virtual/jre-1.8:*"

S="${WORKDIR}/JavaHamcrest-hamcrest-java-${PV}/hamcrest-core/src/main/java"

# The Automatic-Module-Name is still needed
# when this package is used with Hamcrest 2.x
JAVA_AUTOMATIC_MODULE_NAME="org.hamcrest"

JAVA_SRC_DIR="${T}/src"

src_prepare() {
	java-pkg-2_src_prepare
	mkdir -p "${JAVA_SRC_DIR}/org/hamcrest" || die
	cp -v org/hamcrest/Factory.java "${JAVA_SRC_DIR}/org/hamcrest" || die
}

CODE Example dependency for a Java package that needs org.hamcrest.Factory after the migration

CP_DEPEND="
	dev-java/hamcrest:0
	dev-java/hamcrest-Factory:0
"

The benefits of this solution are:

It does not require patching Hamcrest 2.x, so it does not have any potential side effect on packages that already depend on Hamcrest 2.x.
It might be possible to clean up Hamcrest 1.x from the Gentoo repository if the Hamcrest upstream's statement that Hamcrest 2.x does not have any API changes except org.hamcrest.Factory is accurate: this additional ebuild will enable migration of all existing Gentoo packages that depend on Hamcrest 1.x to 2.x because it adds back the only deleted part of the Hamcrest API, so no packages need to depend specifically on Hamcrest 1.x anymore.
It helps avoid the case where two class files for the same class name exist in the class path. The Automatic-Module-Name for dev-java/hamcrest-core approach does nothing with regards to this: it rather enables Hamcrest 1.x and 2.x to be in the class path at the same time. That said, the actual negative impact of having two class files with the same class name is still to be studied though.

The drawbacks of this solution are:

It still does not eliminate the need to patch packages that depend on Hamcrest 1.x. For instance, Debian still had to patch JUnit 4.13.2 itself to migrate it to Hamcrest 2.x.
The new ebuild still needs to set Automatic-Module-Name for its JAR file to allow the JAR file to co-exist with Hamcrest 2.x in the class path, so it still shares the same potential risks pertaining to Automatic-Module-Name with the previous solution.

Add org.hamcrest.Factory to Hamcrest 2.x

This is the same as Debian's Hamcrest 2.x migration strategy: patch both Hamcrest 2.x to add back the org.hamcrest.Factory annotation and packages that depend on Hamcrest 1.x as needed to migrate them to Hamcrest 2.x.

The benefits of this solution are:

Debian has already tested it since they have been using it for a while.
It would not trigger any new issues pertaining to Automatic-Module-Name because it does not involve adding Automatic-Module-Name to any package at all.

The drawbacks of this solution are:

It still does not eliminate the need to patch packages that depend on Hamcrest 1.x – same as the previous solution.
The Hamcrest 2.x version that Gentoo ships will no longer be the vanilla Hamcrest 2.x built from the verbatim upstream source.

Summary

Solution	Setting downstream `Automatic-Module-Name`?	Patching packages that depend on Hamcrest 1.x?	Patching Hamcrest 2.x?
Add `Auto-Module-Name` to dev-java/hamcrest-core	Needed	Not needed	Not needed
Create a standalone package for org.hamcrest.Factory	Needed	Needed	Not needed
Add org.hamcrest.Factory to Hamcrest 2.x	Not needed	Needed	Needed

References

↑ ^1.0 ^1.1 Hamcrest 2.1 release notes

[hamcrest-2.1-1] 1.0 ^1.1 Hamcrest 2.1 release notes

[1]