Project:Toolchain/time64 migration

From Gentoo Wiki
Jump to:navigation Jump to:search

Traditional time_t is 32-bit and is subject to the Y2038 problem. glibc 2.34 onwards supports opting-in to a 64-bit time_t which allows handling time beyond Y2038.

This page discusses how to handle the migration for (32-bit) systems affected by this, including non-glibc environments. 64-bit systems are not affected.

Mixing 32-bit time_t and 64-bit time_t is a recipe for disaster where time_t is exposed in a public API and the consumer/provider are mismatched e.g. net-misc/wget with net-libs/gnutls in bug #828001. This is actually a problem with Large File Support (LFS) too but it seems to happen far less frequently there, or it was minimised by LARGEFILE_SOURCE allowing dual ABI in packages.

For 32-bit arches, we need to coordinate flipping glibc systems over to use 64-bit time_t using CPPFLAGS="-D_TIME_BITS=64" (note that not all packages may support/respect CPPFLAGS). Again, 64-bit arches are already fine.

Problem

Sneaky bits:

  1. Libraries using time_t directly or indirectly needs to be switched at the same time. Applications using any such libraries must be switched at the same time too.
  2. Software might be using time_t (and e.g. off_t, needing Large File Support (LFS)) without it being obvious. For example, it might be handling a stat struct which contains both off_t and st_atim.
  3. Software needs to provide a way to stick with 32-bit time_t for now to avoid making the problem worse, but allow optionally using 64-bit time_t
  4. We likely have to complete Modern C porting first to remove any instances of -Wimplicit-function-declaration otherwise the redirects in glibc for e.g. time->time64 won't actually work. We might be able to avoid this with some hacks like removing the 32-bit symbols entirely from glibc so we get linker errors.


The core issues are:

  1. Auditing packages piecemeal is an unfeasibly large amount of work
  2. glibc has no mechanism to hard-switch yet (and discussion has stalled [1])
  3. Acting alone in a hard-switch (or at all, really) risks incompatibility with other distributions, as all of this obviously affects ABI
  4. time_t or time_t-dependent types in a library's API is not always straightforward to see

Notes

Misc other notes:

Plans

TODO

  • Push for an autoconf release which both fixes some Modern C porting issues and contains the needed macros for Y2038 support. The needed fixes are already in git.
  • Agree on a way with the upstream glibc + distro community on how to hard-switch glibc.
  • Coordinate hard-switching once (plenty of!) testing has been done
  • Achieve consensus on new tuple names for time64'd ABIs


Gentoo-specific TODO items, but they may affect other distros too:

musl

Gentoo specific notes:

  • Need to check how/if at all we handled it in Gentoo
  • Check whether there's more work to be done to ensure user systems are in a consistent state
  • 1.2.0 was added on 2020-02-25 and 1.2.1 (first of 1.2.x) was stabled on 2020-08-20.
    • Possible user systems are in an inconsistent state given we've not rebuilt / news item'd for this?

LFS

We may want to compare with how Large File Support (LFS) / FILE_OFFSET_BITS has been handled. Much more mature support overall in the ecosystem, although it wasn't generally treated like the ABI break it was.

Misc notes:

  • For glibc, time64 needs LFS (example of why: see e.g. stat struct).
  • Meson already adds this as required
  • autoconf has AC_SYS_LARGEFILE which things should be using (but may not be!)


Gentoo specific notes:

  • bug #471102 tracks progress in Gentoo overall.
  • We have append-lfs-flags (and filter-lfs-flags) in flag-o-matic.eclass

Possible solutions

Rebuild after profile change

Note
Not everyone is using autoconf or even passing flags correctly. A safer, though probably more controversial way would be to force changed defaults in gcc and glibc.
  • We need to agree on names for these new ABIs, ideally using new tuple names, and do so with other distributions
  • In current profiles, set gl_cv_type_time_t_bits_macro=no to prevent gnulib autoconf macro from automatically adding -D_TIME_BITS=64 (done, note that gnulib reverted the change (see above))
    • Ditto for ac_cv_type_time_t_y2038?
  • In new profile, add CPPFLAGS="-D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64" (do CFLAGS instead given insufficient packages respect CPPFLAGS).
  • Advise users to adjust custom CPPFLAGS if necessary, and rebuild @world after switching profiles.

Some more thoughts / suggestions:

  • Use as ABI suffix t64, e.g., i686-linux-gnut64
    • This will require fixing some packages (notably, ncurses and clang)
      • clang patch; but we need to figure out backports (Triple changes affect ABI)
    • Both CHOST and CHOST_x86 need to be patched (else e.g. clang uses the wrong tool prefix)
  • We already have autoconf magic in current profiles, that needs to be overridden then (where?)
  • Unfortunately, even default CFLAGS may not be good enough.
  • A safer transition route is possible by copying existing libraries to libt32 and adding RUNPATHs to all existing executables

Revbump ebuilds

Warning
This option is considered far too much work and doesn't achieve ABI compatibility with other distros, so is unlikely to be pursued.
  • In packages where time_t is utilized, revbump and add append-cppflags -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64.
  • This doesn't ensure that libraries and their consumers get rebuilt together, but it should work out in the end.
  • Presumably, we'd have a large package.mask for this.

time64 USE flag

Warning
Not likely to be pursued for the same reasons as "Revbump ebuilds".
  • Add USE=time64 masked everywhere
  • May need lots of eautoreconf (after patching autoconf)
  • Lots of --enable-year2038 added to econfs
  • Unmask on to-be-created features/time64 profile
  • Allows expressing dependencies on time64ness to help rebuild order
  • Allows adding machinery to ebuilds at our own pace, then create new profiles when ready
  • Doesn't cause churn to 64-bit arches

Summary

Solution Consistency during migration Minimises disruption to bystander/unaffected users (64-bit arches) Amount of manual labor needed Avoids flag day External or historical binary compatibility
Rebuild after profile change (just changes cache variables) Provides no guarantees of correct rebuild order and may require several rebuilds to ensure consistent ABI. The change is invisible to these users as they already have 64-bit support and won't be changing profiles. Not much at all wrt packaging minutiae (of course, there's still bug fixing.) No No
New profiles, new CHOST Provides no guarantees of correct rebuild order and may require several rebuilds to ensure consistent ABI. The change is invisible to these users as they already have 64-bit support and won't be changing profiles. Not much at all wrt packaging minutiae (of course, there's still bug fixing). Need to make sure the new CHOST doesnt break anything. Users can migrate at leisure CHOST change cleanly separates ABI
Revbump ebuilds to add time64 CFLAGS Provides no guarantees of correct rebuild order and may require several rebuilds to ensure consistent ABI. All users, even those unaffected, have to rebuild. Lots of ebuild churn. We can make the changes gradually, by traversing leaf packages, and building up to libraries. No
time64 USE flag If dependencies are expressed correctly, we can ensure libraries gain time64 support before consumers, and we can use use= deps to enforce correctness. The USE flag will be masked on irrelevant profiles so only --newuse (-N) users will be affected. A lot of work modifying many ebuilds, especially if we're going to get the dependencies right, which is the advantage from this option... We'd need a flag day to eventually remove the time64 flag, but not to add it. No

See also

External resources

References

  1. ↑ Sam James. On time64 and Large File Support, libc-alpha mailing list, November 11th, 2022. Retrieved on January 19th, 2023.
  2. ↑ Kaz Kylheku. Best practices in regard to -D_TIME_BITS=64, libc-alpha mailing list, January 5th, 2022. Retrieved on January 19th, 2023.
  3. ↑ Rich Felker. musl time64 Release Notes, musl libc project, February 20th, 2020. Retrieved on January 19th, 2023.
Retrieved from "https://wiki.gentoo.org/index.php?title=Project:Toolchain/time64_migration&oldid=1315695"