Compare commits

..

25 Commits

Author SHA1 Message Date
Jia Tan fd1b975b78 Bump version and soname for 5.6.1. 2024-03-09 11:42:50 +08:00
Jia Tan a2cda57249 Add NEWS for 5.6.1 2024-03-09 11:27:27 +08:00
Jia Tan 8583c60211 Translations: Add missing --riscv option to man page translations. 2024-03-09 10:43:20 +08:00
Jia Tan 74b138d2a6 Tests: Update two test files.
The original files were generated with random local to my machine.
To better reproduce these files in the future, a constant seed was used
to recreate these files.
2024-03-09 10:18:29 +08:00
Jia Tan 3ec6dfd656 Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
The first stream in this file is valid, so this tests that xz properly
stops after decompressing it.
2024-03-09 10:08:32 +08:00
Jia Tan a67dcce610 Tests: Update RISC-V test files.
This increases code coverage and tests for possible shifting bugs.
2024-03-09 10:05:32 +08:00
Jia Tan 058337b0f1 liblzma: Fix typos in crc32_fast.c and crc64_fast.c. 2024-03-09 09:52:32 +08:00
Jia Tan cd5de9c1bb Tests: Replace HAVE_MICROLZMA usage in CMake and Autotools builds.
This reverts commit adaacafde6.
2024-03-09 09:49:55 +08:00
Jia Tan 651a1545c8 liblzma: Fix false Valgrind error report with GCC.
With GCC and a certain combination of flags, Valgrind will falsely
trigger an invalid write. This appears to be due to the omission of
instructions to properly save, set up, and restore the frame pointer.

The IFUNC resolver is a leaf function since it only calls a function
that is inlined. So sometimes GCC omits the frame pointer instructions
in the resolver unless this optimization is explictly disabled.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
2024-03-09 09:20:57 +08:00
Lasse Collin 6e97b299f1 liblzma: Fix a typo in a comment in the RISC-V filter. 2024-03-05 23:21:26 +02:00
Jia Tan 4e1c97052b liblzma: Use attribute no_profile_instrument_function with ifunc.
Thanks to Sam James for determining this was the attribute needed to
workaround the GCC bug and for his version of the patch in Gentoo.
2024-03-05 00:34:46 +08:00
Jia Tan ed957d3942 Build: Require attribute no_profile_instrument_function for ifunc usage.
Using __attribute__((__no_profile_instrument_function__)) on the ifunc
resolver works around a bug in GCC -fprofile-generate:
it adds profiling code even to ifunc resolvers which can make
the ifunc resolver crash at program startup. This attribute
was not introduced until GCC 7 and Clang 13, so ifunc won't
be used with prior versions of these compilers.

This bug was brought to our attention by:

    https://bugs.gentoo.org/925415

And was reported to upstream GCC by:

    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11411
2024-03-05 00:27:31 +08:00
Lasse Collin e98ddaf85a liblzma: Fix a comment in the RISC-V filter. 2024-03-04 19:23:18 +02:00
Lasse Collin 319cec142f CMake: Warn if translated man pages are missing. 2024-02-29 16:35:52 +02:00
Lasse Collin 46c3e113d8 CMake: Warn if gettext tools and pre-created .gmo files are missing.
It's only done with CMake >= 3.20 and if library support
for translation was already found.

Sort of fixes: https://github.com/tukaani-project/xz/issues/82
2024-02-29 16:35:52 +02:00
Lasse Collin 86bec8334b xz: Add comments. 2024-02-28 18:33:34 +02:00
Jia Tan 5c91b454c2 xz: Change logging level for thread reduction to highest verbosity only.
Now that multi threaded encoding is the default, users do not need to
see a warning message everytime the number of threads is reduced. On
some machines, this could happen very often. It is not unreasonable for
users to need to set double verbose mode to see this kind of
information.

To see these warning messages -vv or --verbose --verbose must be passed
to set xz into the highest possible verbosity mode.

These warnings had caused automated testing frameworks to fail when they
expected no output to stderr.

Thanks to Sebastian Andrzej Siewior for reporting this and for the
initial version of the patch.
2024-02-28 18:31:04 +02:00
Lasse Collin d0e57b2f15 Fix sorting in THANKS. 2024-02-28 18:31:04 +02:00
Jia Tan d416be55ac Update THANKS. 2024-02-28 18:31:04 +02:00
Chien Wong f06b33edd2 xz: Add missing RISC-V on the filter list in the man page
Signed-off-by: Chien Wong <m@xv97.com>
2024-02-28 18:31:04 +02:00
Jia Tan a100f9111c Build: Fix Linux Landlock feature test in Autotools and CMake builds.
The previous Linux Landlock feature test assumed that having the
linux/landlock.h header file was enough. The new feature tests also
requires that prctl() and the required Landlock system calls are
supported.
2024-02-28 18:31:04 +02:00
Jia Tan d85efdc891 Tests: Add test_microlzma to .gitignore and CMakeLists.txt. 2024-02-28 18:31:04 +02:00
Jia Tan 42ee425673 Tests: Correct license header in test_microlzma.c. 2024-02-28 18:31:04 +02:00
Jia Tan c83349dfd9 Fix typos in NEWS and CMakeLists. 2024-02-28 18:31:04 +02:00
Jia Tan 2d7d862e3f Bump version and soname for 5.6.0. 2024-02-24 15:55:08 +08:00
11 changed files with 82 additions and 197 deletions

8
.github/SECURITY.md vendored
View File

@ -16,7 +16,13 @@ the chance that the exploit will be used before a patch is released.
You may submit a report by emailing us at You may submit a report by emailing us at
[xz@tukaani.org](mailto:xz@tukaani.org), or through [xz@tukaani.org](mailto:xz@tukaani.org), or through
[Security Advisories](https://github.com/tukaani-project/xz/security/advisories/new). [Security Advisories](https://github.com/tukaani-project/xz/security/advisories/new).
While both options are available, we prefer email. While both options are available, we prefer email. In any case, please
provide a clear description of the vulnerability including:
- Affected versions of XZ Utils
- Estimated severity (low, moderate, high, critical)
- Steps to recreate the vulnerability
- All relevant files (core dumps, build logs, input files, etc.)
This project is maintained by a team of volunteers on a reasonable-effort This project is maintained by a team of volunteers on a reasonable-effort
basis. As such, please give us 90 days to work on a fix before basis. As such, please give us 90 days to work on a fix before

View File

@ -333,69 +333,6 @@ endif()
option(BUILD_SHARED_LIBS "Build liblzma as a shared library instead of static") option(BUILD_SHARED_LIBS "Build liblzma as a shared library instead of static")
if(NOT WIN32)
# Symbol versioning only affects ELF shared libraries. The option is
# ignored for static libraries.
#
# Determine the default value so that it's always set with
# shared libraries in mind which helps if the build dir is reconfigured
# from static to shared libs without resetting the cache variables.
set(SYMBOL_VERSIONING_DEFAULT OFF)
if(CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
(CMAKE_SYSTEM_PROCESSOR MATCHES "[Mm]icro[Bb]laze" OR
CMAKE_C_COMPILER_ID STREQUAL "NVHPC"))
# As a special case, GNU/Linux on MicroBlaze gets the generic
# symbol versioning because GCC 12 doesn't support the __symver__
# attribute on MicroBlaze. On Linux, CMAKE_SYSTEM_PROCESSOR comes
# from "uname -m" for native builds (should be "microblaze") or from
# the CMake toolchain file (not perfectly standardized but it very
# likely has "microblaze" in lower case or mixed case somewhere in
# the string).
#
# NVIDIA HPC Compiler doesn't support symbol versioning but
# it uses the linked from the system so the linker script
# can still be used to get the generic symbol versioning.
set(SYMBOL_VERSIONING_DEFAULT "generic")
elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux")
# GNU/Linux-specific symbol versioning for shared liblzma.
# This includes a few extra compatibility symbols for RHEL/CentOS 7
# which are pointless on non-glibc non-Linux systems.
#
# Avoid symvers on Linux with non-glibc like musl and uClibc.
# In Autoconf it's enough to check that $host_os equals linux-gnu
# instead of, for example, linux-musl. CMake doesn't provide such
# a method.
#
# This check is here for now since it's not strictly required
# by anything else.
check_c_source_compiles(
"#include <features.h>
#if defined(__GLIBC__) && !defined(__UCLIBC__)
int main(void) { return 0; }
#else
compile error
#endif
"
IS_LINUX_WITH_GLIBC)
if(IS_LINUX_WITH_GLIBC)
set(SYMBOL_VERSIONING_DEFAULT "linux")
endif()
elseif(CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
set(SYMBOL_VERSIONING_DEFAULT "generic")
endif()
set(SYMBOL_VERSIONING "${SYMBOL_VERSIONING_DEFAULT}" CACHE STRING
"Enable ELF shared library symbol versioning (OFF, generic, linux)")
# Show a dropdown menu in CMake GUI:
set_property(CACHE SYMBOL_VERSIONING PROPERTY STRINGS "OFF;generic;linux")
endif()
add_library(liblzma add_library(liblzma
src/common/mythread.h src/common/mythread.h
src/common/sysdefs.h src/common/sysdefs.h
@ -1339,7 +1276,22 @@ if(WIN32)
# Disable __declspec(dllimport) when linking against static liblzma. # Disable __declspec(dllimport) when linking against static liblzma.
target_compile_definitions(liblzma INTERFACE LZMA_API_STATIC) target_compile_definitions(liblzma INTERFACE LZMA_API_STATIC)
endif() endif()
elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "linux") elseif(BUILD_SHARED_LIBS AND CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
NOT CMAKE_SYSTEM_PROCESSOR MATCHES "[Mm]icro[Bb]laze")
# GNU/Linux-specific symbol versioning for shared liblzma.
# This includes a few extra compatibility symbols for RHEL/CentOS 7
# which are pointless on non-glibc non-Linux systems.
#
# As a special case, GNU/Linux on MicroBlaze gets the generic
# symbol versioning because GCC 12 doesn't support the __symver__
# attribute on MicroBlaze. On Linux, CMAKE_SYSTEM_PROCESSOR comes
# from "uname -m" for native builds (should be "microblaze") or from
# the CMake toolchain file (not perfectly standardized but it very
# likely has "microblaze" in lower case or mixed case somewhere in
# the string).
#
# FIXME? Avoid symvers on Linux with non-glibc like musl?
#
# Note that adding link options doesn't affect static builds # Note that adding link options doesn't affect static builds
# but HAVE_SYMBOL_VERSIONS_LINUX must not be used with static builds # but HAVE_SYMBOL_VERSIONS_LINUX must not be used with static builds
# because it would put symbol versions into the static library which # because it would put symbol versions into the static library which
@ -1355,7 +1307,10 @@ elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "linux")
set_target_properties(liblzma PROPERTIES set_target_properties(liblzma PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_linux.map" LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_linux.map"
) )
elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "generic") elseif(BUILD_SHARED_LIBS AND (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD" OR
CMAKE_SYSTEM_NAME STREQUAL "Linux"))
# Generic symbol versioning for shared liblzma is used on FreeBSD and
# also on GNU/Linux on MicroBlaze.
target_link_options(liblzma PRIVATE target_link_options(liblzma PRIVATE
"-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_generic.map" "-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_generic.map"
) )

43
INSTALL
View File

@ -561,45 +561,10 @@ XZ Utils Installation
sandboxing. If no Landlock support sandboxing. If no Landlock support
is found, configure will give an error. is found, configure will give an error.
--enable-symbol-versions[=VARIANT] --enable-symbol-versions
Use symbol versioning for liblzma shared library. Use symbol versioning for liblzma. This is enabled by
This is enabled by default on GNU/Linux (glibc only), default on GNU/Linux, other GNU-based systems, and
other GNU-based systems, and FreeBSD. FreeBSD.
Symbol versioning is never used for static liblzma. This
option is ignored when not building a shared library.
Supported VARIANTs:
no Disable symbol versioning. This is the
same as using --disable-symbol-versions.
auto Autodetect between "no", "linux",
and "generic".
yes Autodetect between "linux" and
"generic". This forces symbol
versioning to be used when
building a shared library.
generic Generic version is the default for
FreeBSD and GNU/Linux on MicroBlaze.
This is also used on GNU/Linux when
building with NVIDIA HPC Compiler
because the compiler doesn't support
the features required for the "linux"
variant below.
linux Special version for GNU/Linux (glibc
only). This adds a few extra symbol
versions for compatibility with binaries
that have been linked against a liblzma
version that has been patched with
"xz-5.2.2-compat-libs.patch" from
RHEL/CentOS 7. That patch was used
by some build tools outside of
RHEL/CentOS 7 too.
--enable-debug --enable-debug
This enables the assert() macro and possibly some other This enables the assert() macro and possibly some other

1
THANKS
View File

@ -76,7 +76,6 @@ has been important. :-) In alphabetical order:
- Richard Koch - Richard Koch
- Anton Kochkov - Anton Kochkov
- Ville Koskinen - Ville Koskinen
- Sergey Kosukhin
- Marcin Kowalczyk - Marcin Kowalczyk
- Jan Kratochvil - Jan Kratochvil
- Christian Kujau - Christian Kujau

View File

@ -709,44 +709,23 @@ fi
# --with-pic and --without-pic though. As long as neither --with-pic nor # --with-pic and --without-pic though. As long as neither --with-pic nor
# --without-pic is used then we can use #ifdef PIC to detect if the file is # --without-pic is used then we can use #ifdef PIC to detect if the file is
# being built for a shared library. # being built for a shared library.
AS_IF([test "x$enable_symbol_versions" = xno], [ if test "x$enable_symbol_versions" = xno ; then
enable_symbol_versions=no enable_symbol_versions=no
AC_MSG_RESULT([no]) AC_MSG_RESULT([no])
], [test "x$enable_shared" = xno], [ elif test "x$enable_shared" = xno ; then
enable_symbol_versions=no enable_symbol_versions=no
AC_MSG_RESULT([no (not building a shared library)]) AC_MSG_RESULT([no (not building a shared library)])
], [ else
# "yes" means that symbol version are to be used but we need to
# autodetect which variant to use.
if test "x$enable_symbol_versions" = xyes ; then
case "$host_cpu-$host_os" in case "$host_cpu-$host_os" in
microblaze*) microblaze*)
# GCC 12 on MicroBlaze doesn't support # GCC 12 on MicroBlaze doesn't support __symver__
# __symver__ attribute. It's simplest and # attribute. It's simplest and safest to use the
# safest to use the generic version on that # generic version on that platform since then only
# platform since then only the linker script # the linker script is needed. The RHEL/CentOS 7
# is needed. The RHEL/CentOS 7 compatibility # compatibility symbols don't matter on MicroBlaze.
# symbols don't matter on MicroBlaze.
enable_symbol_versions=generic enable_symbol_versions=generic
;; ;;
*-linux*) *-linux*)
# NVIDIA HPC Compiler doesn't support symbol
# versioning but the linker script can still
# be used.
AC_EGREP_CPP([use_generic_symbol_versioning],
[#ifdef __NVCOMPILER
use_generic_symbol_versioning
#endif],
[enable_symbol_versions=generic],
[enable_symbol_versions=linux])
;;
*)
enable_symbol_versions=generic
;;
esac
fi
if test "x$enable_symbol_versions" = xlinux ; then
case "$pic_mode-$enable_static" in case "$pic_mode-$enable_static" in
default-*) default-*)
# Use symvers if PIC is defined. # Use symvers if PIC is defined.
@ -767,6 +746,7 @@ AS_IF([test "x$enable_symbol_versions" = xno], [
possibly picking only src/liblzma/.libs/liblzma.a from the static build.]) possibly picking only src/liblzma/.libs/liblzma.a from the static build.])
;; ;;
esac esac
enable_symbol_versions=linux
AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX], AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX],
[$have_symbol_versions_linux], [$have_symbol_versions_linux],
[Define to 1 to if GNU/Linux-specific details [Define to 1 to if GNU/Linux-specific details
@ -777,12 +757,13 @@ AS_IF([test "x$enable_symbol_versions" = xno], [
time with Libtool if neither --with-pic nor time with Libtool if neither --with-pic nor
--without-pic is used). This define must be --without-pic is used). This define must be
used together with liblzma_linux.map.]) used together with liblzma_linux.map.])
elif test "x$enable_symbol_versions" != xgeneric ; then ;;
AC_MSG_RESULT([]) *)
AC_MSG_ERROR([unknown symbol versioning variant '$enable_symbol_versions']) enable_symbol_versions=generic
fi ;;
esac
AC_MSG_RESULT([yes ($enable_symbol_versions)]) AC_MSG_RESULT([yes ($enable_symbol_versions)])
]) fi
AM_CONDITIONAL([COND_SYMVERS_LINUX], AM_CONDITIONAL([COND_SYMVERS_LINUX],
[test "x$enable_symbol_versions" = xlinux]) [test "x$enable_symbol_versions" = xlinux])

View File

@ -20,7 +20,7 @@ liblzma_la_CPPFLAGS = \
-I$(top_srcdir)/src/liblzma/simple \ -I$(top_srcdir)/src/liblzma/simple \
-I$(top_srcdir)/src/common \ -I$(top_srcdir)/src/common \
-DTUKLIB_SYMBOL_PREFIX=lzma_ -DTUKLIB_SYMBOL_PREFIX=lzma_
liblzma_la_LDFLAGS = -no-undefined -version-info 11:99:6 liblzma_la_LDFLAGS = -no-undefined -version-info 11:1:6
EXTRA_DIST += liblzma_generic.map liblzma_linux.map validate_map.sh EXTRA_DIST += liblzma_generic.map liblzma_linux.map validate_map.sh
if COND_SYMVERS_GENERIC if COND_SYMVERS_GENERIC

View File

@ -19,10 +19,10 @@
#define LZMA_VERSION_MAJOR 5 #define LZMA_VERSION_MAJOR 5
/** \brief Minor version number of the liblzma release. */ /** \brief Minor version number of the liblzma release. */
#define LZMA_VERSION_MINOR 7 #define LZMA_VERSION_MINOR 6
/** \brief Patch version number of the liblzma release. */ /** \brief Patch version number of the liblzma release. */
#define LZMA_VERSION_PATCH 0 #define LZMA_VERSION_PATCH 1
/** /**
* \brief Version stability marker * \brief Version stability marker
@ -32,7 +32,7 @@
* - LZMA_VERSION_STABILITY_BETA * - LZMA_VERSION_STABILITY_BETA
* - LZMA_VERSION_STABILITY_STABLE * - LZMA_VERSION_STABILITY_STABLE
*/ */
#define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_ALPHA #define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE
/** \brief Commit version number of the liblzma release */ /** \brief Commit version number of the liblzma release */
#ifndef LZMA_VERSION_COMMIT #ifndef LZMA_VERSION_COMMIT

View File

@ -67,19 +67,6 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
// This is only for x86-64 and ARM64 for now. This might be fine on // This is only for x86-64 and ARM64 for now. This might be fine on
// other 64-bit processors too. On big endian one should use xor // other 64-bit processors too. On big endian one should use xor
// instead of subtraction and switch to __builtin_clzll(). // instead of subtraction and switch to __builtin_clzll().
//
// Reasons to use subtraction instead of xor:
//
// - On some x86-64 processors (Intel Sandy Bridge to Tiger Lake),
// sub+jz and sub+jnz can be fused but xor+jz or xor+jnz cannot.
// Thus using subtraction has potential to be a tiny amount faster
// since the code checks if the quotient is non-zero.
//
// - Some processors (Intel Pentium 4) used to have more ALU
// resources for add/sub instructions than and/or/xor.
//
// The processor info is based on Agner Fog's microarchitecture.pdf
// version 2023-05-26. https://www.agner.org/optimize/
#define LZMA_MEMCMPLEN_EXTRA 8 #define LZMA_MEMCMPLEN_EXTRA 8
while (len < limit) { while (len < limit) {
const uint64_t x = read64ne(buf1 + len) - read64ne(buf2 + len); const uint64_t x = read64ne(buf1 + len) - read64ne(buf2 + len);

View File

@ -217,14 +217,12 @@ typedef struct {
uint16_t offset; uint16_t offset;
union { union {
// NVHPC has problems with unions that contain pointers that
// are not the first members, so keep "map" at the top.
const name_value_map *map;
struct { struct {
uint32_t min; uint32_t min;
uint32_t max; uint32_t max;
} range; } range;
const name_value_map *map;
} u; } u;
} option_map; } option_map;

View File

@ -25,11 +25,6 @@ decode_buffer(lzma_delta_coder *coder, uint8_t *buffer, size_t size)
} }
// For an unknown reason NVIDIA HPC Compiler needs this pragma
// to produce working code.
#ifdef __NVCOMPILER
# pragma routine novector
#endif
static lzma_ret static lzma_ret
delta_decode(void *coder_ptr, const lzma_allocator *allocator, delta_decode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos, const uint8_t *restrict in, size_t *restrict in_pos,

View File

@ -45,7 +45,6 @@
// and different processors. Overall 0x1F0 seems to be the best choice. // and different processors. Overall 0x1F0 seems to be the best choice.
#ifndef LZMA_RANGE_DECODER_CONFIG #ifndef LZMA_RANGE_DECODER_CONFIG
# if defined(__x86_64__) && !defined(__ILP32__) \ # if defined(__x86_64__) && !defined(__ILP32__) \
&& !defined(__NVCOMPILER) \
&& (defined(__GNUC__) || defined(__clang__)) && (defined(__GNUC__) || defined(__clang__))
# define LZMA_RANGE_DECODER_CONFIG 0x1F0 # define LZMA_RANGE_DECODER_CONFIG 0x1F0
# else # else