Compare commits

...

26 Commits

Author SHA1 Message Date
Jia Tan af071ef770 Docs: Simplify SECURITY.md. 2024-03-26 03:38:12 +08:00
Lasse Collin 0b99783d63 liblzma: memcmplen.h: Add a comment why subtraction is used. 2024-03-22 17:46:30 +02:00
Lasse Collin 8a25ba024d INSTALL: Document arguments of --enable-symbol-versions. 2024-03-15 18:04:10 +02:00
Lasse Collin 49324b711f Build: Use only the generic symbol versioning with NVIDIA HPC Compiler.
This does the previous commit with CMake.

AC_EGREP_CPP uses AC_REQUIRE so the outermost if-commands must
be changed to AS_IF to ensure that things wont break some day.
See 5a5bd7f871.
2024-03-15 18:04:10 +02:00
Lasse Collin c273123ed0 CMake: Use only the generic symbol versioning with NVIDIA HPC Compiler.
It doesn't support the __symver__ attribute or __asm__(".symver ...").
The generic symbol versioning can still be used since it only needs
linker support.
2024-03-15 18:04:10 +02:00
Lasse Collin df7f487648 Update THANKS. 2024-03-15 18:04:10 +02:00
Lasse Collin 3217b82b3e liblzma: Minor comment edits. 2024-03-15 18:03:47 +02:00
Sergey Kosukhin 096bc0e3f8 liblzma: Fix building with NVHPC (NVIDIA HPC SDK).
NVHPC compiler has several issues that make it impossible to
build liblzma:
  - the compiler cannot handle unions that contain pointers that
    are not the first members;
  - the compiler cannot handle the assembler code in range_decoder.h
    (LZMA_RANGE_DECODER_CONFIG has to be set to zero);
  - the compiler fails to produce valid code for delta_decode if the
    vectorization is enabled, which results in failed tests.

This introduces NVHPC-specific workarounds that address the issues.
2024-03-15 17:30:50 +02:00
Lasse Collin 2ad7fad670 CMake: Disable symbol versioning on non-glibc Linux.
This better matches what configure.ac does. For example, musl has
only basic symbol versioning support:

https://wiki.musl-libc.org/functional-differences-from-glibc.html#Symbol_versioning

configure.ac tries to enable symbol versioning only with glibc
so now CMake does the same.
2024-03-15 17:30:50 +02:00
Lasse Collin 82f0c0d39e CMake: Make symbol versioning configurable. 2024-03-15 17:30:50 +02:00
Lasse Collin 45d33bfc45 Build: Style tweaks to configure.ac.
The AC_MSG_ERROR line is overlong anyway as are a few other
AC_MSG_ERROR lines already.
2024-03-13 22:00:23 +02:00
Sergey Kosukhin f56ed6fac6 Build: Let the users override the symbol versioning variant.
There are cases when the users want to decide themselves whether
they want to have the generic (even on GNU/Linux) or the linux
(even if we do not recommend that) symbol versioning variant.
The former might be needed to circumvent compiler issues (i.e.
the compiler does not support all features that are required
for the linux versioning), the latter might help in overriding
the assumptions made in the configure script.
2024-03-13 21:59:55 +02:00
Jia Tan a4f2e20d84 Add NEWS for 5.6.1 2024-03-09 11:27:27 +08:00
Jia Tan f01be8ad75 Translations: Add missing --riscv option to man page translations. 2024-03-09 10:43:20 +08:00
Jia Tan 6e636819e8 Tests: Update two test files.
The original files were generated with random local to my machine.
To better reproduce these files in the future, a constant seed was used
to recreate these files.
2024-03-09 10:18:29 +08:00
Jia Tan a3a29bbd5d Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
The first stream in this file is valid, so this tests that xz properly
stops after decompressing it.
2024-03-09 10:08:32 +08:00
Jia Tan 0b4ccc9145 Tests: Update RISC-V test files.
This increases code coverage and tests for possible shifting bugs.
2024-03-09 10:05:32 +08:00
Jia Tan 8c9b8b2063 liblzma: Fix typos in crc32_fast.c and crc64_fast.c. 2024-03-09 09:52:32 +08:00
Jia Tan b93a8d7631 Tests: Replace HAVE_MICROLZMA usage in CMake and Autotools builds.
This reverts commit adaacafde6.
2024-03-09 09:49:55 +08:00
Jia Tan 82ecc53819 liblzma: Fix false Valgrind error report with GCC.
With GCC and a certain combination of flags, Valgrind will falsely
trigger an invalid write. This appears to be due to the omission of
instructions to properly save, set up, and restore the frame pointer.

The IFUNC resolver is a leaf function since it only calls a function
that is inlined. So sometimes GCC omits the frame pointer instructions
in the resolver unless this optimization is explictly disabled.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
2024-03-09 09:20:57 +08:00
Lasse Collin 3007e74ef2 liblzma: Fix a typo in a comment in the RISC-V filter. 2024-03-05 23:23:16 +02:00
Jia Tan 72d2933bfa liblzma: Use attribute no_profile_instrument_function with ifunc.
Thanks to Sam James for determining this was the attribute needed to
workaround the GCC bug and for his version of the patch in Gentoo.
2024-03-05 01:54:30 +08:00
Jia Tan e5faaebbcf Build: Require attribute no_profile_instrument_function for ifunc usage.
Using __attribute__((__no_profile_instrument_function__)) on the ifunc
resolver works around a bug in GCC -fprofile-generate:
it adds profiling code even to ifunc resolvers which can make
the ifunc resolver crash at program startup. This attribute
was not introduced until GCC 7 and Clang 13, so ifunc won't
be used with prior versions of these compilers.

This bug was brought to our attention by:

    https://bugs.gentoo.org/925415

And was reported to upstream GCC by:

    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11411
2024-03-05 01:54:30 +08:00
Lasse Collin 7eeadd279a liblzma: Fix a comment in the RISC-V filter. 2024-03-04 19:23:18 +02:00
Lasse Collin 5f3d059529 CMake: Warn if translated man pages are missing. 2024-02-29 16:37:19 +02:00
Lasse Collin 4cd1042ee7 CMake: Warn if gettext tools and pre-created .gmo files are missing.
It's only done with CMake >= 3.20 and if library support
for translation was already found.

Sort of fixes: https://github.com/tukaani-project/xz/issues/82
2024-02-29 16:37:19 +02:00
27 changed files with 2355 additions and 2078 deletions

8
.github/SECURITY.md vendored
View File

@ -16,13 +16,7 @@ the chance that the exploit will be used before a patch is released.
You may submit a report by emailing us at You may submit a report by emailing us at
[xz@tukaani.org](mailto:xz@tukaani.org), or through [xz@tukaani.org](mailto:xz@tukaani.org), or through
[Security Advisories](https://github.com/tukaani-project/xz/security/advisories/new). [Security Advisories](https://github.com/tukaani-project/xz/security/advisories/new).
While both options are available, we prefer email. In any case, please While both options are available, we prefer email.
provide a clear description of the vulnerability including:
- Affected versions of XZ Utils
- Estimated severity (low, moderate, high, critical)
- Steps to recreate the vulnerability
- All relevant files (core dumps, build logs, input files, etc.)
This project is maintained by a team of volunteers on a reasonable-effort This project is maintained by a team of volunteers on a reasonable-effort
basis. As such, please give us 90 days to work on a fix before basis. As such, please give us 90 days to work on a fix before

View File

@ -278,9 +278,43 @@ endif()
if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.20") if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.20")
find_package(Intl) find_package(Intl)
find_package(Gettext) find_package(Gettext)
if(Intl_FOUND) if(Intl_FOUND)
option(ENABLE_NLS "Native Language Support (translated messages)" ON) option(ENABLE_NLS "Native Language Support (translated messages)" ON)
# If translation support is enabled but neither gettext tools or
# pre-generated .gmo files exist, translation support cannot be
# enabled.
#
# The detection of pre-generated .gmo files is done by only
# checking for the existence of a single .gmo file; Ukrainian
# is one of many translations that gets regular updates.
if(ENABLE_NLS AND NOT GETTEXT_FOUND AND
NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/po/uk.gmo")
# This only sets the variable, not the cache variable!
set(ENABLE_NLS OFF)
# This message is shown only when new enough CMake is used and
# library support for translations was found. The assumptions is
# that in this situation the user might have interest in the
# translations. This also keeps this code simpler.
message(WARNING "Native language support (NLS) has been disabled. "
"NLS support requires either gettext tools or "
"pre-generated .gmo files. The latter are only "
"available in distribution tarballs. "
"To avoid this warning, NLS can be explicitly "
"disabled by passing -DENABLE_NLS=OFF to cmake.")
endif()
# Warn if NLS is enabled but translated man pages are missing.
if(UNIX AND ENABLE_NLS AND
NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/po4a/man")
message(WARNING "Native language support (NLS) has been enabled "
"but pre-generated translated man pages "
"were not found and thus they won't be installed. "
"Run 'po4a/update-po' to generate them.")
endif()
# The *installed* name of the translation files is "xz.mo". # The *installed* name of the translation files is "xz.mo".
set(TRANSLATION_DOMAIN "xz") set(TRANSLATION_DOMAIN "xz")
endif() endif()
@ -299,6 +333,69 @@ endif()
option(BUILD_SHARED_LIBS "Build liblzma as a shared library instead of static") option(BUILD_SHARED_LIBS "Build liblzma as a shared library instead of static")
if(NOT WIN32)
# Symbol versioning only affects ELF shared libraries. The option is
# ignored for static libraries.
#
# Determine the default value so that it's always set with
# shared libraries in mind which helps if the build dir is reconfigured
# from static to shared libs without resetting the cache variables.
set(SYMBOL_VERSIONING_DEFAULT OFF)
if(CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
(CMAKE_SYSTEM_PROCESSOR MATCHES "[Mm]icro[Bb]laze" OR
CMAKE_C_COMPILER_ID STREQUAL "NVHPC"))
# As a special case, GNU/Linux on MicroBlaze gets the generic
# symbol versioning because GCC 12 doesn't support the __symver__
# attribute on MicroBlaze. On Linux, CMAKE_SYSTEM_PROCESSOR comes
# from "uname -m" for native builds (should be "microblaze") or from
# the CMake toolchain file (not perfectly standardized but it very
# likely has "microblaze" in lower case or mixed case somewhere in
# the string).
#
# NVIDIA HPC Compiler doesn't support symbol versioning but
# it uses the linked from the system so the linker script
# can still be used to get the generic symbol versioning.
set(SYMBOL_VERSIONING_DEFAULT "generic")
elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux")
# GNU/Linux-specific symbol versioning for shared liblzma.
# This includes a few extra compatibility symbols for RHEL/CentOS 7
# which are pointless on non-glibc non-Linux systems.
#
# Avoid symvers on Linux with non-glibc like musl and uClibc.
# In Autoconf it's enough to check that $host_os equals linux-gnu
# instead of, for example, linux-musl. CMake doesn't provide such
# a method.
#
# This check is here for now since it's not strictly required
# by anything else.
check_c_source_compiles(
"#include <features.h>
#if defined(__GLIBC__) && !defined(__UCLIBC__)
int main(void) { return 0; }
#else
compile error
#endif
"
IS_LINUX_WITH_GLIBC)
if(IS_LINUX_WITH_GLIBC)
set(SYMBOL_VERSIONING_DEFAULT "linux")
endif()
elseif(CMAKE_SYSTEM_NAME STREQUAL "FreeBSD")
set(SYMBOL_VERSIONING_DEFAULT "generic")
endif()
set(SYMBOL_VERSIONING "${SYMBOL_VERSIONING_DEFAULT}" CACHE STRING
"Enable ELF shared library symbol versioning (OFF, generic, linux)")
# Show a dropdown menu in CMake GUI:
set_property(CACHE SYMBOL_VERSIONING PROPERTY STRINGS "OFF;generic;linux")
endif()
add_library(liblzma add_library(liblzma
src/common/mythread.h src/common/mythread.h
src/common/sysdefs.h src/common/sysdefs.h
@ -819,10 +916,6 @@ if(MICROLZMA_DECODER)
target_sources(liblzma PRIVATE src/liblzma/common/microlzma_decoder.c) target_sources(liblzma PRIVATE src/liblzma/common/microlzma_decoder.c)
endif() endif()
if (MICROLZMA_ENCODER OR MICROLZMA_DECODER)
add_compile_definitions(HAVE_MICROLZMA)
endif()
############################# #############################
# lzip (.lz) format support # # lzip (.lz) format support #
@ -1048,6 +1141,13 @@ if(USE_ATTR_IFUNC STREQUAL "auto")
#endif #endif
static void func(void) { return; } static void func(void) { return; }
/*
* The attribute __no_profile_instrument_function__ is
* needed with GCC to prevent improper instrumentation in
* the ifunc resolver.
*/
__attribute__((__no_profile_instrument_function__))
static void (*resolve_func(void)) (void) { return func; } static void (*resolve_func(void)) (void) { return func; }
void func_ifunc(void) void func_ifunc(void)
__attribute__((__ifunc__(\"resolve_func\"))); __attribute__((__ifunc__(\"resolve_func\")));
@ -1239,22 +1339,7 @@ if(WIN32)
# Disable __declspec(dllimport) when linking against static liblzma. # Disable __declspec(dllimport) when linking against static liblzma.
target_compile_definitions(liblzma INTERFACE LZMA_API_STATIC) target_compile_definitions(liblzma INTERFACE LZMA_API_STATIC)
endif() endif()
elseif(BUILD_SHARED_LIBS AND CMAKE_SYSTEM_NAME STREQUAL "Linux" AND elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "linux")
NOT CMAKE_SYSTEM_PROCESSOR MATCHES "[Mm]icro[Bb]laze")
# GNU/Linux-specific symbol versioning for shared liblzma.
# This includes a few extra compatibility symbols for RHEL/CentOS 7
# which are pointless on non-glibc non-Linux systems.
#
# As a special case, GNU/Linux on MicroBlaze gets the generic
# symbol versioning because GCC 12 doesn't support the __symver__
# attribute on MicroBlaze. On Linux, CMAKE_SYSTEM_PROCESSOR comes
# from "uname -m" for native builds (should be "microblaze") or from
# the CMake toolchain file (not perfectly standardized but it very
# likely has "microblaze" in lower case or mixed case somewhere in
# the string).
#
# FIXME? Avoid symvers on Linux with non-glibc like musl?
#
# Note that adding link options doesn't affect static builds # Note that adding link options doesn't affect static builds
# but HAVE_SYMBOL_VERSIONS_LINUX must not be used with static builds # but HAVE_SYMBOL_VERSIONS_LINUX must not be used with static builds
# because it would put symbol versions into the static library which # because it would put symbol versions into the static library which
@ -1270,10 +1355,7 @@ elseif(BUILD_SHARED_LIBS AND CMAKE_SYSTEM_NAME STREQUAL "Linux" AND
set_target_properties(liblzma PROPERTIES set_target_properties(liblzma PROPERTIES
LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_linux.map" LINK_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_linux.map"
) )
elseif(BUILD_SHARED_LIBS AND (CMAKE_SYSTEM_NAME STREQUAL "FreeBSD" OR elseif(BUILD_SHARED_LIBS AND SYMBOL_VERSIONING STREQUAL "generic")
CMAKE_SYSTEM_NAME STREQUAL "Linux"))
# Generic symbol versioning for shared liblzma is used on FreeBSD and
# also on GNU/Linux on MicroBlaze.
target_link_options(liblzma PRIVATE target_link_options(liblzma PRIVATE
"-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_generic.map" "-Wl,--version-script=${CMAKE_CURRENT_SOURCE_DIR}/src/liblzma/liblzma_generic.map"
) )
@ -2003,11 +2085,20 @@ if(BUILD_TESTING)
test_index_hash test_index_hash
test_lzip_decoder test_lzip_decoder
test_memlimit test_memlimit
test_microlzma
test_stream_flags test_stream_flags
test_vli test_vli
) )
# MicroLZMA encoder is needed for both encoder and decoder tests.
# If MicroLZMA decoder is not configured but LZMA1 decoder is, then
# test_microlzma will fail to compile because this configuration is
# not possible in the Autotools build, so the test was not made to
# support it since it would have required additional changes.
if (MICROLZMA_ENCODER AND (MICROLZMA_DECODER
OR NOT "lzma1" IN_LIST DECODERS))
list(APPEND LIBLZMA_TESTS test_microlzma)
endif()
foreach(TEST IN LISTS LIBLZMA_TESTS) foreach(TEST IN LISTS LIBLZMA_TESTS)
add_executable("${TEST}" "tests/${TEST}.c") add_executable("${TEST}" "tests/${TEST}.c")

43
INSTALL
View File

@ -561,10 +561,45 @@ XZ Utils Installation
sandboxing. If no Landlock support sandboxing. If no Landlock support
is found, configure will give an error. is found, configure will give an error.
--enable-symbol-versions --enable-symbol-versions[=VARIANT]
Use symbol versioning for liblzma. This is enabled by Use symbol versioning for liblzma shared library.
default on GNU/Linux, other GNU-based systems, and This is enabled by default on GNU/Linux (glibc only),
FreeBSD. other GNU-based systems, and FreeBSD.
Symbol versioning is never used for static liblzma. This
option is ignored when not building a shared library.
Supported VARIANTs:
no Disable symbol versioning. This is the
same as using --disable-symbol-versions.
auto Autodetect between "no", "linux",
and "generic".
yes Autodetect between "linux" and
"generic". This forces symbol
versioning to be used when
building a shared library.
generic Generic version is the default for
FreeBSD and GNU/Linux on MicroBlaze.
This is also used on GNU/Linux when
building with NVIDIA HPC Compiler
because the compiler doesn't support
the features required for the "linux"
variant below.
linux Special version for GNU/Linux (glibc
only). This adds a few extra symbol
versions for compatibility with binaries
that have been linked against a liblzma
version that has been patched with
"xz-5.2.2-compat-libs.patch" from
RHEL/CentOS 7. That patch was used
by some build tools outside of
RHEL/CentOS 7 too.
--enable-debug --enable-debug
This enables the assert() macro and possibly some other This enables the assert() macro and possibly some other

26
NEWS
View File

@ -2,6 +2,32 @@
XZ Utils Release Notes XZ Utils Release Notes
====================== ======================
5.6.1 (2024-03-09)
* liblzma: Fixed two bugs relating to GNU indirect function (IFUNC)
with GCC. The more serious bug caused a program linked with
liblzma to crash on start up if the flag -fprofile-generate was
used to build liblzma. The second bug caused liblzma to falsely
report an invalid write to Valgrind when loading liblzma.
* xz: Changed the messages for thread reduction due to memory
constraints to only appear under the highest verbosity level.
* Build:
- Fixed a build issue when the header file <linux/landlock.h>
was present on the system but the Landlock system calls were
not defined in <sys/syscall.h>.
- The CMake build now warns and disables NLS if both gettext
tools and pre-created .gmo files are missing. Previously,
this caused the CMake build to fail.
* Minor improvements to man pages.
* Minor improvements to tests.
5.6.0 (2024-02-24) 5.6.0 (2024-02-24)
This bumps the minor version of liblzma because new features were This bumps the minor version of liblzma because new features were

1
THANKS
View File

@ -76,6 +76,7 @@ has been important. :-) In alphabetical order:
- Richard Koch - Richard Koch
- Anton Kochkov - Anton Kochkov
- Ville Koskinen - Ville Koskinen
- Sergey Kosukhin
- Marcin Kowalczyk - Marcin Kowalczyk
- Jan Kratochvil - Jan Kratochvil
- Christian Kujau - Christian Kujau

View File

@ -304,13 +304,8 @@ AC_ARG_ENABLE([microlzma], AS_HELP_STRING([--disable-microlzma],
for example, erofs-utils.]), for example, erofs-utils.]),
[], [enable_microlzma=yes]) [], [enable_microlzma=yes])
case $enable_microlzma in case $enable_microlzma in
yes) yes | no)
AC_DEFINE([HAVE_MICROLZMA], [1], AC_MSG_RESULT([$enable_microlzma])
[Define to 1 if MicroLZMA support is enabled.])
AC_MSG_RESULT([yes])
;;
no)
AC_MSG_RESULT([no])
;; ;;
*) *)
AC_MSG_RESULT([]) AC_MSG_RESULT([])
@ -714,61 +709,80 @@ fi
# --with-pic and --without-pic though. As long as neither --with-pic nor # --with-pic and --without-pic though. As long as neither --with-pic nor
# --without-pic is used then we can use #ifdef PIC to detect if the file is # --without-pic is used then we can use #ifdef PIC to detect if the file is
# being built for a shared library. # being built for a shared library.
if test "x$enable_symbol_versions" = xno ; then AS_IF([test "x$enable_symbol_versions" = xno], [
enable_symbol_versions=no enable_symbol_versions=no
AC_MSG_RESULT([no]) AC_MSG_RESULT([no])
elif test "x$enable_shared" = xno ; then ], [test "x$enable_shared" = xno], [
enable_symbol_versions=no enable_symbol_versions=no
AC_MSG_RESULT([no (not building a shared library)]) AC_MSG_RESULT([no (not building a shared library)])
else ], [
case "$host_cpu-$host_os" in # "yes" means that symbol version are to be used but we need to
microblaze*) # autodetect which variant to use.
# GCC 12 on MicroBlaze doesn't support __symver__ if test "x$enable_symbol_versions" = xyes ; then
# attribute. It's simplest and safest to use the case "$host_cpu-$host_os" in
# generic version on that platform since then only microblaze*)
# the linker script is needed. The RHEL/CentOS 7 # GCC 12 on MicroBlaze doesn't support
# compatibility symbols don't matter on MicroBlaze. # __symver__ attribute. It's simplest and
enable_symbol_versions=generic # safest to use the generic version on that
;; # platform since then only the linker script
*-linux*) # is needed. The RHEL/CentOS 7 compatibility
case "$pic_mode-$enable_static" in # symbols don't matter on MicroBlaze.
default-*) enable_symbol_versions=generic
# Use symvers if PIC is defined. ;;
have_symbol_versions_linux=2 *-linux*)
;; # NVIDIA HPC Compiler doesn't support symbol
*-no) # versioning but the linker script can still
# Not building static library. # be used.
# Use symvers unconditionally. AC_EGREP_CPP([use_generic_symbol_versioning],
have_symbol_versions_linux=1 [#ifdef __NVCOMPILER
;; use_generic_symbol_versioning
*) #endif],
AC_MSG_RESULT([]) [enable_symbol_versions=generic],
AC_MSG_ERROR([ [enable_symbol_versions=linux])
;;
*)
enable_symbol_versions=generic
;;
esac
fi
if test "x$enable_symbol_versions" = xlinux ; then
case "$pic_mode-$enable_static" in
default-*)
# Use symvers if PIC is defined.
have_symbol_versions_linux=2
;;
*-no)
# Not building static library.
# Use symvers unconditionally.
have_symbol_versions_linux=1
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([
On GNU/Linux, building both shared and static library at the same time On GNU/Linux, building both shared and static library at the same time
is not supported if --with-pic or --without-pic is used. is not supported if --with-pic or --without-pic is used.
Use either --disable-shared or --disable-static to build one type Use either --disable-shared or --disable-static to build one type
of library at a time. If both types are needed, build one at a time, of library at a time. If both types are needed, build one at a time,
possibly picking only src/liblzma/.libs/liblzma.a from the static build.]) possibly picking only src/liblzma/.libs/liblzma.a from the static build.])
;; ;;
esac esac
enable_symbol_versions=linux AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX],
AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX], [$have_symbol_versions_linux],
[$have_symbol_versions_linux], [Define to 1 to if GNU/Linux-specific details
[Define to 1 to if GNU/Linux-specific details are unconditionally wanted for symbol
are unconditionally wanted for symbol versioning. Define to 2 to if these are wanted
versioning. Define to 2 to if these are wanted only if also PIC is defined (allows building
only if also PIC is defined (allows building both shared and static liblzma at the same
both shared and static liblzma at the same time with Libtool if neither --with-pic nor
time with Libtool if neither --with-pic nor --without-pic is used). This define must be
--without-pic is used). This define must be used together with liblzma_linux.map.])
used together with liblzma_linux.map.]) elif test "x$enable_symbol_versions" != xgeneric ; then
;; AC_MSG_RESULT([])
*) AC_MSG_ERROR([unknown symbol versioning variant '$enable_symbol_versions'])
enable_symbol_versions=generic fi
;;
esac
AC_MSG_RESULT([yes ($enable_symbol_versions)]) AC_MSG_RESULT([yes ($enable_symbol_versions)])
fi ])
AM_CONDITIONAL([COND_SYMVERS_LINUX], AM_CONDITIONAL([COND_SYMVERS_LINUX],
[test "x$enable_symbol_versions" = xlinux]) [test "x$enable_symbol_versions" = xlinux])
@ -915,6 +929,13 @@ if test "x$enable_ifunc" = xauto ; then
#endif #endif
static void func(void) { return; } static void func(void) { return; }
/*
* The attribute __no_profile_instrument_function__ is
* needed with GCC to prevent improper instrumentation in
* the ifunc resolver.
*/
__attribute__((__no_profile_instrument_function__))
static void (*resolve_func (void)) (void) { return func; } static void (*resolve_func (void)) (void) { return func; }
void func_ifunc (void) void func_ifunc (void)
__attribute__((__ifunc__("resolve_func"))); __attribute__((__ifunc__("resolve_func")));

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -135,6 +135,8 @@ typedef uint32_t (*crc32_func_type)(
// This resolver is shared between all three dispatch methods. It serves as // This resolver is shared between all three dispatch methods. It serves as
// the ifunc resolver if ifunc is supported, otherwise it is called as a // the ifunc resolver if ifunc is supported, otherwise it is called as a
// regular function by the constructor or first call resolution methods. // regular function by the constructor or first call resolution methods.
// The function attributes are needed for safe IFUNC resolver usage with GCC.
lzma_resolver_attributes
static crc32_func_type static crc32_func_type
crc32_resolve(void) crc32_resolve(void)
{ {

View File

@ -98,6 +98,7 @@ typedef uint64_t (*crc64_func_type)(
# pragma GCC diagnostic ignored "-Wunused-function" # pragma GCC diagnostic ignored "-Wunused-function"
#endif #endif
lzma_resolver_attributes
static crc64_func_type static crc64_func_type
crc64_resolve(void) crc64_resolve(void)
{ {

View File

@ -128,6 +128,31 @@
# endif # endif
#endif #endif
#ifdef CRC_USE_IFUNC
// Two function attributes are needed to make IFUNC safe with GCC.
//
// no-omit-frame-pointer prevents false Valgrind issues when combined with
// a few other compiler flags. The optimize attribute is supported on
// GCC >= 4.4 and is not supported with Clang.
# if TUKLIB_GNUC_REQ(4,4) && !defined(__clang__)
# define no_omit_frame_pointer \
__attribute__((optimize("no-omit-frame-pointer")))
# else
# define no_omit_frame_pointer
# endif
// The __no_profile_instrument_function__ attribute support is checked when
// determining if ifunc can be used, so it is safe to use unconditionally.
// This attribute is needed because GCC can add profiling to the IFUNC
// resolver, which calls functions that have not yet been relocated leading
// to a crash on liblzma start up.
# define lzma_resolver_attributes \
__attribute__((__no_profile_instrument_function__)) \
no_omit_frame_pointer
#else
# define lzma_resolver_attributes
#endif
// For CRC32 use the generic slice-by-eight implementation if no optimized // For CRC32 use the generic slice-by-eight implementation if no optimized
// version is available. // version is available.
#if !defined(CRC32_ARCH_OPTIMIZED) && !defined(CRC32_GENERIC) #if !defined(CRC32_ARCH_OPTIMIZED) && !defined(CRC32_GENERIC)

View File

@ -67,6 +67,19 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2,
// This is only for x86-64 and ARM64 for now. This might be fine on // This is only for x86-64 and ARM64 for now. This might be fine on
// other 64-bit processors too. On big endian one should use xor // other 64-bit processors too. On big endian one should use xor
// instead of subtraction and switch to __builtin_clzll(). // instead of subtraction and switch to __builtin_clzll().
//
// Reasons to use subtraction instead of xor:
//
// - On some x86-64 processors (Intel Sandy Bridge to Tiger Lake),
// sub+jz and sub+jnz can be fused but xor+jz or xor+jnz cannot.
// Thus using subtraction has potential to be a tiny amount faster
// since the code checks if the quotient is non-zero.
//
// - Some processors (Intel Pentium 4) used to have more ALU
// resources for add/sub instructions than and/or/xor.
//
// The processor info is based on Agner Fog's microarchitecture.pdf
// version 2023-05-26. https://www.agner.org/optimize/
#define LZMA_MEMCMPLEN_EXTRA 8 #define LZMA_MEMCMPLEN_EXTRA 8
while (len < limit) { while (len < limit) {
const uint64_t x = read64ne(buf1 + len) - read64ne(buf2 + len); const uint64_t x = read64ne(buf1 + len) - read64ne(buf2 + len);

View File

@ -217,12 +217,14 @@ typedef struct {
uint16_t offset; uint16_t offset;
union { union {
// NVHPC has problems with unions that contain pointers that
// are not the first members, so keep "map" at the top.
const name_value_map *map;
struct { struct {
uint32_t min; uint32_t min;
uint32_t max; uint32_t max;
} range; } range;
const name_value_map *map;
} u; } u;
} option_map; } option_map;

View File

@ -25,6 +25,11 @@ decode_buffer(lzma_delta_coder *coder, uint8_t *buffer, size_t size)
} }
// For an unknown reason NVIDIA HPC Compiler needs this pragma
// to produce working code.
#ifdef __NVCOMPILER
# pragma routine novector
#endif
static lzma_ret static lzma_ret
delta_decode(void *coder_ptr, const lzma_allocator *allocator, delta_decode(void *coder_ptr, const lzma_allocator *allocator,
const uint8_t *restrict in, size_t *restrict in_pos, const uint8_t *restrict in, size_t *restrict in_pos,

View File

@ -45,6 +45,7 @@
// and different processors. Overall 0x1F0 seems to be the best choice. // and different processors. Overall 0x1F0 seems to be the best choice.
#ifndef LZMA_RANGE_DECODER_CONFIG #ifndef LZMA_RANGE_DECODER_CONFIG
# if defined(__x86_64__) && !defined(__ILP32__) \ # if defined(__x86_64__) && !defined(__ILP32__) \
&& !defined(__NVCOMPILER) \
&& (defined(__GNUC__) || defined(__clang__)) && (defined(__GNUC__) || defined(__clang__))
# define LZMA_RANGE_DECODER_CONFIG 0x1F0 # define LZMA_RANGE_DECODER_CONFIG 0x1F0
# else # else

View File

@ -116,7 +116,7 @@ AUIPC with rd != x0
Zfh, F, D, and Q: Zfh, F, D, and Q:
* RV32I: LB, LH, LW, LBU, LHU, SB, SH, SW * RV32I: LB, LH, LW, LBU, LHU, SB, SH, SW
* RV64I has also: LD, LWU, SD * RV64I has also: LD, LWU, SD
* Zhf: FLH, FSH * Zfh: FLH, FSH
* F: FLW, FSW * F: FLW, FSW
* D: FLD, FSD * D: FLD, FSD
* Q: FLQ, FSQ * Q: FLQ, FSQ
@ -320,11 +320,11 @@ AUIPC with rd == x0
// The left-hand side takes care of (1) and (2). // The left-hand side takes care of (1) and (2).
// (a) The lowest 7 bits are already known to be AUIPC so subtracting 0x17 // (a) The lowest 7 bits are already known to be AUIPC so subtracting 0x17
// makes those bits zeros. // makes those bits zeros.
// (b) If AUIPC rd equals x2, subtracting 0x10 makes bits [11:7] zeros. // (b) If AUIPC rd equals x2, subtracting 0x100 makes bits [11:7] zeros.
// If rd doesn't equal x2, then there will be at least one non-zero bit // If rd doesn't equal x2, then there will be at least one non-zero bit
// and the next step (c) is irrelevant. // and the next step (c) is irrelevant.
// (c) If the lowest two opcode bits of the packed inst2 are set in [13:12], // (c) If the lowest two opcode bits of the packed inst2 are set in [13:12],
// then subtracting 0x300 will make those bits zeros. Otherwise there // then subtracting 0x3000 will make those bits zeros. Otherwise there
// will be at least one non-zero bit. // will be at least one non-zero bit.
// //
// The shift by 18 removes the high bits from the final '>=' comparison and // The shift by 18 removes the high bits from the final '>=' comparison and

View File

@ -42,8 +42,7 @@ check_PROGRAMS = \
test_bcj_exact_size \ test_bcj_exact_size \
test_memlimit \ test_memlimit \
test_lzip_decoder \ test_lzip_decoder \
test_vli \ test_vli
test_microlzma
TESTS = \ TESTS = \
test_check \ test_check \
@ -58,7 +57,6 @@ TESTS = \
test_memlimit \ test_memlimit \
test_lzip_decoder \ test_lzip_decoder \
test_vli \ test_vli \
test_microlzma \
test_files.sh \ test_files.sh \
test_suffix.sh \ test_suffix.sh \
test_compress_prepared_bcj_sparc \ test_compress_prepared_bcj_sparc \
@ -67,6 +65,11 @@ TESTS = \
test_compress_generated_random \ test_compress_generated_random \
test_compress_generated_text test_compress_generated_text
if COND_MICROLZMA
check_PROGRAMS += test_microlzma
TESTS += test_microlzma
endif
if COND_SCRIPTS if COND_SCRIPTS
TESTS += test_scripts.sh TESTS += test_scripts.sh
endif endif

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@ -149,6 +149,17 @@ else
exit 1 exit 1
fi fi
# Test that --single-stream can decompress bad-3-corrupt_lzma2.xz.
# The first Stream in this file should decompress without errors.
# This file cannot be decompressed with xzdec.
I="$srcdir/files/bad-3-corrupt_lzma2.xz"
if test -z "$XZ" || "$XZ" -dc --single-stream $NO_WARN "$I" > /dev/null; then
:
else
echo "Good first Stream failed xz with --single-stream: $I"
exit 1
fi
######### #########
# .lzma # # .lzma #

View File

@ -11,8 +11,6 @@
#include "tests.h" #include "tests.h"
#ifdef HAVE_MICROLZMA
#define BUFFER_SIZE 1024 #define BUFFER_SIZE 1024
#ifdef HAVE_ENCODER_LZMA1 #ifdef HAVE_ENCODER_LZMA1
@ -513,7 +511,6 @@ test_decode_bad_lzma_properties(void)
lzma_end(&strm); lzma_end(&strm);
} }
#endif #endif
#endif
extern int extern int
@ -521,17 +518,16 @@ main(int argc, char **argv)
{ {
tuktest_start(argc, argv); tuktest_start(argc, argv);
#ifndef HAVE_MICROLZMA #ifndef HAVE_ENCODER_LZMA1
tuktest_early_skip("MicroLZMA disabled"); tuktest_early_skip("LZMA1 encoder disabled");
#else #else
# ifdef HAVE_ENCODER_LZMA1
tuktest_run(test_encode_options); tuktest_run(test_encode_options);
tuktest_run(test_encode_basic); tuktest_run(test_encode_basic);
tuktest_run(test_encode_small_out); tuktest_run(test_encode_small_out);
tuktest_run(test_encode_actions); tuktest_run(test_encode_actions);
# endif
# if defined(HAVE_DECODER_LZMA1) && defined(HAVE_ENCODER_LZMA1) // MicroLZMA decoder tests require the basic encoder functionality.
# ifdef HAVE_DECODER_LZMA1
goodbye_world_encoded_size = basic_microlzma_encode(goodbye_world, goodbye_world_encoded_size = basic_microlzma_encode(goodbye_world,
ARRAY_SIZE(goodbye_world), &goodbye_world_encoded); ARRAY_SIZE(goodbye_world), &goodbye_world_encoded);