Compare commits

...

737 Commits
v5.4 ... master

Author SHA1 Message Date
Jia Tan af071ef770 Docs: Simplify SECURITY.md. 2024-03-26 03:38:12 +08:00
Lasse Collin 0b99783d63 liblzma: memcmplen.h: Add a comment why subtraction is used. 2024-03-22 17:46:30 +02:00
Lasse Collin 8a25ba024d INSTALL: Document arguments of --enable-symbol-versions. 2024-03-15 18:04:10 +02:00
Lasse Collin 49324b711f Build: Use only the generic symbol versioning with NVIDIA HPC Compiler.
This does the previous commit with CMake.

AC_EGREP_CPP uses AC_REQUIRE so the outermost if-commands must
be changed to AS_IF to ensure that things wont break some day.
See 5a5bd7f871.
2024-03-15 18:04:10 +02:00
Lasse Collin c273123ed0 CMake: Use only the generic symbol versioning with NVIDIA HPC Compiler.
It doesn't support the __symver__ attribute or __asm__(".symver ...").
The generic symbol versioning can still be used since it only needs
linker support.
2024-03-15 18:04:10 +02:00
Lasse Collin df7f487648 Update THANKS. 2024-03-15 18:04:10 +02:00
Lasse Collin 3217b82b3e liblzma: Minor comment edits. 2024-03-15 18:03:47 +02:00
Sergey Kosukhin 096bc0e3f8 liblzma: Fix building with NVHPC (NVIDIA HPC SDK).
NVHPC compiler has several issues that make it impossible to
build liblzma:
  - the compiler cannot handle unions that contain pointers that
    are not the first members;
  - the compiler cannot handle the assembler code in range_decoder.h
    (LZMA_RANGE_DECODER_CONFIG has to be set to zero);
  - the compiler fails to produce valid code for delta_decode if the
    vectorization is enabled, which results in failed tests.

This introduces NVHPC-specific workarounds that address the issues.
2024-03-15 17:30:50 +02:00
Lasse Collin 2ad7fad670 CMake: Disable symbol versioning on non-glibc Linux.
This better matches what configure.ac does. For example, musl has
only basic symbol versioning support:

https://wiki.musl-libc.org/functional-differences-from-glibc.html#Symbol_versioning

configure.ac tries to enable symbol versioning only with glibc
so now CMake does the same.
2024-03-15 17:30:50 +02:00
Lasse Collin 82f0c0d39e CMake: Make symbol versioning configurable. 2024-03-15 17:30:50 +02:00
Lasse Collin 45d33bfc45 Build: Style tweaks to configure.ac.
The AC_MSG_ERROR line is overlong anyway as are a few other
AC_MSG_ERROR lines already.
2024-03-13 22:00:23 +02:00
Sergey Kosukhin f56ed6fac6 Build: Let the users override the symbol versioning variant.
There are cases when the users want to decide themselves whether
they want to have the generic (even on GNU/Linux) or the linux
(even if we do not recommend that) symbol versioning variant.
The former might be needed to circumvent compiler issues (i.e.
the compiler does not support all features that are required
for the linux versioning), the latter might help in overriding
the assumptions made in the configure script.
2024-03-13 21:59:55 +02:00
Jia Tan a4f2e20d84 Add NEWS for 5.6.1 2024-03-09 11:27:27 +08:00
Jia Tan f01be8ad75 Translations: Add missing --riscv option to man page translations. 2024-03-09 10:43:20 +08:00
Jia Tan 6e636819e8 Tests: Update two test files.
The original files were generated with random local to my machine.
To better reproduce these files in the future, a constant seed was used
to recreate these files.
2024-03-09 10:18:29 +08:00
Jia Tan a3a29bbd5d Tests: Test --single-stream can decompress bad-3-corrupt_lzma2.xz.
The first stream in this file is valid, so this tests that xz properly
stops after decompressing it.
2024-03-09 10:08:32 +08:00
Jia Tan 0b4ccc9145 Tests: Update RISC-V test files.
This increases code coverage and tests for possible shifting bugs.
2024-03-09 10:05:32 +08:00
Jia Tan 8c9b8b2063 liblzma: Fix typos in crc32_fast.c and crc64_fast.c. 2024-03-09 09:52:32 +08:00
Jia Tan b93a8d7631 Tests: Replace HAVE_MICROLZMA usage in CMake and Autotools builds.
This reverts commit adaacafde6.
2024-03-09 09:49:55 +08:00
Jia Tan 82ecc53819 liblzma: Fix false Valgrind error report with GCC.
With GCC and a certain combination of flags, Valgrind will falsely
trigger an invalid write. This appears to be due to the omission of
instructions to properly save, set up, and restore the frame pointer.

The IFUNC resolver is a leaf function since it only calls a function
that is inlined. So sometimes GCC omits the frame pointer instructions
in the resolver unless this optimization is explictly disabled.

This fixes https://bugzilla.redhat.com/show_bug.cgi?id=2267598.
2024-03-09 09:20:57 +08:00
Lasse Collin 3007e74ef2 liblzma: Fix a typo in a comment in the RISC-V filter. 2024-03-05 23:23:16 +02:00
Jia Tan 72d2933bfa liblzma: Use attribute no_profile_instrument_function with ifunc.
Thanks to Sam James for determining this was the attribute needed to
workaround the GCC bug and for his version of the patch in Gentoo.
2024-03-05 01:54:30 +08:00
Jia Tan e5faaebbcf Build: Require attribute no_profile_instrument_function for ifunc usage.
Using __attribute__((__no_profile_instrument_function__)) on the ifunc
resolver works around a bug in GCC -fprofile-generate:
it adds profiling code even to ifunc resolvers which can make
the ifunc resolver crash at program startup. This attribute
was not introduced until GCC 7 and Clang 13, so ifunc won't
be used with prior versions of these compilers.

This bug was brought to our attention by:

    https://bugs.gentoo.org/925415

And was reported to upstream GCC by:

    https://gcc.gnu.org/bugzilla/show_bug.cgi?id=11411
2024-03-05 01:54:30 +08:00
Lasse Collin 7eeadd279a liblzma: Fix a comment in the RISC-V filter. 2024-03-04 19:23:18 +02:00
Lasse Collin 5f3d059529 CMake: Warn if translated man pages are missing. 2024-02-29 16:37:19 +02:00
Lasse Collin 4cd1042ee7 CMake: Warn if gettext tools and pre-created .gmo files are missing.
It's only done with CMake >= 3.20 and if library support
for translation was already found.

Sort of fixes: https://github.com/tukaani-project/xz/issues/82
2024-02-29 16:37:19 +02:00
Lasse Collin a94b42362c xz: Add comments. 2024-02-28 18:26:25 +02:00
Jia Tan bbf112e323 xz: Change logging level for thread reduction to highest verbosity only.
Now that multi threaded encoding is the default, users do not need to
see a warning message everytime the number of threads is reduced. On
some machines, this could happen very often. It is not unreasonable for
users to need to set double verbose mode to see this kind of
information.

To see these warning messages -vv or --verbose --verbose must be passed
to set xz into the highest possible verbosity mode.

These warnings had caused automated testing frameworks to fail when they
expected no output to stderr.

Thanks to Sebastian Andrzej Siewior for reporting this and for the
initial version of the patch.
2024-02-29 00:05:50 +08:00
Lasse Collin 649f644744 Fix sorting in THANKS. 2024-02-26 23:06:13 +02:00
Jia Tan 1255b7d849 Update THANKS. 2024-02-26 23:40:50 +08:00
Chien Wong eee579fff5 xz: Add missing RISC-V on the filter list in the man page
Signed-off-by: Chien Wong <m@xv97.com>
2024-02-26 23:36:04 +08:00
Jia Tan 328c52da8a Build: Fix Linux Landlock feature test in Autotools and CMake builds.
The previous Linux Landlock feature test assumed that having the
linux/landlock.h header file was enough. The new feature tests also
requires that prctl() and the required Landlock system calls are
supported.
2024-02-26 23:27:44 +08:00
Jia Tan eb8ad59e9b Tests: Add test_microlzma to .gitignore and CMakeLists.txt. 2024-02-26 20:06:10 +08:00
Jia Tan 9eed1b9a3a Tests: Correct license header in test_microlzma.c. 2024-02-26 19:56:25 +08:00
Jia Tan 8bf9f72ee1 Fix typos in NEWS and CMakeLists. 2024-02-25 21:41:55 +08:00
Jia Tan 5d8d915ebe Bump version and soname for 5.7.0alpha.
Like 5.5.0alpha, 5.7.0alpha won't be released, it's just to mark that
the branch is not stable.

Once again there is no API/ABI stability for new features in devel
versions. The major soname won't be bumped even if API/ABI of new
features breaks between devel releases.
2024-02-24 16:30:06 +08:00
Jia Tan a18fb1edef Add NEWS for 5.6.0. 2024-02-24 15:50:36 +08:00
Jia Tan 24355c5280 Translations: Remove obsolete and fuzzy matches from some translations.
The French and Brazilian Portuguese man page translations have not been
updated since the switch from public domain to 0BSD. The old GPLv2
strings have now been removed from these files.
2024-02-22 22:27:01 +08:00
Jia Tan 02ca4a7d7b Translations: Patch man pages to avoid fuzzy matches.
This will be fixed in the next round of translations, but this avoids
having a fuzzy match or not fixing the English version.
2024-02-21 00:31:54 +08:00
Jia Tan 898aad9fc7 xzmore: Fix typo in xzmore.1.
Thanks to Yuri Chornoivan.
2024-02-21 00:30:43 +08:00
Jia Tan 5631aa206c Translations: Update the Vietnamese translation. 2024-02-24 12:12:16 +08:00
Jia Tan a65fd7ce9d Translations: Update the Esperanto translation. 2024-02-24 12:06:40 +08:00
Jia Tan cf44e4b7f5 Tests: Add a few test files. 2024-02-23 23:09:59 +08:00
Jia Tan 39f4a1a86a Tests: Add MicroLZMA test. 2024-02-23 20:58:36 +08:00
Jia Tan adaacafde6 Build: Define HAVE_MICROLZMA when it is configured. 2024-02-23 20:57:59 +08:00
Jia Tan eea78216d2 xz: Fix Capsicum sandbox compile error.
user_abort_pipe[] was still being used instead of the parameters.
2024-02-23 20:27:15 +08:00
Jia Tan 32b0a3ce19 Build: Fix ARM64 CRC32 instruction feature test.
Old versions of Clang reported the unsupported function attribute and
__crc32d() function as warnings instead of errors, so the feature test
passed when it shouldn't have, causing a compile error at build time.
-Werror was added to this feature test to fix this. The change is not
needed for CMake because check_c_source_compiles() also performs
linking and the error is caught then.

Thanks to Sebastian Andrzej Siewior for reporting this.
2024-02-23 16:12:32 +08:00
Lasse Collin 4c81c9611f CMake: Add LOCALEDIR to the windres workaround.
LOCALEDIR may contain spaces like in "C:\Program Files".
2024-02-22 19:16:35 +02:00
Lasse Collin de4337fd89 xz: Landlock: Fix error message if input file is a directory.
If xz is given a directory, it should look like this:

    $ xz /usr/bin
    xz: /usr/bin: Is a directory, skipping

The Landlock rules didn't allow opening directories for reading:

    $ xz /usr/bin
    xz: /usr/bin: Permission denied

The simplest fix was to allow opening directories for reading.
While it's a bit silly to allow it solely for the error message,
it shouldn't make the sandbox significantly weaker.

The single-file use case (like when called from GNU tar) is
still as strict as possible: all Landlock restrictions are
enabled before (de)compression starts.
2024-02-22 15:18:25 +02:00
Lasse Collin 120da10ae1 liblzma: Disable branchless C version in range decoder.
Thanks to Sebastian Andrzej Siewior and Sam James for
benchmarking on various systems.
2024-02-22 14:41:29 +02:00
Lasse Collin 00440f52be INSTALL: Clarify that --disable-assembler affects only 32-bit x86. 2024-02-21 17:41:32 +02:00
Lasse Collin 11405be84e Windows: build.bash: Include COPYING.0BSD in the package. 2024-02-21 16:57:20 +02:00
Lasse Collin c27cf64e3e Windows: build.bash: include liblzma-crt-mixing.txt in the package. 2024-02-21 16:57:20 +02:00
Lasse Collin 8d38941bae Windows: Major update to Windows build instructions. 2024-02-21 16:57:20 +02:00
Lasse Collin 4b5b0d3523 Windows: Update windows/README-Windows.txt.
It's for binary packages built with windows/build.bash.
2024-02-21 16:57:20 +02:00
Lasse Collin 1ee716f740 Windows: Update windows/build.bash.
Support for the old MinGW was dropped. Only MinGW-w64 with GCC
is supported now.

The script now supports also cross-compilation from GNU/Linux
(tests are not run). MSYS2 and also the old MSYS 1.0.11 work
for building on Windows. The i686 and x86_64 toolchains must
be in PATH to build both 32-bit and 64-bit versions.

Parallel builds are done if "nproc" from GNU coreutils is available.

MinGW-w64 runtime copyright information file was renamed from
COPYING-Windows.txt to COPYING.MinGW-w64-runtime.txt which
is the filename used by MinGW-w64 itself. Its existence
is now mandatory, it's checked at the beginning of the script.

The file TODO is no longer copied to the package.
2024-02-20 21:04:29 +02:00
Jia Tan 60462e4260 Translations: Update the Romanian man page translations. 2024-02-20 23:32:22 +08:00
Jia Tan 10d733e5b8 Translations: Update the Korean man page translations. 2024-02-20 23:30:25 +08:00
Jia Tan 797a34b72a Translations: Update the Spanish translation. 2024-02-20 21:03:53 +08:00
Jia Tan 5c3751d019 Translations: Update the Romanian translation. 2024-02-20 20:18:07 +08:00
Jia Tan e2d31154ec Translations: Update the Croatian translation. 2024-02-20 20:15:50 +08:00
Jia Tan 704500f994 Translations: Update the German man page translations. 2024-02-20 20:05:44 +08:00
Jia Tan 1cfd3dca3f Translations: Update the German translation. 2024-02-20 19:58:25 +08:00
Jia Tan 28b9b3f16c Translations: Update the Hungarian translation. 2024-02-20 19:56:52 +08:00
Lasse Collin 00b06cd0af CMake: Fix building of lzmainfo when translations are enabled. 2024-02-19 16:48:18 +02:00
Lasse Collin b0d1422b60 CMake: Don't assume that -fvisibility=hidden is supported outside Windows.
The original code was good enough for supporting GNU/Linux
and a few others but it wasn't very portable.

CMake doesn't support Solaris Studio's -xldscope=hidden.
If it ever does, things should still work with this commit
as Solaris Studio supports not only its own __global but also
the GNU C __attribute__((visibility("default"))). Support for the
attribute was added in 2007 to Sun Studio 12 compiler version 5.9.
2024-02-19 16:28:49 +02:00
Lasse Collin 2ced9d34be CMake: Revise the component splitting. 2024-02-19 16:28:49 +02:00
Lasse Collin 426bdc709c CMake: Update the main comment and document CMAKE_BUILD_TYPE=Release. 2024-02-19 16:28:49 +02:00
Lasse Collin 4430e075f7 CMake: Use -O2 instead of -O3 in CMAKE_BUILD_TYPE=Release.
-O3 doesn't seem useful for speed but it makes the code bigger.
CMake makes is difficult for users to simply override the
optimization level: CFLAGS / CMAKE_C_FLAGS aren't helpful because
they go before CMAKE_C_FLAGS_RELEASE. Of course, users can override
CMAKE_C_FLAGS_RELEASE directly but then they have to remember to
add also -DNDEBUG to disable assertions.

This commit changes -O3 to -O2 in CMAKE_C_FLAGS_RELEASE if and only if
CMAKE_C_FLAGS_RELEASE cache variable doesn't already exist. So if
a custom value is passed on the command line (or reconfiguring an
already-configured build), the cache variable won't be modified.
2024-02-19 16:28:49 +02:00
Lasse Collin 025eb6d787 CMake: Handle symbol versioning on MicroBlaze specially.
This is to match configure.ac.
2024-02-19 16:28:49 +02:00
Lasse Collin 2edd1a35b2 CMake: Keep build working even if lib/*.[ch] are removed. 2024-02-19 12:21:37 +02:00
Lasse Collin d753e2ce47 CMake: Install documentation. 2024-02-19 12:21:37 +02:00
Lasse Collin 7a0405bea9 CMake: Bump maximum policy version to 3.28.
CMP0154 doesn't affect us since we don't use FILE_SET.
2024-02-19 12:21:37 +02:00
Lasse Collin c2264ffbe3 CMake: Build lzmainfo. 2024-02-19 12:21:37 +02:00
Lasse Collin 998d0b2953 CMake: Build lzmadec. 2024-02-19 12:21:37 +02:00
Lasse Collin 74e8bc7417 CMake: Add test_scripts.sh to the tests.
In contrast to Automake, skipping of this test when decoders
are disabled is handled at CMake side instead of test_scripts.sh
because CMake-build doesn't create config.h.
2024-02-19 12:21:37 +02:00
Lasse Collin 4808f238a7 CMake: Install scripts.
Compared to the Autotools-based build, this has simpler handling
for the shell (@POSIX_SHELL@) and extra PATH entry for the scripts
(configure has --enable-path-for-scripts=PREFIX). The simpler
metho should be enough for non-ancient systems and Solaris.
2024-02-19 12:21:37 +02:00
Lasse Collin 3462362ebd Scripts: Use @PACKAGE_VERSION@ instead of @VERSION@.
PACKAGE_VERSION was already used in liblzma.pc.in.
This way only one version @foo@ is used.
2024-02-19 12:21:37 +02:00
Lasse Collin 67610c245b CMake: Simplify symlink creation and install translated man pages.
It helps that cmake_install.cmake doesn't parallelize installation
so symlinks can be created so that the target is always known to
exist (a requirement on Windows in some cases).

This bumps the minimum CMake version from 3.13 to 3.14 to use
file(CREATE_LINK ...). It could be made to work on 3.13 by
calling "cmake -E create_symlink" but it's uglier code and
slower in "make install". 3.14 should be a reasonable version
to require nowadays, especially since the Autotools build
is still the primary build system for most OSes.
2024-02-19 12:21:37 +02:00
Lasse Collin 50cc1d8a5a CMake: Add support for building and installing xz with translations.
If gettext tools are available, the .po files listed in po/LINGUAS
are converted using msgfmt. This allows building with translations
directly from xz.git without Autotools.

If gettext tools aren't available, the Autotools-created .gmo files
in the "po" directory will be used. This allows CMake-based build
to use translations from Autotools-generated tarball.

If translation support is found (Intl_FOUND) but both the
gettext tools and the pre-generated .gmo files are missing,
then "make" will fail.
2024-02-19 12:21:37 +02:00
Lasse Collin 746c471643 liblzma: Remove commented-out code. 2024-02-19 11:58:33 +02:00
Lasse Collin 4ce300ce08 xz: Delete old commented-out code. 2024-02-17 23:07:35 +02:00
Lasse Collin cae9a5e0bf xz: Use stricter pledge(2) and Landlock sandbox.
This makes these sandboxing methods stricter when no files are
created or deleted. That is, it's a middle ground between the
initial sandbox and the strictest single-file-to-stdout sandbox:
this allows opening files for reading but output has to go to stdout.
2024-02-17 23:07:35 +02:00
Lasse Collin 02e3505991 xz: Support Landlock ABI version 4.
Linux 6.7 added support for ABI version 4 which restricts
TCP connections which xz won't need and thus those can be
forbidden now. Since the ABI version is handled at runtime,
supporting version 4 won't cause any compatibility issues.

Note that new enough kernel headers are required to get
version 4 support enabled at build time.
2024-02-17 23:07:35 +02:00
Lasse Collin 374868d81d xz: Move sandboxing code to sandbox.c and improve Landlock sandbox.
Landlock is now always used just like pledge(2) is: first in more
permissive mode and later (under certain common conditions) in
a strict mode that doesn't allow opening more files.

I put pledge(2) first in sandbox.c because it's the simplest API
to use and still somewhat fine-grained for basic applications.
So it's the simplest thing to understand for anyone reading sandbox.c.
2024-02-17 23:07:35 +02:00
Lasse Collin 7312dfbb02 xz: Tweak comments. 2024-02-17 23:07:35 +02:00
Lasse Collin c701a5909a xz: Fix message_init() description.
Also explicitly initialize progress_automatic to make it clear
that it can be read before message_init() sets it. Static variable
was initialized to false by default already so this is only for
clarity.
2024-02-17 23:07:35 +02:00
Lasse Collin 9466306719 Build: Makefile.am: Sort EXTRA_DIST.
Dirs first, then files in case-sensitive ASCII order.
2024-02-17 19:42:30 +02:00
Lasse Collin f3440e78c9 Build: Don't install TODO. 2024-02-17 19:42:30 +02:00
Jia Tan a7a3b62e2a Translations: Update the Korean man page translations. 2024-02-18 01:09:11 +08:00
Jia Tan 9b315db2d5 Translations: Update the Korean translation. 2024-02-18 01:08:32 +08:00
Lasse Collin 56246607df Build: Install translated lzmainfo man pages.
All other translated man pages were being installed but
lzmainfo had been forgotten.
2024-02-17 16:23:14 +02:00
Lasse Collin f1d6b88aef liblzma: Avoid implementation-defined behavior in the RISC-V filter.
GCC docs promise that it works and a few other compilers do
too. Clang/LLVM is documented source code only but unsurprisingly
it behaves the same as others on x86-64 at least. But the
certainly-portable way is good enough here so use that.
2024-02-17 16:01:32 +02:00
Lasse Collin 843ddc5f61 liblzma: Wrap a line exceeding 80 chars. 2024-02-17 15:50:21 +02:00
Sebastian Andrzej Siewior e9053c9072 liblzma/rangecoder: Exclude x32 from the x86-64 optimisation.
The x32 port has a x86-64 ABI in term of all registers but uses only
32bit pointer like x86-32. The assembly optimisation fails to compile on
x32. Given the state of x32 I suggest to exclude it from the
optimisation rather than trying to fix it.

Signed-off-by: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
2024-02-17 15:50:21 +02:00
Jia Tan 3d198fb13b Translations: Update the Spanish translation. 2024-02-17 21:05:07 +08:00
Jia Tan cf278bfe60 Translations: Update the Swedish translation. 2024-02-17 20:43:29 +08:00
Jia Tan b0f1a41be5 Translations: Update the Polish translation. 2024-02-17 20:41:38 +08:00
Jia Tan d74ed48b30 Translations: Update the Ukrainian translation. 2024-02-17 20:41:02 +08:00
Lasse Collin 711e22d5c5 Translations: Use the same sentence in xz.pot-header that the TP uses. 2024-02-16 17:53:34 +02:00
Jia Tan fb5f6aaf18 Fix typos discovered by codespell. 2024-02-16 22:54:59 +08:00
Jia Tan c64723bbb0 Translations: Update the Ukrainian man page translations. 2024-02-16 22:54:59 +08:00
Jia Tan 2895195ed0 Translations: Update the Ukrainian translation. 2024-02-16 22:54:59 +08:00
Lasse Collin 4c20781f4c Translations: Omit the generic copyright line from man page headers. 2024-02-15 22:33:49 +02:00
Jia Tan 4323bc3e0c Update m4/.gitignore. 2024-02-15 22:26:43 +08:00
Lasse Collin 5394a1665b Tests: tuktest.h: Treat Clang separately from GCC.
Don't assume that Clang defines __GNUC__ as the extensions
are available in clang-cl as well (and possibly in some other
Clang variants?).
2024-02-14 21:12:58 +02:00
Lasse Collin cce7330b9f Tests: tuktest.h: Add a missing word to a comment. 2024-02-14 21:11:03 +02:00
Lasse Collin 5dd8fc9452 Tests: tuktest.h: Fix the comment about STest. 2024-02-14 21:10:10 +02:00
Jia Tan 6f1790254a Bump version for 5.5.2beta. 2024-02-15 01:53:40 +08:00
Lasse Collin 924fdeedf4 liblzma: Fix validate_map.sh.
Adding the SPDX license identifier changed the line numbers.
2024-02-14 19:46:11 +02:00
Lasse Collin 22140a2df6 Build: Start the generated ChangeLog from around 5.4.0 instead of 5.2.0. 2024-02-14 19:38:34 +02:00
Lasse Collin 0b8cefa136 Fixed NEWS for 5.5.2beta. 2024-02-14 19:27:46 +02:00
Lasse Collin a4557bad96 liblzma: Silence warnings in --enable-small build. 2024-02-14 19:21:45 +02:00
Lasse Collin 38edf47323 Build: Install COPYING.0BSD as part of docs. 2024-02-14 19:15:58 +02:00
Lasse Collin b74e10bd83 Docs: List COPYING.0BSD in README. 2024-02-14 19:15:34 +02:00
Lasse Collin dfdb60ffe9 Docs: Include doc/examples/11_file_info.c in tarballs.
It was added in 2017 in c2e29f06a7
but it never got into any release tarballs because it was
forgotten to be added to Makefile.am.
2024-02-14 19:11:48 +02:00
Lasse Collin 160b686264 liblzma: Silence a warning. 2024-02-14 19:05:58 +02:00
Lasse Collin eeedd4d092 Add NEWS for 5.5.2beta. 2024-02-14 18:32:27 +02:00
Lasse Collin 8af7db854f xz: Mention lzmainfo if trying to use 'lzma --list'.
This kind of fixes the problem reported here:
https://bugs.launchpad.net/ubuntu/+source/xz-utils/+bug/1291020
2024-02-14 18:31:16 +02:00
Lasse Collin 0668907ff7 liblzma: Add comments. 2024-02-14 18:31:16 +02:00
Lasse Collin 109f1913d4 Scripts: Add lz4 support to xzgrep and xzdiff. 2024-02-14 18:31:16 +02:00
Lasse Collin de55485cb2 liblzma: Choose the range decoder variants using a bitmask macro. 2024-02-14 18:31:16 +02:00
Lasse Collin 0709c2b2d7 xz: Fix outdated threading related info on the man page. 2024-02-14 18:31:16 +02:00
Lasse Collin 3182a330c1 liblzma: Range decoder: Add x86-64 inline assembly.
It's compatible with GCC and Clang.
2024-02-14 18:31:16 +02:00
Lasse Collin cba2edc991 liblzma: Range decoder: Add branchless C code.
It's used only for basic bittrees and fixed-size reverse bittree
because those showed a clear benefit on x86-64 with GCC and Clang.
The other methods were more mixed and thus are commented out but
they should be tested on other archs.
2024-02-14 18:31:16 +02:00
Lasse Collin e290a72d6d liblzma: Clarify a comment. 2024-02-14 18:31:16 +02:00
Lasse Collin 5e04706b91 liblzma: LZMA decoder: Optimize loop comparison.
But now it needs one more local variable.
2024-02-14 18:31:16 +02:00
Lasse Collin 88276f9f2c liblzma: Optimize literal_subcoder() macro slightly. 2024-02-14 18:31:16 +02:00
Lasse Collin 5938f6de4d liblzma: LZ decoder: Add unlikely(). 2024-02-14 18:31:16 +02:00
Lasse Collin 9c252e3ed0 liblzma: LZ decoder: Remove a useless unlikely(). 2024-02-14 18:31:16 +02:00
Lasse Collin f3872a5947 liblzma: Optimize LZ decoder slightly.
Now extra buffer space is reserved so that repeating bytes for
any single match will never need to copy from two places (both
the beginning and the end of the buffer). This simplifies
dict_repeat() and helps a little with speed.

This seems to reduce .lzma decompression time about 2 %, so
with .xz and CRC it could be slightly less. The small things
add up still.
2024-02-14 18:31:16 +02:00
Lasse Collin eb518446e5 liblzma: LZMA decoder: Get rid of next_state[].
It's not completely obvious if this is better in the decoder.
It should be good if compiler can avoid creating a branch
(like using CMOV on x86).

This also makes lzma_encoder.c use the new macros.
2024-02-14 18:31:16 +02:00
Lasse Collin e0c0ee475c liblzma: LZMA decoder improvements.
This adds macros for bittree decoding which prepares the code
for alternative C versions and inline assembly.
2024-02-14 18:31:16 +02:00
Jia Tan de5c5e4176 liblzma: Creates Non-resumable and Resumable modes for lzma_decoder.
The new decoder resumes the first decoder loop in the Resumable mode.
Then, the code executes in Non-resumable mode until it detects that it
cannot guarantee to have enough input/output to decode another symbol.

The Resumable mode is how the decoder has always worked. Before decoding
every input bit, it checks if there is enough space and will save its
location to be resumed later. When the decoder has more input/output,
it jumps back to the correct sequence in the Resumable mode code.

When the input/output buffers are large, the Resumable mode is much
slower than the Non-resumable because it has more branches and is harder
for the compiler to optimize since it is in a large switch block.

Early benchmarking shows significant time improvement (8-10% on gcc and
clang x86) by using the Non-resumable code as much as possible.
2024-02-14 18:31:16 +02:00
Jia Tan e446ab7a18 liblzma: Creates separate "safe" range decoder mode.
The new "safe" range decoder mode is the same as old range decoder, but
now the default behavior of the range decoder will not check if there is
enough input or output to complete the operation. When the buffers are
close to fully consumed, the "safe" operations must be used instead. This
will improve speed because it will reduce the number of branches needed
for most of the range decoder operations.
2024-02-14 18:31:16 +02:00
Lasse Collin 7f6d9ca329 doxygen/footer.html: Add missing closing tags and don't open a new tab.
The footer template from Doxygen has the closing </body> </html>
as Doxygen doesn't add them otherwise.

target="_blank" was omitted as it's not useful here but
it can be slightly annoying as one cannot just go back
in the browser history.

Since the footer links to the license file in the same
directory and not to CC website, the rel attributes
can be omitted.
2024-02-14 18:31:16 +02:00
Lasse Collin 26d1527d34 Tweak the expressions in AUTHORS. 2024-02-14 18:31:16 +02:00
Lasse Collin d231d56580 Translations: Add the man page translators into man page header comment.
It looked odd to only have the original English authors listed
in the header comments of the translated files.
2024-02-14 18:31:16 +02:00
Lasse Collin 6d35fcb936 Translations: Translate also messages of lzmainfo.
lzmainfo has had translation support since 2009 at least but
it was never added to po/POTFILES.in so the messages weren't
translated. It's a very rarely needed tool so it's not too bad.

This also adds src/xz/mytime.c to po/POTFILES.in although there
are no translatable strings. It's simpler this way so that it
won't be forgotten if strings were ever added to that file.
2024-02-14 18:31:16 +02:00
Lasse Collin a9f369dd54 Translations: Add custom .pot header with SPDX license identifier.
The same is used for both po/xz.pot and po4a/xz-man.pot.
2024-02-14 18:31:16 +02:00
Lasse Collin 469cd6653b Translations: po4a/update-po: Add copyright notice to xz-man.pot.
All man pages are under 0BSD now so this is simple now.
2024-02-14 18:31:16 +02:00
Lasse Collin 28ce45e38f Update COPYING about the man pages of the scripts. 2024-02-14 18:31:16 +02:00
Lasse Collin e48287bf51 xzdiff, xzgrep, and xzmore: Rewrite the man pages.
The main reason is a kind of silly one:

xz-man.pot contains strings from all man pages in XZ Utils.
The man pages of xzdiff, xzgrep, and xzmore were under GPLv2
and the rest under 0BSD. Thus xz-man.pot contained strings
under two licences. po4a creates the translated man pages
from the combined 0BSD+GPLv2 xz-man.pot.

I haven't liked this mixing in xz-man.pot but the
Translation Project requires that all man pages must be
in the same .pot file. So a separate xz-man-gpl.pot
wasn't an option.

Since these man pages are short, rewriting them was quick enough.
Now xz-man.pot is entirely under 0BSD and marking the per-file
licenses is simpler.

As a bonus, some wording hopefully is now slightly better
although it's perhaps a matter of taste.

NOTE: In xzgrep.1, the EXIT STATUS section was written by me
in the commit d796b6d7fd so that's
why that section could be taken as is from the old xzgrep.1.
2024-02-14 18:31:16 +02:00
Lasse Collin 3e551b111b xzless: Update man page slightly.
The xz tool can decompress three file formats and xzless
has always supported uncompressed files too.
2024-02-14 18:31:16 +02:00
Lasse Collin 40f36da226 Translations: Change po/Makevars to add a copyright notice to po/xz.pot. 2024-02-14 18:31:16 +02:00
Lasse Collin 24192854e2 Translations: Update po/Makevars to use the template from gettext 0.22.4.
Also add SPDX license identifier now that there is a known license.
2024-02-14 18:31:16 +02:00
Lasse Collin b941549573 liblzma: Include the SPDX license identifier 0BSD to generated files.
Perhaps the generated files aren't even copyrightable but
using the same license for them as for the rest of the liblzma
keeps things more consistent for tools that look for license info.
2024-02-14 18:31:16 +02:00
Lasse Collin 8e4ec79483 liblzma: Fix compilation of price_tablegen.c.
It is built and run only manually so this didn't matter
unless one wanted to regenerate the price_table.c.
2024-02-14 18:31:16 +02:00
Lasse Collin e99bff3ffb Add SPDX license identifiers to GPL, LGPL, and FSFULLR files. 2024-02-14 18:31:16 +02:00
Lasse Collin 22af94128b Add SPDX license identifier into 0BSD source code files. 2024-02-14 18:31:16 +02:00
Lasse Collin 23de53421e liblzma: Sync the AUTHORS fix about SHA-256 to lzma.h. 2024-02-14 18:31:16 +02:00
Lasse Collin 689e0228ba Change most public domain parts to 0BSD.
Translations and doc/xz-file-format.txt and doc/lzma-file-format.txt
were not touched.

COPYING.0BSD was added.
2024-02-14 18:31:12 +02:00
Lasse Collin 76946dc433 Fix SHA-256 authors.
The initial commit 5d018dc035
in 2007 had a comment in sha256.c that the code is based on
Crypto++ Library 5.5.1. In 2009 the Authors list in sha256.c
and the AUTHORS file was updated with information that the
code had come from Crypto++ but via 7-Zip. I know I had viewed
7-Zip's SHA-256 code but back then the C code has been identical
enough with Crypto++, so I don't why I thought the author info
would need that extra step via 7-Zip for this single file.

Another error is that I had mixed sha.* and shacal2.* files
when checking for author info in Crypto++. The shacal2.* files
aren't related to liblzma's sha256.c and thus Kevin Springle's
code in Crypto++ isn't either.
2024-02-14 15:23:00 +02:00
Lasse Collin 21d9cbae9e Remove macosx/build.sh.
It was last updated in 2013.
2024-02-14 15:23:00 +02:00
Lasse Collin eac2c3c67f Doc: Remove doc/examples_old.
It was good to keep these around in parallel with the newer examples
but I think it's OK to remove the old ones at this point.
2024-02-14 15:23:00 +02:00
Jia Tan 89ea1a22f4 Tests: Add RISC-V filter support in a few places. 2024-02-13 23:33:27 +08:00
Jia Tan 45663443eb liblzma: Fix build error if only RISC-V BCJ filter is enabled.
If any other BCJ filter was enabled for encoding or decoding, then this
was not a problem.
2024-02-13 23:33:21 +08:00
Jia Tan 2f15597d67 Translations: Update the Korean translation. 2024-02-13 23:26:55 +08:00
Jia Tan df873143ad Translations: Update the Korean man page translations. 2024-02-13 01:55:53 +08:00
Jia Tan b3f415eddb Translations: Update the Chinese (simplified) translation. 2024-02-13 01:53:33 +08:00
Lasse Collin 9860d418d2 xzless: Use ||- in LESSOPEN with with "less" 451 and newer. 2024-02-09 23:21:01 +02:00
Lasse Collin fd0692b052 xzless: Use --show-preproc-errors with "less" 632 and newer.
This makes "less" show a warning if a decompression error occurred.
2024-02-09 23:00:05 +02:00
Jia Tan adb073da76 liblzma: Fix typo discovered by codespell. 2024-02-09 23:59:54 +08:00
Jia Tan 55d9fc883d Translations: Update the Swedish translation. 2024-02-09 20:01:06 +08:00
Jia Tan 55ba4a1ea3 Translations: Update the Spanish translation. 2024-02-08 20:09:04 +08:00
Jia Tan 7f2293cd80 Translations: Update the Spanish translation. 2024-02-07 21:34:35 +08:00
Jia Tan f4af2036bc Translations: Update the Polish translation. 2024-02-07 21:28:32 +08:00
Jia Tan e5e93bb816 Translations: Update the German translation. 2024-02-07 19:40:12 +08:00
Jia Tan 28f18ff8e2 Translations: Update the German man page translations. 2024-02-07 19:27:25 +08:00
Jia Tan cabfbc7947 Translations: Update the Romanian translation. 2024-02-06 23:44:06 +08:00
Jia Tan bf20c94f5d Translations: Update the Romanian man page translations. 2024-02-06 23:45:02 +08:00
Jia Tan 7c25ec9feb Translations: Update the Ukrainian translation. 2024-02-07 20:56:57 +08:00
Jia Tan b3523250e9 Translations: Update the Ukrainian man page translations. 2024-02-06 23:30:03 +08:00
Jia Tan a5c177f514 Update AUTHORS. 2024-02-02 01:39:28 +08:00
Jia Tan 7f68a68c19 liblzma: Update Authors list in crc32_arm64.h. 2024-02-02 01:38:51 +08:00
Jia Tan 97f9ba50b8 liblzma: Check HAVE_USABLE_CLMUL before omitting CRC32 table.
This was split from the prior commit so it could be easily applied to
the 5.4 branch.

Closes: https://github.com/tukaani-project/xz/pull/77
2024-02-01 20:09:11 +08:00
Jia Tan ca9015f4de liblzma: Check HAVE_USABLE_CLMUL before omitting CRC64 table.
If liblzma is configured with --disable-clmul-crc
CFLAGS="-msse4.1 -mpclmul", then it will fail to compile because the
generic version must be used but the CRC tables were not included.
2024-02-01 20:09:11 +08:00
Jia Tan 2f1552a91c liblzma: Only use ifunc in crcXX_fast.c if its needed.
The code was using HAVE_FUNC_ATTRIBUTE_IFUNC instead of CRC_USE_IFUNC.
With ARM64, ifunc is incompatible because it requires non-inline
function calls for runtime detection.
2024-02-01 20:09:11 +08:00
Jia Tan 30a25f3742 Docs: Add --disable-arm64-crc32 description to INSTALL. 2024-02-01 20:09:11 +08:00
Jia Tan 1940f0ec28 liblzma: Omit CRC tables when not needed with ARM64 optimizations.
This is similar to the existing x86-64 CLMUL conditions to omit the
tables. They were slightly refactored to improve readability.
2024-02-01 20:09:11 +08:00
Jia Tan 761f5b69a4 liblzma: Rename crc32_aarch64.h to crc32_arm64.h.
Even though the proper name for the architecture is aarch64, this
project uses ARM64 throughout. So the rename is for consistency.

Additionally, crc32_arm64.h was slightly refactored for the following
changes:

   * Added MSVC, FreeBSD, and macOS support in
     is_arch_extension_supported().

   * crc32_arch_optimized() now checks the size when aligning the
     buffer.

   * crc32_arch_optimized() loop conditions were slightly modified to
     avoid both decrementing the size and incrementing the buffer
     pointer.

   * Use the intrinsic wrappers defined in <arm_acle.h> because GCC and
     Clang name them differently.

   * Minor spacing and comment changes.
2024-02-01 20:09:11 +08:00
Jia Tan 455a08609c liblzma: Refactor crc_common.h.
The CRC_GENERIC is now split into CRC32_GENERIC and CRC64_GENERIC, since
the ARM64 optimizations will be different between CRC32 and CRC64.

For the same reason, CRC_ARCH_OPTIMIZED is split into
CRC32_ARCH_OPTIMIZED and CRC64_ARCH_OPTIMIZED.

ifunc will only be used with x86-64 CLMUL because the runtime detection
methods needed with ARM64 are not compatible with ifunc.
2024-02-01 20:09:11 +08:00
Jia Tan 61908e8160 CMake: Add support for ARM64 CRC32 instruction detection. 2024-02-01 20:09:11 +08:00
Jia Tan c5f6d79cc9 Build: Add support for ARM64 CRC32 instruction detection.
This adds --enable-arm64-crc32/--disable-arm64-crc32 (enabled by
default) for using the ARM64 CRC32 instruction. This can be disabled if
one knows the binary will never need to run on an ARM64 machine
with this instruction extension.
2024-02-01 20:09:09 +08:00
Chenxi Mao 849d0f282a Speed up CRC32 calculation on ARM64
The CRC32 instructions in ARM64 can calculate the CRC32 result
for 8 bytes in a single operation, making the use of ARM64
instructions much faster compared to the general CRC32 algorithm.

Optimized CRC32 will be enabled if ARM64 has CRC extension
running on Linux.

Signed-off-by: Chenxi Mao <chenxi.mao2013@gmail.com>
2024-01-27 21:49:26 +08:00
Jia Tan b43c3e48bf Bump version number for 5.5.1alpha. 2024-01-26 19:05:51 +08:00
Jia Tan c7a7ae1500 Add NEWS for 5.5.1alpha 2024-01-26 19:00:52 +08:00
Jia Tan 0ef8192e8d Add NEWS for 5.4.6. 2024-01-26 18:54:24 +08:00
Lasse Collin 93de7e751d Move doc/logo/xz-logo.png to "doc" and Doxygen footer to "doxygen".
The footer isn't a complete HTML file so having it in the doxygen
directory is a tiny bit clearer.
2024-01-24 20:00:57 +02:00
Jia Tan 00fa01698d README: Add COPYING.CC-BY-SA-4.0 entry to section 1.1.
The Overall documentation section (1.1) table spacing had to be adjusted
since the filename was very long.
2024-01-25 01:39:35 +08:00
Jia Tan e280470040 Build: Add the logo and license to the release. 2024-01-25 01:39:35 +08:00
Jia Tan b1ee6cf259 COPYING: Add the license for the XZ logo. 2024-01-25 01:39:29 +08:00
Jia Tan 31293ae707 Doxygen: Added the XZ logo and copyright information.
The PROJECT_LOGO field is now used to include the XZ logo. The footer
of each page now lists the copyright information instead of the default
footer. The license is also copied to statisfy the copyright and so the
link in the documentation can be local.
2024-01-25 01:06:01 +08:00
Lasse Collin 6daa4d0ea4 xz: Use threaded mode by defaut (as if --threads=0 was used).
This hopefully does more good than bad:

  + It's faster by default.

  + Only the threaded compressor creates files that
    can be decompressed in threaded mode.

  - Compression ratio is worse, usually not too much though.
    When it matters, -T1 must be used.

  - Memory usage increases.

  - Scripts that assume single-threaded mode but don't use -T1 will
    possibly use too much resources, for example, if they run
    multiple xz processes in parallel to compress multiple files.

  - Output from single-threaded and multi-threaded compressors
    differ but such changes could happen for other reasons too
    (they just haven't happened since 5.0.0).
2024-01-23 18:29:28 +02:00
Jia Tan a2dd2dc8e5 CI: Use RISC-V filter when building with BCJ support. 2024-01-23 23:55:44 +08:00
Jia Tan 3060e1070b Tests: Use smaller dictionary size in RISC-V test files. 2024-01-23 23:55:44 +08:00
Jia Tan 44ff2fa5c9 Tests: Skip RISC-V test files if decoder was not built. 2024-01-23 23:55:39 +08:00
Lasse Collin 6133a3f300 xz: Man page: Add more examples of LZMA2 options with BCJ filters. 2024-01-23 23:05:47 +08:00
Lasse Collin 50255feeaa liblzma: RISC-V filter: Use byte-by-byte access.
Not all RISC-V processors support fast unaligned access so
it's better to read only one byte in the main loop. This can
be faster even on x86-64 when compared to reading 32 bits at
a time as half the time the address is only 16-bit aligned.

The downside is larger code size on archs that do support
fast unaligned access.
2024-01-23 23:05:47 +08:00
Jia Tan db5eb5f563 xz: Update xz -lvv for RISC-V filter.
Version 5.6.0 will be shown, even though upcoming alphas and betas
will be able to support this filter. 5.6.0 looks nicer in the output and
people shouldn't be encouraged to use an unstable version in production
in any way.
2024-01-23 23:05:47 +08:00
Jia Tan e2870db5be Tests: Add two RISC-V Filter test files.
These test files achieve 100% code coverage in
src/liblzma/simple/riscv.c. They contain all of the instructions that
should be filtered and a few cases that should not.
2024-01-23 23:05:47 +08:00
Jia Tan b26a898693 xz: Update message in --long-help for RISC-V Filter. 2024-01-23 23:05:47 +08:00
Jia Tan 283f778908 xz: Update the man page for the RISC-V Filter.
A special note was added to suggest using four-byte alignment when the
compressed instruction extension is not present in a RISC-V binary.
2024-01-23 23:05:47 +08:00
Jia Tan ac3691ccca Tests: Add RISC-V Filter test in test_compress.sh. 2024-01-23 23:05:47 +08:00
Jia Tan 2959dbc735 liblzma: Update string_conversion.c to support RISC-V Filter. 2024-01-23 23:05:47 +08:00
Jia Tan 34372a5adb CMake: Support RISC-V BCJ Filter for encoding and decoding. 2024-01-23 23:05:47 +08:00
Jia Tan 440a2eccb0 liblzma: Add RISC-V BCJ filter.
The new Filter ID is 0x0B.

Thanks to Chien Wong <m@xv97.com> for the initial version of the Filter,
the xz CLI updates, and the Autotools build system modifications.

Thanks to Igor Pavlov for his many contributions to the design of
the filter.
2024-01-23 23:05:41 +08:00
Jia Tan 5540f4329b Docs: Update .xz file format specification to 1.2.0.
The new RISC-V filter was added to the specification, in addition to
updating the specification URL.
2024-01-19 23:08:14 +08:00
Jia Tan 22d86192f8 xz: Update website URLs in the man pages. 2024-01-19 23:08:14 +08:00
Jia Tan 6b63c4c613 liblzma: Update website URL. 2024-01-19 23:08:14 +08:00
Jia Tan fce4758018 Docs: Update website URLs. 2024-01-19 23:08:14 +08:00
Jia Tan c26812c5b2 Build: Update website URL. 2024-01-19 23:08:14 +08:00
Lasse Collin fbb3ce541e liblzma: CRC: Add a comment to crc_x86_clmul.h about BUILDING_ macros. 2024-01-11 15:25:00 +02:00
Lasse Collin 4f518c1b6b liblzma: CRC: Remove crc_always_inline, use lzma_always_inline instead.
Now crc_simd_body() in crc_x86_clmul.h is only called once
in a translation unit, we no longer need to be so cautious
about ensuring the always-inline behavior.
2024-01-11 15:24:35 +02:00
Lasse Collin 35c03ec6bf liblzma: CRC: Update CLMUL comments to more generic wording. 2024-01-11 14:39:46 +02:00
Lasse Collin 66f080e801 liblzma: Rename arch-specific CRC functions and macros.
CRC_CLMUL was split to CRC_ARCH_OPTIMIZED and CRC_X86_CLMUL.
CRC_ARCH_OPTIMIZED is defined when an arch-optimized version is used.
Currently the x86 CLMUL implementations are the only arch-optimized
versions, and these also use the CRC_x86_CLMUL macro to tell when
crc_x86_clmul.h needs to be included.

is_clmul_supported() was renamed to is_arch_extension_supported().
crc32_clmul() and crc64_clmul() were renamed to
crc32_arch_optimized() and crc64_arch_optimized().
This way the names make sense with arch-specific non-CLMUL
implementations as well.
2024-01-11 14:29:42 +02:00
Lasse Collin 3dbed75b0b liblzma: Fix a comment in crc_common.h. 2024-01-11 14:29:42 +02:00
Lasse Collin 419f55f9df liblzma: Avoid extern lzma_crc32_clmul() and lzma_crc64_clmul().
A CLMUL-only build will have the crcxx_clmul() inlined into
lzma_crcxx(). Previously a jump to the extern lzma_crcxx_clmul()
was needed. Notes about shared liblzma on ELF platforms:

  - On platforms that support ifunc and -fvisibility=hidden, this
    was silly because CLMUL-only build would have that single extra
    jump instruction of extra overhead.

  - On platforms that support neither -fvisibility=hidden nor linker
    version script (liblzma*.map), jumping to lzma_crcxx_clmul()
    would go via PLT so a few more instructions of overhead (still
    not a big issue but silly nevertheless).

There was a downside with static liblzma too: if an application only
needs lzma_crc64(), static linking would make the linker include the
CLMUL code for both CRC32 and CRC64 from crc_x86_clmul.o even though
the CRC32 code wouldn't be needed, thus increasing code size of the
executable (assuming that -ffunction-sections isn't used).

Also, now compilers are likely to inline crc_simd_body()
even if they don't support the always_inline attribute
(or MSVC's __forceinline). Quite possibly all compilers
that build the code do support such an attribute. But now
it likely isn't a problem even if the attribute wasn't supported.

Now all x86-specific stuff is in crc_x86_clmul.h. If other archs
The other archs can then have their own headers with their own
is_clmul_supported() and crcxx_clmul().

Another bonus is that the build system doesn't need to care if
crc_clmul.c is needed.

is_clmul_supported() stays as inline function as it's not needed
when doing a CLMUL-only build (avoids a warning about unused function).
2024-01-11 14:29:42 +02:00
Lasse Collin e3833e297d liblzma: crc_clmul.c: Add crc_attr_target macro.
This reduces the number of the complex #if directives.
2024-01-11 14:29:42 +02:00
Lasse Collin d164ac0e62 liblzma: Simplify existing cases with lzma_attr_no_sanitize_address. 2024-01-11 14:29:42 +02:00
Lasse Collin 9523c1300d liblzma: #define crc_attr_no_sanitize_address in crc_common.h. 2024-01-11 14:29:38 +02:00
Lasse Collin 93d144f093 liblzma: CRC: Add empty lines.
And remove one too.
2024-01-10 17:19:03 +02:00
Lasse Collin 0c7e854ffd liblzma: crc_clmul.c: Tidy up the location of MSVC pragma.
It makes no difference in practice.
2024-01-10 17:19:03 +02:00
Lasse Collin 15cf3f04f2 Update THANKS. 2023-12-28 17:17:39 +02:00
Lasse Collin cd64dd70d5 liblzma: Use 8-byte method in memcmplen.h on ARM64.
It requires fast unaligned access to 64-bit integers
and a fast instruction to count leading zeros in
a 64-bit integer (__builtin_ctzll()). This perhaps
should be enabled on some other archs too.

Thanks to Chenxi Mao for the original patch:
https://github.com/tukaani-project/xz/pull/75 (the first commit)
According to the numbers there, this may improve encoding
speed by about 3-5 %.

This enables the 8-byte method on MSVC ARM64 too which
should work but wasn't tested.
2023-12-28 17:17:39 +02:00
Lasse Collin 12c90c00f0 liblzma: Check also for __clang__ in memcmplen.h.
This change hopefully makes no practical difference as Clang
likely was detected via __GNUC__ or _MSC_VER already.
2023-12-28 17:17:39 +02:00
Jia Tan 133c5851eb Translations: Update the French translation. 2023-12-21 21:39:08 +08:00
Jia Tan 710cbc186c xz: Add a comment to Capsicum sandbox setup.
This comment is repeated in xzdec.c to help remind us why all the
capabilities are removed from stdin in certain situations.
2023-12-21 20:53:27 +08:00
Jia Tan 4e1c695676 Docs: Update --enable-sandbox option in INSTALL.
xzdec now also uses the sandbox when its configured.
2023-12-21 20:53:27 +08:00
Jia Tan ebddf20214 CMake: Move sandbox detection outside of xz section.
The sandbox is now enabled for xzdec as well, so it no longer belongs
in just the xz section. xz and xzdec are always built, except for older
MSVC versions, so there isn't a need to conditionally show the sandbox
configuration. CMake will do a little unecessary work on older MSVC
versions that can't build xz or xzdec, but this is a very small
downside.
2023-12-21 20:53:23 +08:00
Jia Tan 5feb09266f Build: Allow sandbox to be configured for just xzdec.
If xz is disabled, then xzdec can still use the sandbox.
2023-12-20 22:43:44 +08:00
Jia Tan d74fb5f060 xzdec: Add sandbox support for Pledge, Capsicum, and Landlock.
A very strict sandbox is used when the last file is decompressed. The
likely most common use case of xzdec is to decompress a single file.
The Pledge sandbox is applied to the entire process with slightly more
relaxed promises, until the last file is processed.

Thanks to Christian Weisgerber for the initial patch adding Pledge
sandboxing.
2023-12-19 21:18:28 +08:00
Jia Tan b34b6a9912 liblzma: Initialize lzma_lz_encoder pointers with NULL.
This fixes the recent change to lzma_lz_encoder that used memzero
instead of the NULL constant. On some compilers the NULL constant
(always 0) may not equal the NULL pointer (this only needs to guarentee
to not point to valid memory address).

Later code compares the pointers to the NULL pointer so we must
initialize them with the NULL pointer instead of 0 to guarentee
code correctness.
2023-12-20 21:38:39 +08:00
Jia Tan 183a62f0b5 liblzma: Set all values in lzma_lz_encoder to NULL after allocation.
The first member of lzma_lz_encoder doesn't necessarily need to be set
to NULL since it will always be set before anything tries to use it.
However the function pointer members must be set to NULL since other
functions rely on this NULL value to determine if this behavior is
supported or not.

This fixes a somewhat serious bug, where the options_update() and
set_out_limit() function pointers are not set to NULL. This seems to
have been forgotten since these function pointers were added many years
after the original two (code() and end()).

The problem is that by not setting this to NULL we are relying on the
memory allocation to zero things out if lzma_filters_update() is called
on a LZMA1 encoder. The function pointer for set_out_limit() is less
serious because there is not an API function that could call this in an
incorrect way. set_out_limit() is only called by the MicroLZMA encoder,
which must use LZMA1 where set_out_limit() is always set. Its currently
not possible to call set_out_limit() on an LZMA2 encoder at this time.

So calling lzma_filters_update() on an LZMA1 encoder had undefined
behavior since its possible that memory could be manipulated so the
options_update member pointed to a different instruction sequence.

This is unlikely to be a bug in an existing application since it relies
on calling lzma_filters_update() on an LZMA1 encoder in the first place.
For instance, it does not affect xz because lzma_filters_update() can
only be used when encoding to the .xz format.

This is fixed by using memzero() to set all members of lzma_lz_encoder
to NULL after it is allocated. This ensures this mistake will not occur
here in the future if any additional function pointers are added.
2023-12-16 20:51:38 +08:00
Jia Tan 1a1bb381db liblzma: Tweak a comment. 2023-12-16 20:30:55 +08:00
Jia Tan 55810780e0 liblzma: Make parameter names in function definition match declaration.
lzma_raw_encoder() and lzma_raw_encoder_init() used "options" as the
parameter name instead of "filters" (used by the declaration). "filters"
is more clear since the parameter represents the list of filters passed
to the raw encoder, each of which contains filter options.
2023-12-16 20:28:21 +08:00
Jia Tan 5dad6f628a liblzma: Improve lzma encoder init function consistency.
lzma_encoder_init() did not check for NULL options, but
lzma2_encoder_init() did. This is more of a code style improvement than
anything else to help make lzma_encoder_init() and lzma2_encoder_init()
more similar.
2023-12-16 20:18:47 +08:00
Jia Tan e1b1a9d637 Docs: Update repository URL in Changelog. 2023-12-16 11:20:20 +08:00
Jia Tan f9b82bc64a CI: Update Upload Artifact Action. 2023-12-15 16:56:31 +08:00
Jia Tan d0b24efe6c Tests: Silence -Wsign-conversion warning on GCC version < 10.
Since GCC version 10, GCC no longer complains about simple implicit
integer conversions with Arithmetic operators.

For instance:

    uint8_t a = 5;
    uint32_t b = a + 5;

Give a warning on GCC 9 and earlier but this:

    uint8_t a = 5;
    uint32_t b = (a + 5) * 2;

Gives a warning with GCC 10+.
2023-12-07 21:48:07 +08:00
Jia Tan 4a972a8ee3 Update THANKS. 2023-12-07 20:06:57 +08:00
Jia Tan ee2f483500 Tests: Minor cleanups to OSS-Fuzz files.
Most of these fixes are small typos and tweaks. A few were caused by bad
advice from me. Here is the summary of what is changed:

- Author line edits

- Small comment changes/additions

- Using the return value in the error messages in the fuzz targets'
  coder initialization code

- Removed fuzz_encode_stream.options. This set a max length, which may
  prevent some worthwhile code paths from being properly exercised.

- Removed the max_len option from fuzz_decode_stream.options for the
  same reason as fuzz_encode_stream. The alone decoder fuzz target still
  has this restriction.

- Altered the dictionary contents for fuzz_lzma.dict. Instead of keeping
  the properties static and varying the dictionary size, the properties
  are varied and the dictionary size is kept small. The dictionary size
  doesn't have much impact on the code paths but the properties do.

Closes: https://github.com/tukaani-project/xz/pull/73
2023-12-07 20:06:57 +08:00
Maksym Vatsyk 483bb90eec Tests: Add fuzz_encode_stream ossfuzz target.
This fuzz target handles .xz stream encoding. The first byte of input
is used to dynamically set the preset level in order to increase the
fuzz coverage of complex critical code paths.
2023-12-07 20:06:57 +08:00
Maksym Vatsyk 7ca8c9869d Tests: Add fuzz_decode_alone OSS-Fuzz target
This fuzz target that handles LZMA alone decoding. A new fuzz
dictionary .dict was also created with common LZMA header values to
help speed up the discovery of valid headers.
2023-12-07 20:06:57 +08:00
Maksym Vatsyk 37581a77ad Tests: Update OSS-Fuzz Makefile.
All .c files can be built as separate fuzz targets. This simplifies
the Makefile by allowing us to use wildcards instead of having a
Makefile target for each fuzz target.
2023-12-07 20:06:54 +08:00
Maksym Vatsyk 28ce6a1c2a Tests: Move common OSS-Fuzz target code to .h file. 2023-12-07 20:06:54 +08:00
Maksym Vatsyk bf0521ea15 Tests: Rename OSS-Fuzz files. 2023-12-07 20:06:51 +08:00
Jia Tan 685094b8e1 Update THANKS. 2023-11-30 23:10:43 +08:00
Kian-Meng Ang 3b3023e00b Tests: Fix typos 2023-11-30 23:08:05 +08:00
Kian-Meng Ang 424d46ead8 xz: Fix typo 2023-11-30 23:08:05 +08:00
Jia Tan 35558adf9c Update THANKS. 2023-11-30 20:41:00 +08:00
Jia Tan fd170e8557 CI: Test musl libc builds on Ubuntu runner. 2023-11-30 20:09:46 +08:00
Jia Tan db2b4aa068 CI: Allow ci_build.sh to set a different C compiler. 2023-11-30 20:09:46 +08:00
Jia Tan ff7badef53 CMake: Use consistent indentation with check_c_source_compiles(). 2023-11-30 20:09:46 +08:00
Jia Tan d4af167570 CMake: Change __attribute__((__ifunc__())) detection.
This renames ALLOW_ATTR_IFUNC to USE_ATTR_IFUNC and applies the ifunc
detection changes that were made to the Autotools build.

Fixes: https://github.com/tukaani-project/xz/issues/70
2023-11-30 20:07:34 +08:00
Jia Tan 20ecee40a0 Docs: Update INSTALL for --enable_ifunc change. 2023-11-30 20:05:09 +08:00
Jia Tan ffb456593d Build: Change --enable-ifunc handling.
Some compilers support __attribute__((__ifunc__())) even though the
dynamic linker does not. The compiler is able to create the binary
but it will fail on startup. So it is not enough to just test if
the attribute is supported.

The default value for enable_ifunc is now auto, which will attempt
to compile a program using __attribute__((__ifunc__())). There are
additional checks in this program if glibc is being used or if it
is running on FreeBSD.

Setting --enable-ifunc will skip this test and always enable
__attribute__((__ifunc__())), even if is not supported.
2023-11-30 20:04:42 +08:00
Lasse Collin 12b89bcc99 xz: Tweak a comment. 2023-11-23 17:39:10 +02:00
Jia Tan 2ab2e4b5a5 xz: Use is_tty() in message.c. 2023-11-23 22:40:27 +08:00
Jia Tan 584e3a258f xz: Create separate is_tty() function.
The new is_tty() will report if a file descriptor is a terminal or not.
On POSIX systems, it is a wrapper around isatty(). However, the native
Windows implementation of isatty() will return true for all character
devices, not just terminals. So is_tty() has a special case for Windows
so it can use alternative Windows API functions to determine if a file
descriptor is a terminal.

This fixes a bug with MSVC and MinGW-w64 builds that refused to read from
or write to non-terminal character devices because xz thought it was a
terminal. For instance:

    xz foo -c > /dev/null

would fail because /dev/null was assumed to be a terminal.
2023-11-23 22:40:20 +08:00
Jia Tan 6b05f827f5 tuklib_integer: Fix typo discovered by codespell.
Based on internet dictionary searches, 'choise' is an outdated spelling
of 'choice'.
2023-11-22 20:39:41 +08:00
Lasse Collin 659aca0d69 xz: Move the check for --suffix with --format=raw a few lines earlier.
Now it reads from argv[] instead of args->arg_names.
2023-11-18 01:56:09 +08:00
Jia Tan ca278eb2b7 Tests: Create test_suffix.sh.
This tests some complicated interactions with the --suffix= option.
The suffix option must be used with --format=raw, but can optionally
be used to override the default .xz suffix.

This test also verifies some recent bugs have been correctly solved
and to hopefully avoid further regressions in the future.
2023-11-18 01:56:05 +08:00
Jia Tan 2a732aba22 xz: Fix a bug with --files and --files0 in raw mode without a suffix.
The following command caused a segmentation fault:

    xz -Fraw --lzma1 --files=foo

when foo was a valid file. The usage of --files or --files0 was not
being checked when compressing or decompressing in raw mode without a
suffix. The suffix checking code was meant to validate that all files
to be processed are "-" (if not writing to standard out), meaning the
data is only coming from standard in. In this case, there were no file
names to check since --files and --files0 store their file name in a
different place.

Later code assumed the suffix was set and caused a segmentation fault.
Now, the above command results in an error.
2023-11-17 23:16:55 +08:00
Jia Tan 299920bab9 Tests: Fix typo in a comment. 2023-11-17 20:04:58 +08:00
Jia Tan f481523baa xz: Refactor suffix test with raw format.
The previous version set opt_stdout, but this caused an issue with
copying an input file to standard out when decompressing an unknown file
type. The following needs to result in an error:

    echo foo | xz -df

since -c, --stdout is not used. This fixes the previous error by not
setting opt_stdout.
2023-11-15 23:40:13 +08:00
Jia Tan 837ea40b1c xz: Move suffix check after stdout mode is detected.
This fixes a bug introduced in cc5aa9ab13
when the suffix check was initially moved. This caused a situation that
previously worked:

    echo foo | xz -Fraw --lzma1 | wc -c

to fail because the old code knew that this would write to standard out
so a suffix was not needed.
2023-11-14 20:27:46 +08:00
Jia Tan d4f4a4d040 xz: Detect when all data will be written to standard out earlier.
If the -c, --stdout argument is not used, then we can still detect when
the data will be written to standard out if all of the provided
filenames are "-" (denoting standard in) or if no filenames are
provided.
2023-11-14 20:27:04 +08:00
Jia Tan 2ade7246e7 liblzma: Add missing comments to lz_encoder.h. 2023-11-09 01:21:53 +08:00
Jia Tan 5fe1450603 Add NEWS for 5.4.5. 2023-11-01 20:58:58 +08:00
Lasse Collin 46007049cd liblzma: Fix compilation of fastpos_tablegen.c.
The macro lzma_attr_visibility_hidden has to be defined to make
fastpos.h usable. The visibility attribute is irrelevant to
fastpos_tablegen.c so simply #define the macro to an empty value.

fastpos_tablegen.c is never built by the included build systems
and so the problem wasn't noticed earlier. It's just a standalone
program for generating fastpos_table.c.

Fixes: https://github.com/tukaani-project/xz/pull/69
Thanks to GitHub user Jamaika1.
2023-10-31 21:41:09 +02:00
Jia Tan 148e20607e Build: Fix text wrapping in an output message. 2023-10-31 21:54:11 +08:00
Lasse Collin 8c36ab79cb liblzma: Add a note why crc_always_inline exists for now.
Solaris Studio is a possible example (not tested) which
supports the always_inline attribute but might not get
detected by the common.h #ifdefs.
2023-10-30 18:44:32 +02:00
Lasse Collin e7a86b94cd liblzma: Use lzma_always_inline in memcmplen.h. 2023-10-30 18:44:32 +02:00
Lasse Collin dcfe563299 liblzma: #define lzma_always_inline in common.h. 2023-10-30 18:44:32 +02:00
Lasse Collin 41113fe30a liblzma: Use lzma_attr_visibility_hidden on private extern declarations.
These variables are internal to liblzma and not exposed in the API.
2023-10-30 18:06:25 +02:00
Lasse Collin a2f5ca706a liblzma: #define lzma_attr_visibility_hidden in common.h.
In ELF shared libs:

-fvisibility=hidden affects definitions of symbols but not
declarations.[*] This doesn't affect direct calls to functions
inside liblzma as a linker can replace a call to lzma_foo@plt
with a call directly to lzma_foo when -fvisibility=hidden is used.

[*] It has to be like this because otherwise every installed
    header file would need to explictly set the symbol visibility
    to default.

When accessing extern variables that aren't defined in the
same translation unit, compiler assumes that the variable has
the default visibility and thus indirection is needed. Unlike
function calls, linker cannot optimize this.

Using __attribute__((__visibility__("hidden"))) with the extern
variable declarations tells the compiler that indirection isn't
needed because the definition is in the same shared library.

About 15+ years ago, someone told me that it would be good if
the CRC tables would be defined in the same translation unit
as the C code of the CRC functions. While I understood that it
could help a tiny amount, I didn't want to change the code because
a separate translation unit for the CRC tables was needed for the
x86 assembly code anyway. But when visibility attributes are
supported, simply marking the extern declaration with the
hidden attribute will get identical result. When there are only
a few affected variables, this is trivial to do. I wish I had
understood this back then already.
2023-10-30 18:03:39 +02:00
Lasse Collin 2c7ee92e44 liblzma: Refer to MinGW-w64 instead of MinGW in the API headers.
MinGW (formely a MinGW.org Project, later the MinGW.OSDN Project
at <https://osdn.net/projects/mingw/>) has GCC 9.2.0 as the
most recent GCC package (released 2021-02-02). The project might
still be alive but majority of people have switched to MinGW-w64.
Thus it seems clearer to refer to MinGW-w64 in our API headers too.
Building with MinGW is likely to still work but I haven't tested it
in the recent years.
2023-10-26 21:46:06 +03:00
Lasse Collin 597f49b614 CMake: Use -D_FILE_OFFSET_BITS=64 if (and only if) needed.
A CMake option LARGE_FILE_SUPPORT is created if and only if
-D_FILE_OFFSET_BITS=64 affects sizeof(off_t).

This is needed on many 32-bit platforms and even with 64-bit builds
with MinGW-w64 to get support for files larger than 2 GiB.
2023-10-26 21:46:06 +03:00
Lasse Collin 1bc548b821 CMake: Generate and install liblzma.pc if not using MSVC.
Autotools based build uses -pthread and thus adds it to Libs.private
in liblzma.pc. CMake doesn't use -pthread at all if pthread functions
are available in libc so Libs.private doesn't get -pthread either.
2023-10-26 21:46:06 +03:00
Lasse Collin 2add71966f CMake: Rearrange the PACKAGE_ variables.
The windres workaround now replaces spaces with \x20 so
the package name isn't repeated.

These changes will help with creation of liblzma.pc.
2023-10-26 21:46:06 +03:00
Lasse Collin a7d1b2825c liblzma: Add Cflags.private to liblzma.pc.in for MSYS2.
It properly adds -DLZMA_API_STATIC when compiling code that
will be linked against static liblzma. Having it there on
systems other than Windows does no harm.

See: https://www.msys2.org/docs/pkgconfig/
2023-10-26 21:46:06 +03:00
Lasse Collin 80e0750e39 CMake: Create liblzma.def when building liblzma.dll with MinGW-w64. 2023-10-26 21:46:06 +03:00
Lasse Collin 08d12595f4 CMake: Change one CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_LIST_DIR.
In this case they have identical values.
2023-10-26 21:46:06 +03:00
Lasse Collin e67aaf698d CMake/Windows: Fix the import library filename.
Both PREFIX and IMPORT_PERFIX have to be set to "" to get
liblzma.dll and liblzma.dll.a.
2023-10-26 18:58:21 +03:00
Lasse Collin 88588b1246 Build: Detect -fsanitize= in CFLAGS and incompatible build options.
Now configure will fail if -fsanitize= is found in CFLAGS
and sanitizer-incompatible ifunc or Landlock sandboxing
would be used. These are incompatible with one or more sanitizers.
It's simpler to reject all -fsanitize= uses instead of trying to
pass those that might not cause problems.

CMake-based build was updated similarly. It lets the configuration
finish (SEND_ERROR instead of FATAL_ERROR) so that both error
messages can be seen at once.
2023-10-25 20:18:04 +03:00
Jia Tan 5e3d890f88 CI: Disable sandboxing in fsanitize=address,undefined job.
The sandboxing on Linux now supports Landlock, which restricts all
supported filesystem actions after xz opens the files it needs. The
sandbox is only enabled when one file is input and we are writing to
standard out. With fsanitize=address,undefined, the instrumentation
needs to read additional files after the sandbox is in place. This
forces all xz based test to fail, so the sandbox must instead be
disabled.
2023-10-24 00:50:08 +08:00
Jia Tan b1408987ea CI: Allow disabling the sandbox in ci_build.sh. 2023-10-24 00:15:39 +08:00
Lasse Collin 91c435cf1c CMake: Don't shadow the cache entry ENABLE_THREADS with a normal variable.
Using set(ENABLE_THREADS "posix") is confusing because it sets
a new normal variable and leaves the cache entry with the same
name unchanged. The intent wasn't to change the cache entry so
this switches to a different variable name.
2023-10-22 19:03:52 +03:00
Lasse Collin fa1609eb93 Docs: Update INSTALL about sandboxing support. 2023-10-22 19:03:52 +03:00
Lasse Collin 8276c7f41c xz: Support basic sandboxing with Linux Landlock (ABI versions 1-3).
It is enabled only when decompressing one file to stdout,
similar to how Capsicum is used.

Landlock was added in Linux 5.13.
2023-10-22 19:03:52 +03:00
Lasse Collin 3a1e9fd031 CMake: Edit threading related messages.
It's mostly to change from "thread method" to "threading method".
2023-10-22 19:03:52 +03:00
Lasse Collin bf01135252 CMake: Use FATAL_ERROR if user-supplied options aren't understood.
This way typos are caught quickly and compounding error messages
are avoided (a single typo could cause more than one error).

This keeps using SEND_ERROR when the system is lacking a feature
(like threading library or sandboxing method). This way the whole
configuration log will be generated in case someone wishes to
report a problem upstream.
2023-10-22 19:03:52 +03:00
Lasse Collin 3f53870c24 CMake: Add sandboxing support. 2023-10-22 19:03:52 +03:00
Lasse Collin 2e2cd11535 Simplify detection of Capsicum support.
This removes support for FreeBSD 10.0 and 10.1 which used
<sys/capability.h> instead of <sys/capsicum.h>. Support for
FreeBSD 10.1 ended on 2016-12-31. So now FreeBSD >= 10.2 is
required to enable Capsicum support.

This also removes support for Capsicum on Linux (libcaprights)
which seems to have been unmaintained since 2017 and Linux 4.11:
https://github.com/google/capsicum-linux
2023-10-22 19:03:52 +03:00
Lasse Collin c57858b60e xz/Windows: Allow clock_gettime with POSIX threads.
If winpthreads are used for threading, it's OK to use clock_gettime()
from winpthreads too.
2023-10-22 18:59:45 +03:00
Lasse Collin dd32f628bb mythread.h: Make MYTHREAD_POSIX compatible with MinGW-w64's winpthreads.
This might be almost useless but it doesn't need much extra code either.
2023-10-22 18:59:45 +03:00
Lasse Collin 680e52cdd0 CMake: Check for clock_gettime() even on Windows.
This mirrors configure.ac although currently MinGW-w64 builds
don't use clock_gettime() even if it is found.
2023-10-22 18:59:45 +03:00
Lasse Collin 1c1a8c3ee4 Build: Check for clock_gettime() even if not using POSIX threads.
See the new comment in the code.

This also makes the check for clock_gettime() run with MinGW-w64
with which we don't want to use clock_gettime(). The previous
commit already took care of this situation.
2023-10-22 18:59:45 +03:00
Lasse Collin 46fd991cd2 xz/Windows: Ensure that clock_gettime() isn't used with MinGW-w64.
This commit alone doesn't change anything in the real-world:

  - configure.ac currently checks for clock_gettime() only
    when using pthreads.

  - CMakeLists.txt doesn't check for clock_gettime() on Windows.

So clock_gettime() wasn't used with MinGW-w64 before either.

clock_gettime() provides monotonic time and it's better than
gettimeofday() in this sense. But clock_gettime() is defined
in winpthreads, and liblzma or xz needs nothing else from
winpthreads. By avoiding clock_gettime(), we avoid the dependency on
libwinpthread-1.dll or the need to link against the static version.

As a bonus, GetTickCount64() and MinGW-w64's gettimeofday() can be
faster than clock_gettime(CLOCK_MONOTONIC, &tv). The resolution
is more than good enough for the progress indicator in xz.
2023-10-22 18:59:45 +03:00
Lasse Collin cdb4d91f24 xz/Windows: Use GetTickCount64() with MinGW-w64 if using Vista threads. 2023-10-22 18:59:45 +03:00
Jia Tan 988e09f27b liblzma: Move is_clmul_supported() back to crc_common.h.
This partially reverts creating crc_clmul.c
(8c0f9376f5) where is_clmul_supported()
was moved, extern'ed, and renamed to lzma_is_clmul_supported(). This
caused a problem when the function call to lzma_is_clmul_supported()
results in a call through the PLT. ifunc resolvers run very early in
the dynamic loading sequence, so the PLT may not be setup properly at
this point. Whether the PLT is used or not for
lzma_is_clmul_supported() depened upon the compiler-toolchain used and
flags.

In liblzma compiled with GCC, for instance, GCC will go through the PLT
for function calls internal to liblzma if the version scripts and
symbol visibility hiding are not used. If lazy-binding is disabled,
then it would have made any program linked with liblzma fail during
dynamic loading in the ifunc resolver.
2023-10-21 00:01:29 +08:00
Jia Tan 105c7ca90d Build: Remove check for COND_CHECK_CRC32 in check/Makefile.inc.
Currently crc32 is always enabled, so COND_CHECK_CRC32 must always be
set. Because of this, it makes the recent change to conditionally
compile check/crc_clmul.c appear wrong since that file has CLMUL
implementations for both CRC32 and CRC64.
2023-10-19 16:23:32 +08:00
Jia Tan 1397571704 CMake: Add ALLOW_CLMUL_CRC option to enable/disable CLMUL.
The option is enabled by default, but will only be visible to a user
listing cache variables or using a CMake GUI application if the
immintrin.h header file is found.

This mirrors our Autotools build --disable-clmul-crc functionality.
2023-10-19 16:09:01 +08:00
Jia Tan c60b25569d liblzma: Fix -fsanitize=address failure with crc_clmul functions.
After forcing crc_simd_body() to always be inlined it caused
-fsanitize=address to fail for lzma_crc32_clmul() and
lzma_crc64_clmul(). The __no_sanitize_address__ attribute was added
to lzma_crc32_clmul() and lzma_crc64_clmul(), but not removed from
crc_simd_body(). ASAN and inline functions behavior has changed over
the years for GCC specifically, so while strictly required we will
keep __attribute__((__no_sanitize_address__)) on crc_simd_body() in
case this becomes a requirement in the future.

Older GCC versions refuse to inline a function with ASAN if the
caller and callee do not agree on sanitization flags
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89124#c3). If the
function was forced to be inlined, it will not compile if the callee
function has __no_sanitize_address__ but the caller doesn't.
2023-10-19 01:15:20 +08:00
Lasse Collin 9a78971261 tuklib_integer: Update the CMake test for fast unaligned access. 2023-10-18 19:02:45 +03:00
Lasse Collin 2f81ac852b Build: Enabled unaligned access by default on PowerPC64LE and some RISC-V.
PowerPC64LE wasn't tested but it seems like a safe change.
POWER8 supports unaligned access in little endian mode. Testing
on godbolt.org shows that GCC uses unaligned access by default.

The RISC-V macro __riscv_misaligned_fast is very new and not
in any stable compiler release yet.

Documentation in INSTALL was updated to match.

Documentation about an autodetection bug when using ARM64 GCC
with -mstrict-align was added to INSTALL.

CMake files weren't updated yet.
2023-10-18 19:02:45 +03:00
Lasse Collin c8f715f1bc tuklib_integer: Revise unaligned reads and writes on strict-align archs.
In XZ Utils context this doesn't matter much because
unaligned reads and writes aren't used in hot code
when TUKLIB_FAST_UNALIGNED_ACCESS isn't #defined.
2023-10-18 19:02:45 +03:00
Lasse Collin 6828242735 tuklib_integer: Add missing write64be and write64le fallback functions. 2023-10-18 19:02:45 +03:00
Jia Tan 1c8884f0af liblzma: Set the MSVC optimization fix to only cover lzma_crc64_clmul().
After testing a 32-bit Release build on MSVC, only lzma_crc64_clmul()
has the bug. crc_simd_body() and lzma_crc32_clmul() do not need the
optimizations disabled.
2023-10-18 23:54:41 +08:00
Lasse Collin 5ce0f7a48b liblzma: CRC_USE_GENERIC_FOR_SMALL_INPUTS cannot be used with ifunc. 2023-10-18 23:54:41 +08:00
Lasse Collin 2773538049 liblzma: Include common.h in crc_common.h.
crc_common.h depends on common.h. The headers include common.h except
when there is a reason to not do so.
2023-10-18 23:54:41 +08:00
Jia Tan e13b7947b9 liblzma: Add include guards to crc_common.h. 2023-10-18 23:54:41 +08:00
Jia Tan 40abd88afc liblzma: Add the crc_always_inline macro to crc_simd_body().
Forcing this to be inline has a significant speed improvement at the
cost of a few repeated instructions. The compilers tested on did not
inline this function since it is large and is used twice in the same
translation unit.
2023-10-18 23:54:41 +08:00
Jia Tan a5966c276b liblzma: Create crc_always_inline macro.
This macro must be used instead of the inline keyword. On MSVC, it is
a replacement for __forceinline which is an MSVC specific keyword that
should not be used with inline (it will issue a warning if it is).

It does not use a build system check to determine if
__attribute__((__always_inline__)) since all compilers that can use
CLMUL extensions (except the special case for MSVC) should support this
attribute. If this assumption is incorrect then it will result in a bug
report instead of silently producing slow code.
2023-10-18 23:54:41 +08:00
Jia Tan 96b663f67c liblzma: Refactor CRC comments.
A detailed description of the three dispatch methods was added. Also,
duplicated comments now only appear in crc32_fast.c or were removed from
both crc32_fast.c and crc64_fast.c if they appeared in crc_clmul.c.
2023-10-18 23:54:41 +08:00
Jia Tan 8c0f9376f5 liblzma: Create crc_clmul.c.
Both crc32_clmul() and crc64_clmul() are now exported from
crc32_clmul.c as lzma_crc32_clmul() and lzma_crc64_clmul(). This
ensures that is_clmul_supported() (now lzma_is_clmul_supported()) is
not duplicated between crc32_fast.c and crc64_fast.c.

Also, it encapsulates the complexity of the CLMUL implementations into a
single file and reduces the complexity of crc32_fast.c and crc64_fast.c.
Before, CLMUL code was present in crc32_fast.c, crc64_fast.c, and
crc_common.h.

During the conversion, various cleanups were applied to code (thanks to
Lasse Collin) including:

- Require using semicolons with MASK_/L/H/LH macros.
- Variable typing and const handling improvements.
- Improvements to comments.
- Fixes to the pragmas used.
- Removed unneeded variables.
- Whitespace improvements.
- Fixed CRC_USE_GENERIC_FOR_SMALL_INPUTS handling.
- Silenced warnings and removed the need for some #pragmas
2023-10-18 23:54:36 +08:00
Jia Tan a3ebc2c516 liblzma: Define CRC_USE_IFUNC in crc_common.h.
When ifunc is supported, we can define a simpler macro instead of
repeating the more complex check in both crc32_fast.c and crc64_fast.c.
2023-10-18 20:41:11 +08:00
Hans Jansen f1cd9d7194 liblzma: Added crc32_clmul to crc32_fast.c. 2023-10-13 20:54:05 +08:00
Hans Jansen 93e6fb08b2 liblzma: Moved CLMUL CRC logic to crc_common.h.
crc64_fast.c was updated to use the code from crc_common.h instead.
2023-10-13 20:54:05 +08:00
Hans Jansen 233885a437 liblzma: Rename crc_macros.h to crc_common.h. 2023-10-13 20:54:05 +08:00
Gabriela Gutierrez 37947d4a75 CI: Bump and ref actions by commit SHA in windows-ci.yml
Referencing actions by commit SHA in GitHub workflows guarantees you are using an immutable version. Actions referenced by tags and branches are more vulnerable to attacks, such as the tag being moved to a malicious commit or a malicious commit being pushed to the branch.

It's important to make sure the SHA's are from the original repositories and not forks.

For reference:

https://github.com/msys2/setup-msys2/releases/tag/v2.20.1
27b3aa77f6

https://github.com/actions/checkout/releases/tag/v4.1.0
8ade135a41

https://github.com/actions/upload-artifact/releases/tag/v3.1.3
a8a3f3ad30

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2023-10-13 20:03:13 +08:00
Gabriela Gutierrez f28cc9bd48 CI: Bump and ref actions by commit SHA in ci.yml
Referencing actions by commit SHA in GitHub workflows guarantees you are using an immutable version. Actions referenced by tags and branches are more vulnerable to attacks, such as the tag being moved to a malicious commit or a malicious commit being pushed to the branch.

It's important to make sure the SHA's are from the original repositories and not forks.

For reference:

https://github.com/actions/checkout/releases/tag/v4.1.0
8ade135a41

https://github.com/actions/upload-artifact/releases/tag/v3.1.3
a8a3f3ad30

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2023-10-13 20:03:13 +08:00
Jia Tan f74f174006 Build: Update visibility.m4 from Gnulib.
Updating from version 6 -> 8 from upstream. Declarations for variables
and function bodies were added to avoid unnecessary failures with
-Werror.
2023-10-12 20:12:18 +08:00
Lasse Collin 5c4bca521e Update THANKS. 2023-10-06 19:36:35 +03:00
Lasse Collin d91cb6e884 CMake/Windows: Fix when the windres workaround is applied.
CMake doesn't set WIN32 on CYGWIN but the workaround is
probably needed on Cygwin too. Same for MSYS and MSYS2.

The workaround must not be used with Clang that is acting in
MSVC mode. This fixes it by checking for the known environments
that need the workaround instead of using "NOT MSVC".

Thanks to Martin Storsjö.
0570308ddd (commitcomment-129098431)
2023-10-06 19:32:06 +03:00
Jia Tan 01e34aa117 CI: Disable CLANG64 MSYS2 environment until bug is resolved.
lld 17.0.1 searches for libraries to link first in the toolchain
directories before the local directory when building. The is a problem
for us because liblzma.a is installed in MSYS2 CLANG64 by default and
xz.exe will thus use the installed library instead of the one being
built.

This causes tests to fail when they are expecting features to be
disabled. More importantly, it will compile xz.exe with an incorrect
liblzma and could cause unexpected behavior by being unable to update
liblzma code in static builds. The CLANG64 environment can be tested
again once this is fixed.

Link to bug: https://github.com/llvm/llvm-project/issues/67779.
2023-09-29 22:11:54 +08:00
Jia Tan 30d0c35327 CMake: Rename xz and man page symlink custom targets.
The Ninja Generator for CMake cannot have a custom target and its
BYPRODUCTS have the same name. This has prevented Ninja builds on
Unix-like systems since the xz symlinks were introduced in
80a1a8bb83.
2023-09-29 20:14:39 +08:00
Jia Tan 506d03127a CMake: Specify LINKER_LANGUAGE for libgnu target to fix Ninja Generator.
CMake is unable to guess the linker language for just a header file so
it must be explicitly set.
2023-09-29 19:58:44 +08:00
Lasse Collin 0570308ddd CMake: Fix Windows build with Clang/LLVM 17.
llvm-windres 17.0.0 has more accurate emulation of GNU windres, so
the hack for GNU windres must now be used with llvm-windres too.

LLVM 16.0.6 has the old behavior and there likely won't be more
16.x releases. So we can simply check for >= 17.0.0.

See also:
2bcc0fdc58
2023-09-27 20:49:46 +03:00
Lasse Collin 5a9af95f85 liblzma: Update a comment.
The C standards don't allow an empty translation unit which can be
avoided by declaring something, without exporting any symbols.

When I committed f644473a21 I had
a feeling that some specific toolchain somewhere didn't like
empty object files (assembler or maybe "ar" complained) but
I cannot find anything to confirm this now. Quite likely I
remembered nonsense. I leave this here as a note to my future self. :-)
2023-09-26 21:47:13 +03:00
Jia Tan 8ebaf3f665 liblzma: Avoid compiler warning without creating extra symbol.
When the generic fast crc64 method is used, then we omit
lzma_crc64_table[][]. Similar to
d9166b52cf, we can avoid compiler warnings
with -Wempty-translation-unit (Clang) or -pedantic (GCC) by creating a
never used typedef instead of an extra symbol.
2023-09-27 00:04:40 +08:00
Lasse Collin 092d21db2e Build: Update the comment about -Werror usage in checks. 2023-09-26 17:24:15 +03:00
Lasse Collin a37a276338 Build: Fix __attribute__((ifunc(...))) detection with clang -Wall.
Now if user-supplied CFLAGS contains -Wall -Wextra -Wpedantic
the two checks that need -Werror will still work.

At CMake side there is add_compile_options(-Wall -Wextra)
but it didn't affect the -Werror tests. So with both Autotools
and CMake only user-supplied CFLAGS could make the checks fail
when they shouldn't.

This is not a full fix as things like -Wunused-macros in
user-supplied CFLAGS will still cause problems with both
GCC and Clang.
2023-09-26 15:00:43 +03:00
Lasse Collin 9c42f93693 Build: Fix underquoted AC_LANG_SOURCE.
It made no practical difference in this case.
2023-09-26 13:51:31 +03:00
Lasse Collin 9f1444a8a5 Build: Silence two Autoconf warnings.
There were two uses of AC_COMPILE_IFELSE that didn't use
AC_LANG_SOURCE and Autoconf warned about these. The omission
had been intentional but it turned out that this didn't do
what I thought it would.

Autoconf 2.71 manual gives an impression that AC_LANG_SOURCE
inserts all #defines that have been made with AC_DEFINE so
far (confdefs.h). The idea was that omitting AC_LANG_SOURCE
would mean that only the exact code included in the
AC_COMPILE_IFELSE call would be compiled.

With C programs this is not true: the #defines get added without
AC_LANG_SOURCE too. There seems to be no neat way to avoid this.
Thus, with the C language at least, adding AC_LANG_SOURCE makes
no other difference than silencing a warning from Autoconf. The
generated "configure" remains identical. (Docs of AC_LANG_CONFTEST
say that the #defines have been inserted since Autoconf 2.63b and
that AC_COMPILE_IFELSE uses AC_LANG_CONFTEST. So the behavior is
documented if one also reads the docs of macros that one isn't
calling directly.)

Any extra code, including #defines, can cause problems for
these two tests because these tests must use -Werror.
CC=clang CFLAGS=-Weverything is the most extreme example.
It enables -Wreserved-macro-identifier which warns about
#define __EXTENSIONS__ 1 because it begins with two underscores.
It's possible to write a test file that passes -Weverything but
it becomes impossible when Autoconf inserts confdefs.h.

So this commit adds AC_LANG_SOURCE to silence Autoconf warnings.
A different solution is needed for -Werror tests.
2023-09-26 13:50:19 +03:00
Jia Tan 519e47c281 CMake: Remove accidental extra newline. 2023-09-26 01:17:11 +08:00
Jia Tan bbb42412da Build: Remove Gnulib dependency from tests.
The tests do not use any Gnulib replacements so they do not need to link
libgnu.a or have /lib in the include path.
2023-09-26 00:47:26 +08:00
Jia Tan d265f6b756 CMake: Remove /lib from tests include path.
The tests never included anything from /lib, so this was not needed.
2023-09-26 00:43:43 +08:00
Jia Tan 9fb5de41f2 Scripts: Change quoting style from `...' to '...'. 2023-09-24 22:10:41 +08:00
Jia Tan eaebdef4d4 xz: Change quoting style from `...' to '...'. 2023-09-24 22:10:18 +08:00
Jia Tan f6667702bf liblzma: Change quoting style from `...' to '...'.
This was done for both internal and API headers.
2023-09-24 22:09:47 +08:00
Jia Tan be012b8097 Build: Change quoting style from `...' to '...'. 2023-09-24 22:09:16 +08:00
Jia Tan ce162db07f Docs: Change quoting style from `...' to '...'.
These days the ` and ' do not look symmetric. This quoting style has
been changed in various apps over the years including the GNU tools.
2023-09-24 22:05:02 +08:00
Jia Tan db17656721 lib: Silence -Wsign-conversion in getopt.c. 2023-09-24 21:25:01 +08:00
Jia Tan a6234f677d Build: Update getopt.m4 from Gnulib.
This file was modified from upstream since we do not need to replace
getopt() and can avoid complexity and feature tests.
2023-09-24 20:48:52 +08:00
Jia Tan 84808b68f1 CMake: Add /lib to include path. 2023-09-26 00:09:53 +08:00
Jia Tan 01804a0b4b CMake: Update libgnu target with new header files. 2023-09-24 20:36:34 +08:00
Jia Tan d34558388f lib: Update Makefile.am for new header files. 2023-09-23 00:47:52 +08:00
Jia Tan 52bf644bdf lib: Update getopt1.c from Gnulib.
The only difference was maintaining the conditional inclusion for
config.h.
2023-09-24 20:34:03 +08:00
Jia Tan 7e884c00d0 lib: Update getopt.in.h from Gnulib with modifications.
We can still avoid modifying the contents of this file during
configuration to simplify the build systems. Gnulib added replacements
for inclusions guards for Cygwin. Cygwin should not need getopt_long
replacement so this feature can be omitted.

<unistd.h> is conditionally included to avoid MSVC since it is not
available.

The definition for _GL_ARG_NONNULL was also copied into this file from
Gnulib since this stage is usually done during gnulib-tool.
2023-09-23 03:27:00 +08:00
Jia Tan cff05f8206 lib: Update getopt_int.h from Gnulib. 2023-09-23 00:31:55 +08:00
Jia Tan 04bd86a4b0 lib: Update getopt.c from Gnulib with modifications.
The code maintains the prior modifications of conditionally including
config.h and disabling NLS support.

_GL_UNUSED is repalced with the simple cast to void trick. _GL_UNUSED
is only used for these two parameters so its simpler than having to
define it.
2023-09-23 00:27:23 +08:00
Jia Tan 56b42be728 lib: Add getopt-cdefs.h for getopt_long update.
This was modified slightly from Gnulib. In Gnulib, it expects the
@HAVE_SYS_CDEFS_H@ to be replaced. Instead, we can set HAVE_SYS_CDEFS_H
on systems that have it and avoid copying another file into the build
directory. Since we are not using gnulib-tool, copying extra files
requires extra build system updates (and special handling with CMake) so
we should avoid when possible.
2023-09-23 00:18:56 +08:00
Jia Tan 9834e591a4 lib: Copy new header files from Gnulib without modification.
The getopt related files have changed from Gnulib by splitting up
getopt.in.h into more modular header files. We could have kept
everything in just getopt.in.h, but this will help us continue to update
in the future.
2023-09-23 00:15:25 +08:00
Lasse Collin 5b7a6f06e9 Windows: Update the version requirement comments from Win95 to W2k. 2023-09-24 17:48:13 +03:00
Lasse Collin e582f8e0fe tuklib_physmem: Comment out support for Windows versions older than 2000. 2023-09-24 17:48:13 +03:00
Lasse Collin 7d73d1f0e0 sysdefs.h: Update the comment about __USE_MINGW_ANSI_STDIO. 2023-09-24 16:32:32 +03:00
Lasse Collin 2a9929af0a xz: Windows: Don't (de)compress to special files like "con" or "nul".
Before this commit, the following writes "foo" to the
console and deletes the input file:

    echo foo | xz > con_xz
    xz --suffix=_xz --decompress con_xz

It cannot happen without --suffix because names like con.xz
are also special and so attempting to decompress con.xz
(or compress con to con.xz) will already fail when opening
the input file.

Similar thing is possible when compressing. The following
writes to "nul" and the input file "n" is deleted.

    echo foo | xz > n
    xz --suffix=ul n

Now xz checks if the destination is a special file before
continuing. DOS/DJGPP version had a check for this but
Windows (and OS/2) didn't.
2023-09-22 20:16:40 +03:00
Lasse Collin 01311b81f0 CMake: Wrap two overlong lines that are possible to wrap. 2023-09-22 20:10:10 +03:00
Lasse Collin 152d0771dd CMake: Add a comment about threads on Cygwin. 2023-09-22 20:10:10 +03:00
Lasse Collin 6df988ccef MSVC: Remove Visual Studio project files and update INSTALL-MSVC.txt.
CMake is now the preferred build file generator when building
with MSVC.
2023-09-22 20:10:10 +03:00
Lasse Collin edd563daf0 CMake: Require VS2015 or later for building xzdec.
xzdec might build with VS2013 but it hasn't been tested.
It was never supported before and VS2013 is old anyway
so for simplicity only liblzma is supported with VS2013.
2023-09-22 20:10:10 +03:00
Lasse Collin daea64d158 CMake: Allow building xz with Visual Studio 2015 and later.
Building the command line tools xz and xzdec with the combination
of CMake + Visual Studio 2015/2017/2019/2022 works now.

VS2013 update 2 should still be able to build liblzma.
VS2013 cannot build the xz command line tool because xz
needs snprintf() that roughly conforms to C99.
VS2013 is old and no extra code will be added to support it.

Thanks to Kelvin Lee and Jia Tan for testing.
2023-09-22 20:06:27 +03:00
Lasse Collin 8c2d197c94 MSVC: #define inline and restrict only when needed.
This also drops the check for _WIN32 as that shouldn't be needed.
2023-09-22 20:06:27 +03:00
Lasse Collin af66cd5859 CMake: Add support for replacement getopt_long (lib/getopt*).
Thanks to Jia Tan for the initial work. I added the libgnu target
and made a few related minor edits.
2023-09-22 20:06:27 +03:00
Lasse Collin e3288fdb45 CMake: Bump maximum policy version to 3.27.
There are several new policies. CMP0149 may affect the Windows SDK
version that CMake will choose by default. The new behavior is more
predictable, always choosing the latest SDK version by default.

The other new policies shouldn't affect this package.
2023-09-22 20:06:27 +03:00
Lasse Collin aff1b479c7 lib/getopt*.c: Include <config.h> only HAVE_CONFIG_H is defined.
The CMake-based build doesn't use config.h.

Up-to-date getopt_long in Gnulib is LGPLv2 so at some
point it could be included in XZ Utils too but for now
this commit is enough to make CMake-based build possible.
2023-09-22 20:06:27 +03:00
Lasse Collin aa0cd585d2 Doxygen: Add more C macro names to PREDEFINED. 2023-09-22 20:06:27 +03:00
Lasse Collin ee7709bae5 liblzma: Move a few __attribute__ uses in function declarations.
The API headers have many attributes but these were left
as is for now.
2023-09-22 20:06:27 +03:00
Lasse Collin 217958d887 xz, xzdec, lzmainfo: Use tuklib_attr_noreturn.
For compatibility with C23's [[noreturn]], tuklib_attr_noreturn
must be at the beginning of declaration (before "extern" or
"static", and even before any GNU C's __attribute__).

This commit also moves all other function attributes to
the beginning of function declarations. "extern" is kept
at the beginning of a line so the attributes are listed on
separate lines before "extern" or "static".
2023-09-22 20:06:27 +03:00
Lasse Collin 18a66fbac0 Remove incorrect uses of __attribute__((__malloc__)).
xrealloc() is obviously incorrect, modern GCC docs even
mention realloc() as an example where this attribute
cannot be used.

liblzma's lzma_alloc() and lzma_alloc_zero() would be
correct uses most of the time but custom allocators
may use a memory pool or otherwise hold the pointer
so aliasing issues could happen in theory.

The xstrdup() case likely was correct but I removed it anyway.
Now there are no __malloc__ attributes left in the code.
The allocations aren't in hot paths so this should make
no practical difference.
2023-09-22 20:06:27 +03:00
Lasse Collin 74b0e900c9 Build: Omit -Wc99-c11-compat since it warns about _Noreturn. 2023-09-22 20:06:26 +03:00
Lasse Collin 90c94dddfd tuklib: Update tuklib_attr_noreturn for C11/C17 and C23.
This makes no difference for GCC or Clang as they support
GNU C's __attribute__((__noreturn__)) but this helps with MSVC:

  - VS 2019 version 16.7 and later support _Noreturn if the
    options /std:c11 or /std:c17 are used. This gets handled
    with the check for __STDC_VERSION__ >= 201112.

  - When MSVC isn't in C11/C17 mode, __declspec(noreturn) is used.

C23 will deprecate _Noreturn (and <stdnoreturn.h>)
for [[noreturn]]. This commit anticipates that but
the final __STDC_VERSION__ value isn't known yet.
2023-09-22 20:06:21 +03:00
Lasse Collin 189f725813 Update THANKS. 2023-09-22 20:00:38 +03:00
Lasse Collin 79334e7f20 MSVC: xz: Make file_io.c and file_io.h compatible with MSVC.
Thanks to Kelvin Lee for the original patches
and testing the modifications I made.
2023-09-22 20:00:38 +03:00
Lasse Collin c660b8d78b MSVC: xz: Use GetTickCount64() to implement mytime_now().
It's available since Windows Vista.
2023-09-22 20:00:38 +03:00
Kelvin Lee 5c6f892d41 MSVC: xz: Use _stricmp() instead of strcasecmp() in suffix.c. 2023-09-22 20:00:38 +03:00
Kelvin Lee e241051f50 MSVC: xz: Use _isatty() from <io.h> to implement isatty(). 2023-09-22 20:00:38 +03:00
Kelvin Lee d14bba8fc2 MSVC: xz: Use _fileno() instead of fileno(). 2023-09-22 20:00:38 +03:00
Kelvin Lee c4edd36767 MSVC: xzdec: Use _fileno and _setmode. 2023-09-22 20:00:37 +03:00
Kelvin Lee cfd1054b9b MSVC: Don't #include <unistd.h>. 2023-09-22 20:00:37 +03:00
Lasse Collin adef92f235 Update THANKS. 2023-09-19 14:03:45 +03:00
Jia Tan 953e775941 CI: Enable CLMUL in address sanitization test.
The crc64_clmul() function should be ignored by the address sanitizer
now so these builds should still pass.
2023-09-14 21:43:34 +08:00
Lasse Collin f167e79bc9 Update THANKS. 2023-09-14 16:35:46 +03:00
Lasse Collin 4f44ef8675 liblzma: Mark crc64_clmul() with __attribute__((__no_sanitize_address__)).
Thanks to Agostino Sarubbo.
Fixes: https://github.com/tukaani-project/xz/issues/62
2023-09-14 16:34:07 +03:00
Jia Tan 7379bb3eed CMake: Fix time.h checks not running on second CMake run.
If CMake was configured more than once, HAVE_CLOCK_GETTIME and
HAVE_CLOCK_MONOTONIC would not be set as compile definitions. The check
for librt being needed to provide HAVE_CLOCK_GETTIME was also
simplified.
2023-09-12 22:49:03 +08:00
Jia Tan 5d691fe582 CMake: Fix unconditionally defining HAVE_CLOCK_MONOTONIC.
If HAVE_CLOCK_GETTIME was defined, then HAVE_CLOCK_MONOTONIC was always
added as a compile definition even if the check for it failed.
2023-09-12 22:34:06 +08:00
Lasse Collin eccf128665 xz: Refactor thousand separator detection and disable it on MSVC.
Now the two variations of the format strings are created with
a macro, and the whole detection code can be easily disabled
on platforms where thousand separator formatting is known to
not work (MSVC has no support, and on DJGPP 2.05 it can have
problems in some cases).
2023-08-31 19:50:05 +03:00
Lasse Collin f7093cd9d1 xz: Fix a too relaxed assertion and remove uses of SSIZE_MAX.
SSIZE_MAX isn't readily available on MSVC. Removing it means
that there is one thing less to worry when porting to MSVC.
2023-08-31 18:14:43 +03:00
Jia Tan 74c3449d8b Tests: Improve invalid unpadded size check in test_lzma_index_append().
This check was extended to test the code added to fix a failing assert
in ae5c07b22a.
2023-08-28 23:14:45 +08:00
Jia Tan 2544274a8b Tests: Improve comments in test_index.c. 2023-08-28 23:05:55 +08:00
Jia Tan 49be29d638 Update THANKS. 2023-08-28 23:05:46 +08:00
Jia Tan 721e3d9f7a liblzma: Update assert in vli_ceil4().
The argument to vli_ceil4() should always guarantee the return value
is also a valid lzma_vli. Thus the highest three valid lzma_vli values
are invalid arguments. All uses of the function ensure this so the
assert is updated to match this.
2023-08-28 23:05:34 +08:00
Jia Tan ae5c07b22a liblzma: Add overflow check for Unpadded size in lzma_index_append().
This was not a security bug since there was no path to overflow
UINT64_MAX in lzma_index_append() or when it calls index_file_size().
The bug was discovered by a failing assert() in vli_ceil4() when called
from index_file_size() when unpadded_sum (the sum of the compressed size
of current Stream and the unpadded_size parameter) exceeds LZMA_VLI_MAX.

Previously, the unpadded_size parameter was checked to be not greater
than UNPADDED_SIZE_MAX, but no check was done once compressed_base was
added.

This could not have caused an integer overflow in index_file_size() when
called by lzma_index_append(). The calculation for file_size breaks down
into the sum of:

- Compressed base from all previous Streams
- 2 * LZMA_STREAM_HEADER_SIZE (size of the current Streams header and
  footer)
- stream_padding (can be set by lzma_index_stream_padding())
- Compressed base from the current Stream
- Unpadded size (parameter to lzma_index_append())

The sum of everything except for Unpadded size must be less than
LZMA_VLI_MAX. This is guarenteed by overflow checks in the functions
that can set these values including lzma_index_stream_padding(),
lzma_index_append(), and lzma_index_cat(). The maximum value for
Unpadded size is enforced by lzma_index_append() to be less than or
equal UNPADDED_SIZE_MAX. Thus, the sum cannot exceed UINT64_MAX since
LZMA_VLI_MAX is half of UINT64_MAX.

Thanks to Joona Kannisto for reporting this.
2023-08-28 23:04:56 +08:00
Jia Tan 1057765aaa Translations: Update the Esperanto translation. 2023-08-28 22:18:29 +08:00
Jia Tan f2e94d064f Translations: Update the Esperanto translation. 2023-08-26 20:10:23 +08:00
Jia Tan 2b871f4dbf Docs: Update INSTALL for --enable-threads method win95.
The Autotools build allows win95 threads and --enable-small together now
if the compiler supports __attribute__((__constructor__)).
2023-08-14 20:39:22 +08:00
Jia Tan 356ad5b26b CMake: Conditionally allow win95 threads and --enable-small. 2023-08-14 20:39:18 +08:00
Jia Tan de574404c4 Build: Conditionally allow win95 threads and --enable-small.
When the compiler supports __attribute__((__constructor__))
mythread_once() is never used, even with --enable-small. A configuration
with win95 threads and --enable-small will compile and be thread safe so
it can be allowed.

This isn't a very common configuration since MSVC does not support
__attribute__((__constructor__)), but MINGW32 and CLANG32 environments
for MSYS2 can use win95 threads and have
__attribute__((__constructor__)) support.
2023-08-09 20:35:16 +08:00
Jamaika1 6bf33b704c
mythread.h: Fix typo error in Vista threads mythread_once().
The "once_" variable was accidentally referred to as just "once". This
prevented building with Vista threads when
HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR was not defined.
2023-08-08 20:07:59 +08:00
Jia Tan 80cb961e53 codespell: Add .codespellrc to set default options.
The .codespellrc allows setting default options to avoid false positive
matches, set additional dictionaries, etc. For now, codespell can be
used locally before committing doc and comment changes.

It should help prevent silly errors and fix up commits in the future.
2023-08-04 22:17:11 +08:00
Jia Tan cd678a6077 Tests: Style fixes to test_lzip_decoder.c. 2023-08-03 20:10:21 +08:00
Jia Tan 1cac5ed4fa Translations: Update the Chinese (simplified) translation. 2023-08-03 15:56:20 +08:00
Lasse Collin 16068f6c30 xz: Omit an empty paragraph on the man page. 2023-08-02 17:15:12 +03:00
Jia Tan 9ae4371b51 Add NEWS for 5.4.4. 2023-08-02 20:30:07 +08:00
Lasse Collin e8c2203b2c build-aux/manconv.sh: Fix US-ASCII and UTF-8 output.
groff defaults to SGR escapes. Using -P-c passes -c to grotty
which restores the old behavior. Perhaps there is a better way to
get pure plain text output but this works for now.
2023-08-02 15:19:43 +03:00
Lasse Collin 9a706167b0 Update THANKS. 2023-08-01 19:10:43 +03:00
Lasse Collin 33e25a0f56 Update THANKS. 2023-08-01 18:22:24 +03:00
ChanTsune 81db3b8898 mythread.h: Disable signal functions in builds targeting Wasm + WASI.
signal.h in WASI SDK doesn't currently provide sigprocmask()
or sigset_t. liblzma doesn't need them so this change makes
liblzma and xzdec build against WASI SDK. xz doesn't build yet
and the tests don't either as tuktest needs setjmp() which
isn't (yet?) implemented in WASI SDK.

Closes: https://github.com/tukaani-project/xz/pull/57
See also: https://github.com/tukaani-project/xz/pull/56

(The original commit was edited a little by Lasse Collin.)
2023-08-01 18:18:05 +03:00
Jia Tan 71c638c611 Add newline to end of .gitignore.
Newline was accidentally removed in commit
01cbb7f023.
2023-08-01 21:58:51 +08:00
Dimitri Papadopoulos Orfanos 42df7c7aa1
Docs: Fix typos found by codespell 2023-07-31 20:02:21 +08:00
Jia Tan 01cbb7f023 Update .gitignore. 2023-07-28 22:17:50 +08:00
Jia Tan f97a1afd56 CMake: Conditionally allow the creation of broken symlinks.
The CMake build will try to create broken symlinks on Unix and Unix-like
platforms. Cygwin and MSYS2 are Unix-like, but may not be able to create
broken symlinks. The value of the CYGWIN or MSYS environment variables
determine if broken symlinks are valid.

The default for MSYS2 does not allow for broken symlinks, so the CMake
build has been broken for MSYS2 since commit
80a1a8bb83.
2023-07-28 22:03:08 +08:00
Jia Tan 7190f4cc7c CI: Fix windows-ci dependency installation.
All of the MSYS2 environments need make, and it does not come with the
toolchain package. The toolchain package will install the needed
compiler toolchains since without this package CMake cannot properly
generate the Makefiles.
2023-07-28 21:56:48 +08:00
Jia Tan a048f472cd CI: Update ci_build.sh CMake to always make Unix Makefiles.
The default for many of the MSYS2 environments is for CMake to create
Ninja build files. This would complicate the build script since we would
need a different command to run the tests. Its simpler to always use
Unix Makefiles so that "make test" is always a usable target for
testing.
2023-07-28 21:54:22 +08:00
Jia Tan 7870396a0c CI: Test CMake builds and test framework with MSYS2. 2023-07-25 20:17:23 +08:00
Jia Tan 6497d1f887 CI: Windows CI rename system matrix variable -> msys2_env.
Calling the MSYS2 environment "system" was a bit vague and should be
more specific.
2023-07-25 20:14:53 +08:00
Jia Tan 785e4121d9 CI: Add Clang64 MSYS2 environment to Windows CI. 2023-07-24 23:11:45 +08:00
Jia Tan d9166b52cf liblzma: Prevent an empty translation unit in Windows builds.
To workaround Automake lacking Windows resource compiler support, an
empty source file is compiled to overwrite the resource files for static
library builds. Translation units without an external declaration are
not allowed by the C standard and result in a warning when used with
-Wempty-translation-unit (Clang) or -pedantic (GCC).
2023-07-24 23:11:13 +08:00
Jia Tan db5019d691 Translations: Update the Vietnamese translation. 2023-07-22 18:37:56 +08:00
Jia Tan f3a055f762 CI: Add Windows runner for Autotools builds with MSYS2.
Only a subset of the tests run by the Linux and MacOS Autotools builds
are run. The most interesting tests are the ones that disable threads,
encoders, and decoders.

The Windows runner will only be run manually since these tests will
likely take much longer than the Linux and MacOS runners. This runner
should be used before merging any large features and before releases.

Currently the clang64 environment fails to due to a warning and
-Werror is enabled for the CI tests. This is still an early version
since the CMake build can be done for MSVC and optionally each of the
MSYS2 environments. GitHub does not allow manually running the CI tests
unless the workflow is checked on the default branch so checking in a
minimum version is a good idea.

Thanks to Arthur S for the original proposing the original patch.

Closes: https://github.com/tukaani-project/xz/pull/34
2023-07-22 18:31:02 +08:00
Jia Tan 556536a352 CI: Add argument to ci_build.sh to pass flags to autogen.sh. 2023-07-22 18:13:43 +08:00
Jia Tan 39a32d36fc Tests: Skip .lz files in test_files.sh if not configured.
Previously if the lzip decoder was not configured then test_files.sh
would pass the lzip tests instead of skipping them.
2023-07-21 18:05:44 +08:00
Jia Tan 194d12724b Tests: Add ARM64 filter test to test_compress.sh. 2023-07-20 22:11:13 +08:00
Jia Tan d850365c44 Translations: Update the Croatian translation. 2023-07-20 20:30:05 +08:00
Jia Tan 24049eb7ac Translations: Update the Korean man page translations. 2023-07-20 20:28:32 +08:00
Jia Tan 4d4a4fa07d Translations: Update the Korean translation. 2023-07-20 20:25:24 +08:00
Jia Tan 237f06d9c5 Translations: Update the Polish translation. 2023-07-20 20:24:05 +08:00
Jia Tan 80c2c83213 Translations: Update the German man page translations. 2023-07-20 20:22:23 +08:00
Jia Tan fdbde14503 Translations: Update the German translation. 2023-07-20 20:18:44 +08:00
Jia Tan 9f3bf5ff5b Translations: Update the Chinese (simplified) translation. 2023-07-20 20:17:10 +08:00
Jia Tan 376938c588 Translations: Update the Swedish translation. 2023-07-20 20:15:47 +08:00
Jia Tan 26b0bc6eb8 Translations: Update the Ukrainian man page translations. 2023-07-20 20:14:00 +08:00
Jia Tan 2d02c8b764 Translations: Update the Ukrainian translation. 2023-07-20 20:09:15 +08:00
Jia Tan f881018b50 Translations: Update the Spanish translation. 2023-07-20 20:06:57 +08:00
Jia Tan 791fe6d3ff Translations: Update the Romanian translation. 2023-07-20 20:05:19 +08:00
Jia Tan 8827e90704 Translations: Update the Romanian man page translations. 2023-07-20 20:02:56 +08:00
Jia Tan 0184d344fa liblzma: Suppress -Wunused-function warning.
Clang 16.0.0 and earlier have a bug that the ifunc resolver function
triggers the -Wunused-function warning. The resolver function is static
and only "used" by the __attribute__((__ifunc()__)).

At this time, the bug is still unresolved, but has been reported:
https://github.com/llvm/llvm-project/issues/63957

This is not a problem in GCC.
2023-07-19 23:36:00 +08:00
Jia Tan 43845fa70f liblzma: Reword lzma_str_list_filters() documentation.
This further improves the documentation from commit
f36ca7982f. The previous wording of
"supported options" was slightly misleading since the options that are
printed are the ones that are relevant for encoding/decoding. It is not
about which options can or must be specified.
2023-07-18 22:57:58 +08:00
Jia Tan 818701ba1c liblzma: Improve comment in string_conversion.c.
The comment used "flag" when referring to decoder options. Just
referring to them as options is more clear and consistent.
2023-07-18 22:56:47 +08:00
Lasse Collin b6b7d06585 xz: Translate the second "%s: " in message.c since French needs "%s : ".
This string is used to print a filename when using "xz -v" and
stderr isn't a terminal.
2023-07-18 17:37:33 +03:00
Lasse Collin be644042c3 xz: Make "%s: %s" translatable because French needs "%s : %s". 2023-07-18 17:34:18 +03:00
Lasse Collin 97fd5cb669 liblzma: Tweak #if condition in memcmplen.h.
Maybe ICC always #defines _MSC_VER on Windows but now
it's very clear which code will get used.
2023-07-18 13:57:54 +03:00
Lasse Collin 40392c19f7 liblzma: Omit unnecessary parenthesis in a preprocessor directive. 2023-07-18 13:49:43 +03:00
Jia Tan abc1d5601b xz: Update Authors list in a few files. 2023-07-18 00:51:48 +08:00
Jia Tan 289034a168 Docs: Add a new section to INSTALL for Tests.
The new Tests section describes basic information about the tests, how
to run them, and important details when cross compiling. We have had a
few questions about how to compile the tests without running them, so
hopefully this information will help others with the same question in the
future.

Fixes: https://github.com/tukaani-project/xz/issues/54
2023-07-18 00:45:31 +08:00
Jia Tan 1119e5f5a5 Docs: Update README.
This adds an entry to "Other implementations of the .xz format" for
XZ for Java.
2023-07-17 23:53:30 +08:00
Jia Tan f99e2e4e53 xz: Fix typo in man page.
The Memory limit information section described three output
columns when it actually has six. This was reworded to
"multiple" to make it more future proof.
2023-07-17 23:34:55 +08:00
Jia Tan f907705eb1 xz: Minor clean up for coder.c
* Moved max_block_list_size from a global to local variable.
* Reworded error message in validate_block_list_filter().
* Removed helper function filter_chain_error().
* Changed 1 << X to 1U << X in many places
2023-07-17 23:34:55 +08:00
Jia Tan 9adc9e5615 xz: Update man page Authors and date. 2023-07-17 23:34:55 +08:00
Jia Tan c12e429f26 xz: Add a section to man page for robot mode --filters-help. 2023-07-17 23:34:55 +08:00
Jia Tan e10f2db5d1 xz: Slight reword in xz man page for consistency.
Changed will print => prints in xz --robot --version description to
match --robot --info-memory description.
2023-07-17 23:34:55 +08:00
Jia Tan f5dc172a40 xz: Reorder robot mode subsections in the man page.
The order is now consistent with the order the command line arguments
are documented earlier in the man page. The new order is:
1. --list
2. --info-memory
3. --version

Instead of the previous order:
1. --version
2. --info-memory
3. --list
2023-07-17 23:34:55 +08:00
Jia Tan 9628be23ae xz: Update man page for new --filters-help option. 2023-07-17 23:34:55 +08:00
Jia Tan a165d7df19 xz: Add a new --filters-help option.
The --filters-help can be used to help create filter chains with the
--filters and --filtersX options. The message in --long-help is too
short to fully explain the syntax to construct complex filter chains.

In --robot mode, xz will only print the output from liblzma function
lzma_str_list_filters.
2023-07-17 23:34:55 +08:00
Jia Tan 95f1a414b1 xz: Update the man page for --block-list and --filtersX
The --block-list option description needed updating since the new
--filtersX option changes how it can be used. The new entry for
--filters1=FILTERS ... --filter9=FILTERS was created right after
the --filters option.
2023-07-17 23:34:55 +08:00
Jia Tan 47a63cad2a xz: Update --long-help for the new --filtersX option. 2023-07-17 23:34:55 +08:00
Jia Tan 8b9913a13d xz: Ignore filter chains that are set but never used in --block-list.
If a filter chain is set but not used in --block-list, it introduced
unexpected behavior such as requiring an unneeded amount of memory to
compress, reducing the number of threads in multi-threaded encoding, and
printing an incorrect amount of memory needed to decompress.

This also renames filters_init_mask => filters_used_mask. A filter is
assumed to be used if it is specified in --filtersX until
coder_set_compression_settings() determines which filters are referenced
in --block-list.
2023-07-17 23:34:55 +08:00
Jia Tan 183819bfd9 xz: Set the Block size for mt encoding correctly.
When opt_block_size is not used, the Block size for mt encoder is
derived from the minimum of the largest Block specified by
--block-list and the recommended Block size on all filter chains
calculated by lzma_mt_block_size(). This avoids using unnecessary
memory and ensures that all Blocks are large enough for the most memory
needy filter chain.
2023-07-17 23:34:55 +08:00
Jia Tan afb2dbec3d xz: Validate --flush-timeout for all specified filter chains. 2023-07-17 23:34:55 +08:00
Jia Tan 5f0c5a0438 xz: Allows --block-list filters to scale down memory usage.
Previously, only the default filter chain could have its memory usage
adjusted. The filter chains specified with --filtersX were not checked
for memory usage. Now, all used filter chains will be adjusted if
necessary.
2023-07-17 23:34:55 +08:00
Jia Tan 479fd58d60 xz: Do not include block splitting if encoders are disabled.
The block splitting logic and split_block() function are not needed if
encoders are disabled. This will help slightly reduce the binary size
when built without encoders and allow split_block() to use functions
that require encoders being enabled.
2023-07-17 23:34:55 +08:00
Jia Tan f86ede2250 xz: Free filters[] in debug mode.
This will only free filter chains created with --filters1-9 since the
default filter chain may be set from a static function variable. The
complexity to free the default filter chain is not worth the burden on
code maintenance.
2023-07-17 23:34:55 +08:00
Jia Tan f281cd0d69 xz: Add a message if --block-list is used outside of xz compresssion.
--block-list is only supported with compression in xz format. This avoids
silently ignoring when --block-list is unused.
2023-07-17 23:34:55 +08:00
Jia Tan d6af7f3470 xz: Create command line options for filters[1-9].
The new command line options are meant to be combined with --block-list.
They work as an optional extension to --block-list to specify a custom
filter chain for each block listed. The new options allow the creation
of up to 9 reusable filter chains. For instance:

xz --block-list=1:10MiB,3:5MiB,,2:5MiB,1:0 --filters1=delta--lzma2 \
--filters2=x86--lzma2 --filters3=arm64--lzma2

Will create the following blocks:
1. A block of size 10 MiB with filter chain delta, lzma2.
2. A block of size 5 MiB with filter chain arm64, lzma2.
3. A block of size 5 MiB with filter chain arm64, lzma2.
4. A block of size 5 MiB with filter chain x86, lzma2.
5. A block containing the rest of the file contents with filter chain
   delta, lzma2.
2023-07-17 23:34:55 +08:00
Jia Tan 072d292501 xz: Use lzma_filters_free() in forget_filter_chain().
This is a little cleaner than the previous implementation of
forget_filter_chain(). It is also more consistent since
lzma_str_to_filters() will always terminate the filter chain so there
is no need to terminate it later in coder_set_compression_settings().
2023-07-17 23:34:55 +08:00
Jia Tan 3d21da5cff xz: Separate string to filter conversion into a helper function.
Converting from string to filter will also need to be done for block
specific filter chains.
2023-07-17 23:34:55 +08:00
Jia Tan a6583726e5 Tests: Use new --filters option in test_compress.sh 2023-07-17 23:34:55 +08:00
Jia Tan 5f3b898d07 xz: Update --long-help and man page for new --filters option. 2023-07-17 23:34:55 +08:00
Jia Tan 9ded880a02 xz: Add --filters option to CLI.
The --filters option uses the new lzma_str_to_filters() function
to convert a string into a full filter chain. Using this option
will reset all previous filters set by --preset, --[filter], or
--filters.
2023-07-17 23:34:55 +08:00
Jia Tan 2c189bb00a Tests: Improve feature testing for skipping.
Fixed a bug where test_compress_* would all fail if arm64 or armthumb
filters were enabled for compression but arm was disabled. Since the
grep tests only checked for "define HAVE_ENCODER_ARM", this would match
on HAVE_ENCODER_ARM64 or HAVE_ENCODER_ARMTHUMB.

Now the config.h feature test requires " 1" at the end to prevent the
prefix problem. have_feature() was also updated for this even though
there were known current bugs affecting it. This is just in case future
features have a similar prefix problem.
2023-07-14 21:30:25 +08:00
Jia Tan 80a6b9bcad Translations: Update the Chinese (traditional) translation. 2023-07-10 20:56:28 +08:00
Jia Tan 17f8844e6f liblzma: Remove non-portable empty initializer.
Commit 78704f36e7 added an empty
initializer {} to prevent a warning. The empty initializer is a GNU
extension and results in a build failure on MSVC. The -wpedantic flag
warns about empty initializers.
2023-07-08 21:24:19 +08:00
Jia Tan 3aca4f629c Translations: Update the Vietnamese translation. 2023-07-08 20:03:59 +08:00
Jia Tan 66bdcfa85f Tests: Fix memory leaks in test_index.
Several tests were missing calls to lzma_index_end() to clean up the
lzma_index structs. The memory leaks were discovered by using
-fsanitize=address with GCC.
2023-06-29 00:07:16 +08:00
Jia Tan fe3bd438fb Tests: Fix memory leaks in test_block_header.
test_block_header was not properly freeing the filter options between
calls to lzma_block_header_decode(). The memory leaks were discovered by
using -fsanitize=address with GCC.
2023-06-29 00:07:05 +08:00
Jia Tan 78704f36e7 liblzma: Prevent uninitialzed warning in mt stream encoder.
This change only impacts the compiler warning since it was impossible
for the wait_abs struct in stream_encode_mt() to be used before it was
initialized since mythread_condtime_set() will always be called before
mythread_cond_timedwait().

Since the mythread.h code is different between the POSIX and
Windows versions, this warning was only present on Windows builds.

Thanks to Arthur S for reporting the warning and providing an initial
patch.
2023-06-29 00:06:16 +08:00
Jia Tan e3356a204c liblzma: Prevent warning for MSYS2 Windows build.
In lzma_memcmplen(), the <intrin.h> header file is only included if
_MSC_VER and _M_X64 are both defined but _BitScanForward64() was
previously used if _M_X64 was defined. GCC for MSYS2 defines _M_X64 but
not _MSC_VER so _BitScanForward64() was used without including
<intrin.h>.

Now, lzma_memcmplen() will use __builtin_ctzll() for MSYS2 GCC builds as
expected.
2023-06-28 23:59:51 +08:00
Jia Tan 45e250a9e9 CI: Add test with -fsanitize=address,undefined.
ci_build.sh was updated to accept disabling of __attribute__ ifunc
and CLMUL. This will allow -fsanitize=address to pass because ifunc
is incompatible with -fsanitize=address. The CLMUL implementation has
optimizations that potentially read past the buffer and mask out the
unwanted bytes.

This test will only run on Autotools Linux.
2023-06-28 23:59:47 +08:00
Jia Tan 596ee722cd CI: Upgrade checkout action from v2 to v3. 2023-06-28 20:48:49 +08:00
Jia Tan 86118ea320 Update THANKS. 2023-06-27 23:56:06 +08:00
Jia Tan 3d1fdddf92 Docs: Document the configure option --disable-ifunc in INSTALL. 2023-06-27 23:56:06 +08:00
Lasse Collin b4cf7a2822 Minor tweaks to style and comments. 2023-06-27 23:56:06 +08:00
Lasse Collin 23fb9e3a32 CMake: Rename CHECK_ATTR_IFUNC to ALLOW_ATTR_IFUNC.
It's so that there's a clear difference in wording compared
to liblzma's integrity check types.
2023-06-27 23:56:06 +08:00
Lasse Collin ee44863ae8 liblzma: Add ifunc implementation to crc64_fast.c.
The ifunc method avoids indirection via the function pointer
crc64_func. This works on GNU/Linux and probably on FreeBSD too.
The previous __attribute((__constructor__)) method is kept for
compatibility with ELF platforms which do support ifunc.

The ifunc method has some limitations, for example, building
liblzma with -fsanitize=address will result in segfaults.
The configure option --disable-ifunc must be used for such builds.

Thanks to Hans Jansen for the original patch.
Closes: https://github.com/tukaani-project/xz/pull/53
2023-06-27 23:55:59 +08:00
Hans Jansen b72d212024 Add ifunc check to CMakeLists.txt
CMake build system will now verify if __attribute__((__ifunc__())) can be
used in the build system. If so, HAVE_FUNC_ATTRIBUTE_IFUNC will be
defined to 1.
2023-06-27 15:33:15 +03:00
Hans Jansen 23b5c36fb7 Add ifunc check to configure.ac
configure.ac will now verify if __attribute__((__ifunc__())) can be used in
the build system. If so, HAVE_FUNC_ATTRIBUTE_IFUNC will be defined to 1.
2023-06-27 15:33:15 +03:00
Jia Tan dbb3a536ed CI: Add apt update command before installing dependencies.
Without the extra command, all of the CI tests were automatically
failing because the Ubuntu servers could not be reached properly.
2023-06-07 00:23:02 +08:00
Jia Tan 6bcd516812 Update THANKS. 2023-06-07 00:10:38 +08:00
Benjamin Buch 0d94ba6922
CMake: Protects against double find_package
Boost iostream uses `find_package` in quiet mode and then again uses
`find_package` with required. This second call triggers a 
`add_library cannot create imported target "ZLIB::ZLIB" because another
target with the same name already exists.`

This can simply be fixed by skipping the alias part on secondary
`find_package` runs.
2023-06-06 21:32:45 +08:00
Jia Tan 045d7aae28 Translations: Update the Esperanto translation. 2023-05-31 23:57:57 +08:00
Jia Tan b0cc7c2dce Translations: Update the Croatian translation. 2023-05-31 23:57:57 +08:00
Jia Tan af045ef6f8 Translations: Update the Chinese (simplified) translation. 2023-05-31 20:15:53 +08:00
Jia Tan e6b92d5817 Translations: Update German translation of man pages. 2023-05-17 23:12:13 +08:00
Jia Tan 592961ccdb Translations: Update the German translation. 2023-05-17 23:09:18 +08:00
Jia Tan 13572cb2c3 Translations: Update the Croatian translation. 2023-05-17 20:30:01 +08:00
Jia Tan 4e6e425ea8 Translations: Update Korean translation of man pages. 2023-05-17 20:26:54 +08:00
Jia Tan d5ef1f6faf Translations: Update the Korean translation. 2023-05-17 20:13:01 +08:00
Jia Tan e22d0b0f2e Translations: Update the Spanish translation. 2023-05-16 23:49:09 +08:00
Jia Tan f50da74d52 Translations: Update the Romanian translation. 2023-05-16 23:47:23 +08:00
Jia Tan 4b9ad60a73 Translations: Update Romanian translation of man pages. 2023-05-16 23:45:43 +08:00
Jia Tan cb6fd57f88 Translations: Update Ukrainian translation of man pages. 2023-05-16 23:44:45 +08:00
Jia Tan c3e8fcbc2d Translations: Update the Ukrainian translation. 2023-05-16 23:44:45 +08:00
Jia Tan 27b81b84fc Translations: Update the Polish translation. 2023-05-16 23:44:36 +08:00
Jia Tan 8024ad636a Translations: Update the Swedish translation. 2023-05-16 22:52:14 +08:00
Jia Tan 6699a29673 Translations: Update the Esperanto translation. 2023-05-16 21:21:38 +08:00
Jia Tan f36ca7982f liblzma: Slightly rewords lzma_str_list_filters() documentation.
Reword "options required" to "supported options". The previous may have
suggested that the options listed were all required anytime a filter is
used for encoding or decoding. The reword makes this more clear that
adjusting the options is optional.
2023-05-13 21:21:54 +08:00
Jia Tan 3374a5359e liblzma: Adds lzma_nothrow to MicroLZMA API functions.
None of the liblzma functions may throw an exception, so this
attribute should be applied to all liblzma API functions.
2023-05-12 00:00:47 +08:00
Jia Tan 8f23657498 liblzma: Exports lzma_mt_block_size() as an API function.
The lzma_mt_block_size() was previously just an internal function for
the multithreaded .xz encoder. It is used to provide a recommended Block
size for a given filter chain.

This function is helpful to determine the maximum Block size for the
multithreaded .xz encoder when one wants to change the filters between
blocks. Then, this determined Block size can be provided to
lzma_stream_encoder_mt() in the lzma_mt options parameter when
intializing the coder. This requires one to know all the filter chains
they are using before starting to encode (or at least the filter chain
that will need the largest Block size), but that isn't a bad limitation.
2023-05-11 23:54:44 +08:00
Jia Tan d0f33d672a liblzma: Creates IS_ENC_DICT_SIZE_VALID() macro.
This creates an internal liblzma macro to test if the dictionary size
is valid for encoding.
2023-05-11 22:28:45 +08:00
Jia Tan c247d06e1f Add NEWS for 5.4.3. 2023-05-04 22:15:35 +08:00
Jia Tan 77050b7836 Add NEWS for 5.2.12. 2023-05-04 22:15:30 +08:00
Jia Tan 713e15e43e Translations: Update the Croatian translation. 2023-05-04 20:38:52 +08:00
Jia Tan 9ad64bdf30 tuklib_integer.h: Reverts previous commit.
Previous commit 6be460dde0 would cause an
error if the integer size was 32 bit.
2023-05-04 20:30:25 +08:00
Jia Tan 6be460dde0 tuklib_integer.h: Changes two other UINT_MAX == UINT32_MAX to >=. 2023-05-04 19:25:20 +08:00
Lasse Collin 44c0c5eae9 tuklib_integer.h: Fix a recent copypaste error in Clang detection.
Wrong line was changed in 7062348bf3.
Also, this has >= instead of == since ints larger than 32 bits would
work too even if not relevant in practice.
2023-05-03 22:55:16 +03:00
Jia Tan 2cf5ae5b5b CI: Adds a build and test for small configuration. 2023-04-25 22:14:28 +08:00
Jia Tan 16b81a057a CI: ci_build.sh allows configuring small build. 2023-04-25 20:05:26 +08:00
Jia Tan 78ccd93951 Update THANKS. 2023-04-20 20:15:00 +08:00
Jia Tan f41df2ac2f Windows: Include <intrin.h> when needed.
Legacy Windows did not need to #include <intrin.h> to use the MSVC
intrinsics. Newer versions likely just issue a warning, but the MSVC
documentation says to include the header file for the intrinsics we use.

GCC and Clang can "pretend" to be MSVC on Windows, so extra checks are
needed in tuklib_integer.h to only include <intrin.h> when it will is
actually needed.
2023-04-19 22:22:16 +08:00
Jia Tan 7062348bf3 tuklib_integer: Use __builtin_clz() with Clang.
Clang has support for __builtin_clz(), but previously Clang would
fallback to either the MSVC intrinsic or the regular C code. This was
discovered due to a bug where a new version of Clang required the
<intrin.h> header file in order to use the MSVC intrinsics.

Thanks to Anton Kochkov for notifying us about the bug.
2023-04-19 21:59:03 +08:00
Lasse Collin 3938718ce3 liblzma: Update project maintainers in lzma.h.
AUTHORS was updated earlier, lzma.h was simply forgotten.
2023-04-14 18:42:33 +03:00
Jia Tan 2a89670ab2 liblzma: Cleans up old commented out code. 2023-04-13 20:45:19 +08:00
Jia Tan 0fbb2b87a7 Docs: Add missing word to SECURITY.md. 2023-04-07 20:46:41 +08:00
Jia Tan fb9c50f38a Update THANKS. 2023-04-07 20:43:56 +08:00
Jia Tan 537c6cd8a9 Docs: Minor edits to SECURITY.md. 2023-04-07 20:43:56 +08:00
Gabriela Gutierrez 6549df8dd5 Docs: Create SECURITY.md
Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2023-04-07 20:43:49 +08:00
Jia Tan d0faa85df5 CI: Tests for disabling threading on CMake builds. 2023-03-29 23:46:43 +08:00
Jia Tan 8be5cc3b13 CI: Removes CMakeCache.txt between builds.
If the cache file is not removed, CMake will not reset configurations
back to their default values. In order to make the tests independent, it
is simplest to purge the cache. Unfortunatly, this will slow down the
tests a little and repeat some checks.
2023-03-29 23:46:43 +08:00
Jia Tan 2cb6028fc3 CMake: Update liblzma-config.cmake generation.
Now that the threading is configurable, the liblzma CMake package only
needs the threading library when using POSIX threads.
2023-03-29 23:46:38 +08:00
Jia Tan 4d7fac0b07 CMake: Allows setting thread method.
The thread method is now configurable for the CMake build. It matches
the Autotools build by allowing ON (pick the best threading method),
OFF (no threading), posix, win95, and vista. If both Windows and
posix threading are both available, then ON will choose Windows
threading. Windows threading will also not use:

target_link_libraries(liblzma Threads::Threads)

since on systems like MinGW-w64 it would link the posix threads
without purpose.
2023-03-29 23:24:12 +08:00
Jia Tan 20cd905d89 CI: Runs CMake feature tests.
Now, CMake will run similar feature disable tests that the Autotools
version did before. In order to do this without repeating lines in
ci.yml, it now makes sense to use the GitHub Workflow matrix to create
a loop.
2023-03-24 23:41:06 +08:00
Jia Tan 4fabdb269f CI: ci_build.sh allows CMake features to be configured.
Also included various clean ups for style and helper functions for
repeated work.
2023-03-24 23:08:24 +08:00
Jia Tan cf3d1f130e CI: Change ci_build.sh to use bash instead of sh.
This script is only meant to be run as part of the CI build/test process
on machines that are known to have bash (Ubuntu and MacOS). If this
assumption changes in the future, then the bash specific commands will
need to be replaced with a more portable option. For now, it is
convenient to use bash commands.
2023-03-24 20:06:33 +08:00
Jia Tan ddfe164368 CMake: Only build xzdec if decoders are enabled. 2023-03-24 20:05:59 +08:00
Jia Tan 116e81f002 Build: Removes redundant check for LZMA1 filter support. 2023-03-23 21:48:52 +08:00
Lasse Collin 0ba234f692 CMake: Bump maximum policy version to 3.26.
It adds only one new policy related to FOLDERS which we don't use.
This makes it clear that the code is compatible with the policies
up to 3.26.
2023-03-23 21:48:52 +08:00
Jia Tan b0891684b4 CMake: Conditionally build xz list.* files if decoders are enabled. 2023-03-23 21:48:52 +08:00
Jia Tan 2c1a830efb CMake: Allow configuring features as cache variables.
This allows users to change the features they build either in
CMakeCache.txt or by using a CMake GUI. The sources built for
liblzma are affected by this too, so only the necessary files
will be compiled.
2023-03-23 21:48:52 +08:00
Lasse Collin 8be136f667 Build: Add a comment that AC_PROG_CC_C99 is needed for Autoconf 2.69.
It's obsolete in Autoconf >= 2.70 and just an alias for AC_PROG_CC
but Autoconf 2.69 requires AC_PROG_CC_C99 to get a C99 compiler.
2023-03-21 14:07:51 +02:00
Lasse Collin 53cc475f26 Build: configure.ac: Use AS_IF and AS_CASE where required.
This makes no functional difference in the generated configure
(at least with the Autotools versions I have installed) but this
change might prevent future bugs like the one that was just
fixed in the commit 5a5bd7f871.
2023-03-21 14:04:37 +02:00
Lasse Collin 3b8890a402 Update THANKS. 2023-03-21 13:12:03 +02:00
Lasse Collin 5a5bd7f871 Build: Fix --disable-threads breaking the building of shared libs.
This is broken in the releases 5.2.6 to 5.4.2. A workaround
for these releases is to pass EGREP='grep -E' as an argument
to configure in addition to --disable-threads.

The problem appeared when m4/ax_pthread.m4 was updated in
the commit 6629ed929c which
introduced the use of AC_EGREP_CPP. AC_EGREP_CPP calls
AC_REQUIRE([AC_PROG_EGREP]) to set the shell variable EGREP
but this was only executed if POSIX threads were enabled.
Libtool code also has AC_REQUIRE([AC_PROG_EGREP]) but Autoconf
omits it as AC_PROG_EGREP has already been required earlier.
Thus, if not using POSIX threads, the shell variable EGREP
would be undefined in the Libtool code in configure.

ax_pthread.m4 is fine. The bug was in configure.ac which called
AX_PTHREAD conditionally in an incorrect way. Using AS_CASE
ensures that all AC_REQUIREs get always run.

Thanks to Frank Busse for reporting the bug.
Fixes: https://github.com/tukaani-project/xz/issues/45
2023-03-21 13:11:49 +02:00
Lasse Collin dfe1710784 liblzma: Silence -Wsign-conversion in SSE2 code in memcmplen.h.
Thanks to Christian Hesse for reporting the issue.
Fixes: https://github.com/tukaani-project/xz/issues/44
2023-03-19 22:45:59 +02:00
Jia Tan f0c580c5fc Add NEWS for 5.4.2. 2023-03-18 22:23:39 +08:00
Jia Tan af4925e604 Add NEWS for 5.2.11. 2023-03-18 22:10:12 +08:00
Lasse Collin 5a7b930efa Update the copy of GNU GPLv3 from gnu.org to COPYING.GPLv3. 2023-03-18 16:00:54 +02:00
Lasse Collin b473a92891 Change a few HTTP URLs to HTTPS.
The xz man page timestamp was intentionally left unchanged.
2023-03-18 15:56:07 +02:00
Jia Tan 8b2f6001b4 CMake: Fix typo in a comment. 2023-03-18 00:40:28 +08:00
Lasse Collin 76e2315e14 Windows: build.bash: Copy liblzma API docs to the output package. 2023-03-17 18:36:22 +02:00
Lasse Collin 133cf55edc Windows: Add microlzma_*.c to the VS project files.
These should have been included in 5.3.2alpha already.
2023-03-17 08:53:38 +02:00
Lasse Collin 75c9ca450f CMake: Add microlzma_*.c to the build.
These should have been included in 5.3.2alpha already.
2023-03-17 08:43:51 +02:00
Lasse Collin 0cc3313bd4 Build: Update comments about unaligned access to mention 64-bit. 2023-03-17 08:41:36 +02:00
Lasse Collin 5e57e33013 Tests: Update .gitignore. 2023-03-17 00:02:34 +02:00
Lasse Collin 0007394d54 po4a/update-po: Display the script name consistently in error messages. 2023-03-17 00:02:34 +02:00
Jia Tan 509157c80c Doc: Rename Doxygen HTML doc directory name liblzma => api.
When the docs are installed, calling the directory "liblzma" is
confusing since multiple other files in the doc directory are for
liblzma. This should also make it more natural for distros when they
package the documentation.
2023-03-17 01:42:28 +08:00
Jia Tan fd90e2f4c2 liblzma: Remove note from lzma_options_bcj about the ARM64 exception.
This was left in by mistake since an early version of the ARM64 filter
used a different struct for its options.
2023-03-17 01:42:28 +08:00
Jia Tan 4f50763b98 CI: Add doxygen as a dependency.
Autogen now requires --no-doxygen or having doxygen installed to run
without errors.
2023-03-17 01:42:28 +08:00
Lasse Collin f68f4b27f6 COPYING: Add a note about the included Doxygen-generated HTML. 2023-03-17 01:42:28 +08:00
Jia Tan 8979308528 Doc: Update PACKAGERS with details about liblzma API docs install. 2023-03-17 01:42:28 +08:00
Jia Tan 55ba6e9300 liblzma: Add set lzma.h as the main page for Doxygen documentation.
The \mainpage command is used in the first block of comments in lzma.h.
This changes the previously nearly empty index.html to use the first
comment block in lzma.h for its contents.

lzma.h is no longer documented separately, but this is for the better
since lzma.h only defined a few macros that users do not need to use.
The individual API header files all have a disclaimer that they should
not be #included directly, so there should be no confusion on the fact
that lzma.h should be the only header used by applications.

Additionally, the note "See ../lzma.h for information about liblzma as
a whole." was removed since lzma.h is now the main page of the
generated HTML and does not have its own page anymore. So it would be
confusing in the HTML version and was only a "nice to have" when
browsing the source files.
2023-03-17 01:42:28 +08:00
Jia Tan 16f2125559 Build: Generate doxygen documentation in autogen.sh.
Another command line option (--no-doxygen) was added to disable
creating the doxygen documenation in cases where it not wanted or
if the doxygen tool is not installed.
2023-03-17 01:42:28 +08:00
Jia Tan 1321852a3b Build: Create doxygen/update-doxygen script.
This is a helper script to generate the Doxygen documentation. It can be
run in 'liblzma' or 'internal' mode by setting the first argument. It
will default to 'liblzma' mode and only generate documentation for the
liblzma API header files.

The helper script will be run during the custom mydist hook when we
create releases. This hook already alters the source directory, so its
fine to do it here too. This way, we can include the Doxygen generated
files in the distrubtion and when installing.

In 'liblzma' mode, the JavaScript is stripped from the .html files and
the .js files are removed. This avoids license hassle from jQuery and
other libraries that Doxygen 1.9.6 puts into jquery.js in minified form.
2023-03-17 01:42:28 +08:00
Jia Tan b1216a7772 Build: Install Doxygen docs and include in distribution if generated.
Added a install-data-local target to install the Doxygen documentation
only when it has been generated. In order to correctly remove the docs,
a corresponding uninstall-local target was added.

If the doxygen docs exist in the source tree, they will also be included
in the distribution now too.
2023-03-17 01:42:28 +08:00
Lasse Collin c97d12f300 Doxygen: Refactor Doxyfile.in to doxygen/Doxyfile.
Instead of having Doxyfile.in configured by Autoconf, the Doxyfile
can have the tags that need to be configured piped into the doxygen
command through stdin with the overrides after Doxyfile's contents.

Going forward, the documentation should be generated in two different
modes: liblzma or internal.

liblzma is useful for most users. It is the documentation for just
the liblzma API header files. This is the default.

internal is for people who want to understand how xz and liblzma work.
It might be useful for people who want to contribute to the project.
2023-03-17 01:42:28 +08:00
Jia Tan 1b7661faa4 Tests: Remove unused macros and functions. 2023-03-13 20:49:53 +08:00
Jia Tan af55191102 liblzma: Defines masks for return values from lzma_index_checks(). 2023-03-13 20:49:53 +08:00
Jia Tan 8f38cdd9ab Tests: Refactors existing lzma_index tests.
Converts the existing lzma_index tests into tuktests and covers every
API function from index.h except for lzma_file_info_decoder, which can
be tested in the future.
2023-03-13 20:49:53 +08:00
Lasse Collin 717aa3651c xz: Simplify the error-label in Capsicum sandbox code.
Also remove unneeded "sandbox_allowed = false;" as this code
will never be run more than once (making it work with multiple
input files isn't trivial).
2023-03-11 18:46:45 +02:00
Lasse Collin a0eecc235d xz: Make Capsicum sandbox more strict with stdin and stdout. 2023-03-08 23:22:15 +08:00
Jia Tan 916448d624 Revert: "Add warning if Capsicum sandbox system calls are unsupported."
The warning causes the exit status to be 2, so this will cause problems
for many scripted use cases for xz. The sandbox usage is already very
limited already, so silently disabling this allows it to be more usable.
2023-03-08 23:22:11 +08:00
Jia Tan 01587dda2a xz: Fix -Wunused-label in io_sandbox_enter().
Thanks to Xin Li for recommending the fix.
2023-03-07 20:02:22 +08:00
Jia Tan 5fb9367866 xz: Add warning if Capsicum sandbox system calls are unsupported.
The warning is only used when errno == ENOSYS. Otherwise, xz still
issues a fatal error.
2023-03-06 21:37:45 +08:00
Jia Tan 61ee82cb12 xz: Skip Capsicum sandbox system calls when they are unsupported.
If a system has the Capsicum header files but does not actually
implement the system calls, then this would render xz unusable. Instead,
we can check if errno == ENOSYS and not issue a fatal error.
2023-03-06 21:27:53 +08:00
Jia Tan f070722b57 xz: Reorder cap_enter() to beginning of capsicum sandbox code.
cap_enter() puts the process into the sandbox. If later calls to
cap_rights_limit() fail, then the process can still have some extra
protections.
2023-03-06 21:08:26 +08:00
Jia Tan f1ab1f6b33 liblzma: Clarify lzma_lzma_preset() documentation in lzma12.h.
lzma_lzma_preset() does not guarentee that the lzma_options_lzma are
usable in an encoder even if it returns false (success). If liblzma
is built with default configurations, then the options will always be
usable. However if the match finders hc3, hc4, or bt4 are disabled, then
the options may not be usable depending on the preset level requested.

The documentation was updated to reflect this complexity, since this
behavior was unclear before.
2023-03-01 21:42:31 +08:00
Lasse Collin 4b7fb3bf41 CMake: Require that the C compiler supports C99 or a newer standard.
Thanks to autoantwort for reporting the issue and suggesting
a different patch:
https://github.com/tukaani-project/xz/pull/42
2023-02-27 18:38:35 +02:00
Jia Tan 9aa7fdeb04 Tests: Small tweak to test-vli.c.
The static global variables can be disabled if encoders and decoders
are not built. If they are not disabled and -Werror is used, it will
cause an usused warning as an error.
2023-02-24 21:11:18 +08:00
Jia Tan 3cf72c4bcb liblzma: Replace '\n' -> newline in filter.h documentation.
The '\n' renders as a newline when the comments are converted to html
by Doxygen.
2023-02-24 21:09:39 +08:00
Jia Tan 002006be62 liblzma: Shorten return description for two functions in filter.h.
Shorten the description for lzma_raw_encoder_memusage() and
lzma_raw_decoder_memusage().
2023-02-24 21:09:39 +08:00
Jia Tan 463d9359b8 liblzma: Reword a few lines in filter.h 2023-02-24 21:09:39 +08:00
Jia Tan 01441df92c liblzma: Improve documentation in filter.h.
All functions now explicitly specify parameter and return values.
The notes and code annotations were moved before the parameter and
return value descriptions for consistency.

Also, the description above lzma_filter_encoder_is_supported() about
not being able to list available filters was removed since
lzma_str_list_filters() will do this.
2023-02-24 21:09:39 +08:00
Lasse Collin 805b45cd60 Update THANKS. 2023-02-23 20:46:16 +02:00
Lasse Collin 30e95bb44c liblzma: Avoid null pointer + 0 (undefined behavior in C).
In the C99 and C17 standards, section 6.5.6 paragraph 8 means that
adding 0 to a null pointer is undefined behavior. As of writing,
"clang -fsanitize=undefined" (Clang 15) diagnoses this. However,
I'm not aware of any compiler that would take advantage of this
when optimizing (Clang 15 included). It's good to avoid this anyway
since compilers might some day infer that pointer arithmetic implies
that the pointer is not NULL. That is, the following foo() would then
unconditionally return 0, even for foo(NULL, 0):

    void bar(char *a, char *b);

    int foo(char *a, size_t n)
    {
        bar(a, a + n);
        return a == NULL;
    }

In contrast to C, C++ explicitly allows null pointer + 0. So if
the above is compiled as C++ then there is no undefined behavior
in the foo(NULL, 0) call.

To me it seems that changing the C standard would be the sane
thing to do (just add one sentence) as it would ensure that a huge
amount of old code won't break in the future. Based on web searches
it seems that a large number of codebases (where null pointer + 0
occurs) are being fixed instead to be future-proof in case compilers
will some day optimize based on it (like making the above foo(NULL, 0)
return 0) which in the worst case will cause security bugs.

Some projects don't plan to change it. For example, gnulib and thus
many GNU tools currently require that null pointer + 0 is defined:

    https://lists.gnu.org/archive/html/bug-gnulib/2021-11/msg00000.html

    https://www.gnu.org/software/gnulib/manual/html_node/Other-portability-assumptions.html

In XZ Utils null pointer + 0 issue should be fixed after this
commit. This adds a few if-statements and thus branches to avoid
null pointer + 0. These check for size > 0 instead of ptr != NULL
because this way bugs where size > 0 && ptr == NULL will likely
get caught quickly. None of them are in hot spots so it shouldn't
matter for performance.

A little less readable version would be replacing

    ptr + offset

with

    offset != 0 ? ptr + offset : ptr

or creating a macro for it:

    #define my_ptr_add(ptr, offset) \
            ((offset) != 0 ? ((ptr) + (offset)) : (ptr))

Checking for offset != 0 instead of ptr != NULL allows GCC >= 8.1,
Clang >= 7, and Clang-based ICX to optimize it to the very same code
as ptr + offset. That is, it won't create a branch. So for hot code
this could be a good solution to avoid null pointer + 0. Unfortunately
other compilers like ICC 2021 or MSVC 19.33 (VS2022) will create a
branch from my_ptr_add().

Thanks to Marcin Kowalczyk for reporting the problem:
https://github.com/tukaani-project/xz/issues/36
2023-02-23 20:41:22 +02:00
Jia Tan fa9065fac5 liblzma: Adjust container.h for consistency with filter.h. 2023-02-23 20:27:59 +08:00
Jia Tan 00a721b63d liblzma: Fix small typos and reword a few things in filter.h. 2023-02-23 20:27:59 +08:00
Jia Tan 5b1c171d4f liblzma: Convert list of flags in lzma_mt to bulleted list. 2023-02-23 20:27:59 +08:00
Jia Tan dbd47622eb liblzma: Fix typo in documentation in container.h
lzma_microlzma_decoder -> lzma_microlzma_encoder
2023-02-23 20:27:59 +08:00
Jia Tan 14cd30806d liblzma: Improve documentation for container.h
Standardizing each function to always specify parameters and return
values. Also moved the parameters and return values to the end of each
function description.
2023-02-23 20:27:59 +08:00
Jia Tan c9c8bfae35 CMake: Add LZIP decoder test to list of tests. 2023-02-22 21:10:28 +08:00
Lasse Collin b9f171dd00 Update THANKS. 2023-02-17 20:56:49 +02:00
Lasse Collin 2ee86d20e4 Build: Use only the generic symbol versioning on MicroBlaze.
On MicroBlaze, GCC 12 is broken in sense that
__has_attribute(__symver__) returns true but it still doesn't
support the __symver__ attribute even though the platform is ELF
and symbol versioning is supported if using the traditional
__asm__(".symver ...") method. Avoiding the traditional method is
good because it breaks LTO (-flto) builds with GCC.

See also: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101766

For now the only extra symbols in liblzma_linux.map are the
compatibility symbols with the patch that spread from RHEL/CentOS 7.
These require the use of __symver__ attribute or __asm__(".symver ...")
in the C code. Compatibility with the patch from CentOS 7 doesn't
seem valuable on MicroBlaze so use liblzma_generic.map on MicroBlaze
instead. It doesn't require anything special in the C code and thus
no LTO issues either.

An alternative would be to detect support for __symver__
attribute in configure.ac and CMakeLists.txt and fall back
to __asm__(".symver ...") but then LTO would be silently broken
on MicroBlaze. It sounds likely that MicroBlaze is a special
case so let's treat it as a such because that is simpler. If
a similar issue exists on some other platform too then hopefully
someone will report it and this can be reconsidered.

(This doesn't do the same fix in CMakeLists.txt. Perhaps it should
but perhaps CMake build of liblzma doesn't matter much on MicroBlaze.
The problem breaks the build so it's easy to notice and can be fixed
later.)

Thanks to Vincent Fazio for reporting the problem and proposing
a patch (in the end that solution wasn't used):
https://github.com/tukaani-project/xz/pull/32
2023-02-17 20:48:28 +02:00
Lasse Collin d831072cce liblzma: Very minor API doc tweaks.
Use "member" to refer to struct members as that's the term used
by the C standard.

Use lzma_options_delta.dist and such in docs so that in Doxygen's
HTML output they will link to the doc of the struct member.

Clean up a few trailing white spaces too.
2023-02-16 21:09:00 +02:00
Jia Tan f029daea39 liblzma: Adjust spacing in doc headers in bcj.h. 2023-02-17 00:54:33 +08:00
Jia Tan a5de68bac2 liblzma: Adjust documentation in bcj.h for consistent style. 2023-02-17 00:49:47 +08:00
Jia Tan efa498c13b liblzma: Rename field => member in documentation.
Also adjusted preset value => preset level.
2023-02-17 00:49:47 +08:00
Lasse Collin 718b22a6c5 liblzma: Silence a warning from MSVC.
It gives C4146 here since unary minus with unsigned integer
is still unsigned (which is the intention here). Doing it
with substraction makes it clearer and avoids the warning.

Thanks to Nathan Moinvaziri for reporting this.
2023-02-16 17:59:50 +02:00
Jia Tan 87c53553fa liblzma: Improve documentation for stream_flags.h
Standardizing each function to always specify parameters and return
values. Also moved the parameters and return values to the end of each
function description.

A few small things were reworded and long sentences broken up.
2023-02-16 21:04:54 +08:00
Jia Tan 13d99e75a5 liblzma: Improve documentation in lzma12.h.
All functions now explicitly specify parameter and return values.
2023-02-15 22:21:44 +08:00
Jia Tan 43ec344c86 liblzma: Improve documentation in check.h.
All functions now explicitly specify parameter and return values.
Also moved the note about SHA-256 functions not being exported to the
top of the file.
2023-02-15 00:59:16 +08:00
Jia Tan 9c71db4e88 liblzma: Improve documentation in index.h
All functions now explicitly specify parameter and return values.
2023-02-15 00:20:44 +08:00
Jia Tan 421f2f2e16 liblzma: Reword a comment in index.h. 2023-02-15 00:20:44 +08:00
Jia Tan b675394849 liblzma: Omit lzma_index_iter's internal field from Doxygen docs.
Add \private above this field and its sub-fields since it is not meant
to be modified by users.
2023-02-15 00:20:44 +08:00
Jia Tan 0c9e4fc2ad liblzma: Fix documentation for LZMA_MEMLIMIT_ERROR.
LZMA_MEMLIMIT_ERROR was missing the "<" character needed to put
documentation after a member.
2023-02-14 20:41:05 +08:00
Jia Tan 816fec125a liblzma: Improve documentation for base.h.
Standardizing each function to always specify params and return values.
Also fixed a small grammar mistake.
2023-02-14 20:41:05 +08:00
Jia Tan 862dacef1a liblzma: Add one more missing [out] annotation in vli.h 2023-02-14 00:12:34 +08:00
Jia Tan 867b08ae42 liblzma: Minor improvements to vli.h.
Added [out] annotations to parameters that are pointers and can have
their value changed. Also added a clarification to lzma_vli_is_valid.
2023-02-14 00:08:33 +08:00
Jia Tan 90d0e628ff liblzma: Add comments for macros in delta.h.
Document LZMA_DELTA_DIST_MIN and LZMA_DELTA_DIST_MAX for completeness
and to avoid Doxygen warnings.
2023-02-10 21:38:25 +08:00
Jia Tan 9255fffdb1 liblzma: Improve documentation in index_hash.h.
All functions now explicitly specify parameter and return values.
Also reworded the description of lzma_index_hash_init() for readability.
2023-02-10 21:35:23 +08:00
Lasse Collin 1dbe12b90c xz: Improve the comment about start_time in mytime.c.
start_time is relative to an arbitary point in time, it's not
time of day, so using it for anything else than time differences
wouldn't make sense.
2023-02-07 19:07:45 +02:00
Jia Tan 7673ef5aa8 Build: Adjust CMake version search regex.
Now, the LZMA_VERSION_MAJOR, LZMA_VERSION_MINOR, and LZMA_VERSION_PATCH
macros do not need to be on consecutive lines in version.h. They can be
separated by more whitespace, comments, or even other content, as long
as they appear in the proper order (major, minor, patch).
2023-02-04 21:06:35 +08:00
Jia Tan b8bce89be7 xz: Add a comment clarifying the use of start_time in mytime.c. 2023-02-04 20:11:51 +08:00
Jia Tan 912af91b10 liblzma: Improve documentation for version.h.
Specified parameter and return values for API functions and documented
a few more of the macros.
2023-02-04 20:11:36 +08:00
Jia Tan 850adec171 Docs: Omit SIGTSTP not handled from TODO. 2023-02-03 22:52:55 +08:00
Jia Tan 2c78a83c6f liblzma: Fix bug in lzma_str_from_filters() not checking filters[] length.
The bug is only a problem in applications that do not properly terminate
the filters[] array with LZMA_VLI_UNKNOWN or have more than
LZMA_FILTERS_MAX filters. This bug does not affect xz.
2023-02-03 00:42:27 +08:00
Jia Tan e01f01b9af Tests: Create test_filter_str.c.
Tests lzma_str_to_filters(), lzma_str_from_filters(), and
lzma_str_list_filters() API functions.
2023-02-03 00:42:27 +08:00
Jia Tan 8dfc029e7a liblzma: Fix typos in comments in string_conversion.c. 2023-02-03 00:42:27 +08:00
Jia Tan 54ad83c1ae liblzma: Clarify block encoder and decoder documentation.
Added a few sentences to the description for lzma_block_encoder() and
lzma_block_decoder() to highlight that the Block Header must be coded
before calling these functions.
2023-02-03 00:22:53 +08:00
Jia Tan f680e771b3 Update lzma_block documentation for lzma_block_uncomp_encode(). 2023-02-03 00:22:53 +08:00
Jia Tan 504cf4af89 liblzma: Minor edits to lzma_block header_size documentation. 2023-02-03 00:22:53 +08:00
Jia Tan 115b720fb5 liblzma: Enumerate functions that read version in lzma_block. 2023-02-03 00:22:53 +08:00
Jia Tan 85ea0979ad liblzma: Clarify comment in block.h. 2023-02-03 00:22:53 +08:00
Jia Tan 1f7ab90d9c liblzma: Improve documentation for block.h.
Standardizing each function to always specify params and return values.
Output pointer parameters are also marked with doxygen style [out] to
make it clear. Any note sections were also moved above the parameter and
return sections for consistency.
2023-02-03 00:22:53 +08:00
Jia Tan c563a4bc55 liblzma: Clarify a comment about LZMA_STR_NO_VALIDATION.
The flag description for LZMA_STR_NO_VALIDATION was previously confusing
about the treatment for filters than cannot be used with .xz format
(lzma1) without using LZMA_STR_ALL_FILTERS. Now, it is clear that
LZMA_STR_NO_VALIDATION is not a super set of LZMA_STR_ALL_FILTERS.
2023-02-01 23:39:45 +08:00
Jia Tan 315c64c7e1 CI: Update .gitignore for artifacts directory in build-aux.
The workflow action for our CI pipeline can only reference artifacts in
the source directory, so we should ignore these files if the ci_build.sh
is run locally.
2023-02-01 21:47:35 +08:00
Jia Tan 2c1341f4fa CI: Add quotes around variables in a few places. 2023-02-01 21:47:35 +08:00
Jia Tan 3a401b0e0c CI: Upload test logs as artifacts if a test fails. 2023-02-01 21:47:35 +08:00
Lasse Collin 610dde15a8 xz: Use clock_gettime() even if CLOCK_MONOTONIC isn't available.
mythread.h and thus liblzma already does it.
2023-01-27 20:02:49 +02:00
Lasse Collin 2e02877288 po4a/po4a.conf: Sort the language identifiers in alphabetical order. 2023-01-27 19:41:19 +02:00
Lasse Collin ff592c616e xz: Add SIGTSTP handler for progress indicator time keeping.
This way, if xz is stopped the elapsed time and estimated time
remaining won't get confused by the amount of time spent in
the stopped state.

This raises SIGSTOP. It's not clear to me if this is the correct way.
POSIX and glibc docs say that SIGTSTP shouldn't stop the process if
it is orphaned but this commit doesn't attempt to handle that.

Search for SIGTSTP in section 2.4.3:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html
2023-01-27 19:37:47 +02:00
Jia Tan 3b1c8ac8d1 Translations: Add Brazilian Portuguese translation of man pages.
Thanks to Rafael Fontenelle.
2023-01-27 20:16:54 +08:00
Lasse Collin a15a7552f9 Build: Avoid different quoting style in --enable-doxygen doc. 2023-01-26 17:51:06 +02:00
Lasse Collin af5a4bd5af tuklib_physmem: Check for __has_warning before GCC version.
Clang can be configured to fake a too high GCC version so
this way it's more robust.
2023-01-26 17:39:46 +02:00
Jia Tan f35d98e206 liblzma: Fix documentation in filter.h for lzma_str_to_filters()
The previous documentation for lzma_str_to_filters() was technically
correct, but misleading. lzma_str_to_filters() returns NULL on success,
which is in practice always defined to 0. This is the same value as
LZMA_OK, but lzma_str_to_filters() does not return lzma_ret so we should
be more clear.
2023-01-24 20:48:50 +08:00
Lasse Collin 2f78ecc593 Revert "tuklib_common: Define __has_warning if it is not defined."
This reverts commit 82e3c968bf.

Macros in the reserved namespace (_foo or __foo) shouldn't be #defined
without a very good reason. Here the alternative would have been
to #define tuklib_has_warning(str) to an approriate value.

Also the tuklib_* files should stay namespace clean if possible.
2023-01-24 20:20:51 +08:00
Lasse Collin 8366cf8738 tuklib_physmem: Clean up the way -Wcast-function-type is silenced on Windows.
__has_warning and other __has_foo macros are meant to become
compiler-agnostic so it's not good to check for __clang__ with it.

This also relied on tuklib_common.h for #defining __has_warning
which was confusing as #defining reserved macros is generally
not a good idea.
2023-01-24 20:20:40 +08:00
Lasse Collin 683a3c7e2f xz: Flip the return value of suffix_is_set to match the documentation.
Also edit style to match the existing coding style in the project.
2023-01-24 20:20:04 +08:00
Jia Tan cc5aa9ab13 xz: Refactor duplicated check for custom suffix when using --format=raw 2023-01-21 22:10:51 +08:00
Jia Tan 9663141274 liblzma: Set documentation on all reserved fields to private.
This prevents the reserved fields from being part of the generated
Doxygen documentation.
2023-01-21 21:37:48 +08:00
Jia Tan 6327a045f3 Doxygen: Update Doxyfile.in from 1.4.7 to 1.8.17.
A few Doxygen tags were obsolete from 1.4.7. Version 1.8.17 released
in 2019, so this should be compatible with resonable modern distros.
The purpose of Doxygen these days is for docs on the website, so it
doesn't necessarily have to work for everyone. Just when the maintainers
want to update the docs.
2023-01-20 00:51:12 +08:00
Jia Tan bbf71b69eb Doxygen: Make Doxygen only produce liblzma API documentation by default.
Doxygen is now configurable in autotools only with
--enable-doxygen=[api|all]. The default is "api", which will only
generate HTML output for liblzma API functions. The LaTex documentation
output was also disabled.
2023-01-20 00:51:12 +08:00
Jia Tan 6fcf4671b6 liblzma: Highlight liblzma API headers should not be included directly.
This improves the generated Doxygen HTML files to better highlight
how to properly use the liblzma API header files.
2023-01-20 00:51:12 +08:00
Jia Tan b43ff180fb tuklib_physmem: Silence warning from -Wcast-function-type on MinGW-w64.
tuklib_physmem depends on GetProcAddress() for both MSVC and MinGW-w64
to retrieve a function address. The proper way to do this is to cast the
return value to the type of function pointer retrieved. Unfortunately,
this causes a cast-function-type warning, so the best solution is to
simply ignore the warning.
2023-01-19 20:35:09 +08:00
Jia Tan 82e3c968bf tuklib_common: Define __has_warning if it is not defined.
clang supports the __has_warning macro to determine if the version of
clang compiling the code supports a given warning. If we do not define
it for other compilers, it may cause a preprocessor error.
2023-01-19 20:32:40 +08:00
Jia Tan b2ba1a489d CI: Reorder 32-bit build first for Linux autotool builds.
The 32-bit build needs to be first so the configure cache only needs to
be reset one time. The 32-bit build sets the CFLAGS env variable, so any
build using that flag after will fail unless the cache is reset.
2023-01-18 22:11:05 +08:00
Jia Tan dd1c113574 CI: Enable --config-cache in autotool builds.
If CFLAGS are set in a build, the cache must be cleared with
"make distclean", or by deleting the cache file.
2023-01-18 21:51:43 +08:00
Jia Tan d3e1147705 xz: Add missing comment for coder_set_compression_settings() 2023-01-16 21:35:45 +08:00
Jia Tan 123255b6ed xz: Do not set compression settings with raw format in list mode.
Calling coder_set_compression_settings() in list mode with verbose mode
on caused the filter chain and memory requirements to print. This was
unnecessary since the command results in an error and not consistent
with other formats like lzma and alone.
2023-01-16 20:55:10 +08:00
Jia Tan 571919c47b Translations: Update the Brazilian Portuguese translation. 2023-01-13 22:21:49 +08:00
Jia Tan 81cb02e2c2 CI: Disable shared and nls from various jobs in autotool runners.
Disabling shared library generation and linking should help speed up the
runners. The shared library is still being tested in the 32 bit build
and the full feature.

Disabling nls is to check for any unexpected warnings or errors.
2023-01-12 23:43:06 +08:00
Jia Tan 58a052198a CI: Reorder the 32-bit job in the Ubuntu runner.
Run the 32 bit job sooner since this is a more interesting test than
some of the later jobs.
2023-01-12 23:39:19 +08:00
Jia Tan 4110a998b8 CI: Allow disabling Native Language Support. 2023-01-12 23:09:03 +08:00
Jia Tan 0dec634e70 CI: Only run autogen.sh if it has not already run. 2023-01-12 23:02:20 +08:00
Jia Tan 32287dc8de CI: Allow disabling shared library in autotools builds. 2023-01-12 22:58:36 +08:00
Jia Tan 77d1ebcc99 CI: Improve Usage readability and add -h option. 2023-01-12 22:44:18 +08:00
Lasse Collin a8bb8358d1 Build: Omit -Wmissing-noreturn from the default warnings.
It's not that important. It can be annoying in builds that
disable many features since in those cases the tests programs
will correctly trigger this warning with Clang.
2023-01-12 13:04:05 +02:00
Lasse Collin 52dc033d0b xz: Use ssize_t for the to-be-ignored return value from write(fd, ptr, 1).
It makes no difference here as the return value fits into an int
too and it then gets ignored but this looks better.
2023-01-12 06:05:58 +02:00
Lasse Collin b1a6d180a3 xz: Silence warnings from -Wsign-conversion in a 32-bit build. 2023-01-12 06:01:12 +02:00
Lasse Collin 31c21c734b liblzma: Silence another warning from -Wsign-conversion in a 32-bit build.
It doesn't warn on a 64-bit system because truncating
a ptrdiff_t (signed long) to uint32_t is diagnosed under
-Wconversion by GCC and -Wshorten-64-to-32 by Clang.
2023-01-12 05:38:48 +02:00
Lasse Collin 37fbdfb726 liblzma: Silence a warning from -Wsign-conversion in a 32-bit build. 2023-01-12 04:46:45 +02:00
Lasse Collin 5ce6ddc221 Build: Make configure add more warning flags for GCC and Clang.
-Wstrict-aliasing was removed from the list since it is enabled
by -Wall already.

A normal build is clean with these on GNU/Linux x86-64 with
GCC 12.2.0 and Clang 14.0.6.
2023-01-12 04:17:24 +02:00
Lasse Collin bfc3a0a8ac Tests: Fix warnings from clang --Wassign-enum.
Explicitly casting the integer to lzma_check silences the warning.
Since such an invalid value is needed in multiple tests, a constant
INVALID_LZMA_CHECK_ID was added to tests.h.

The use of 0x1000 for lzma_block.check wasn't optimal as if
the underlying type is a char then 0x1000 will be truncated to 0.
However, in these test cases the value is ignored, thus even with
such truncation the test would have passed.
2023-01-12 04:14:18 +02:00
Lasse Collin 49245bb31e Tests: Silence warnings from -Wsign-conversion.
Note that assigning an unsigned int to lzma_check doesn't warn
on GNU/Linux x86-64 since the enum type is unsigned on that
platform. The enum can be signed on some other platform though
so it's best to use enumeration type lzma_check in these situations.
2023-01-12 03:56:24 +02:00
Lasse Collin 3f13bf6b9e liblzma: Silence warnings from clang -Wconditional-uninitialized.
This is similar to 2ce4f36f17.
The actual initialization of the variables is done inside
mythread_sync() macro. Clang doesn't seem to see that
the initialization code inside the macro is always executed.
2023-01-12 03:19:59 +02:00
Lasse Collin 6c886cc5b3 Fix warnings from clang -Wdocumentation. 2023-01-12 03:11:40 +02:00
Lasse Collin a0e7fb1c1e Tests: test_lzip_decoder: Remove trailing white-space. 2023-01-12 03:04:28 +02:00
Lasse Collin c0f8d6782f Tests: test_lzip_decoder: Silence warnings from -Wsign-conversion. 2023-01-12 03:03:55 +02:00
Jia Tan 62efd48a82
Add NEWS for 5.4.1. 2023-01-11 23:58:16 +08:00
Jia Tan d1561c47ec
xz: Fix warning -Wformat-nonliteral on clang in message.c.
clang and gcc differ in how they handle -Wformat-nonliteral. gcc will
allow a non-literal format string as long as the function takes its
format arguments as a va_list.
2023-01-11 22:46:48 +08:00
Jia Tan 8c0f115cc4 Tests: Fix test_filter_flags copy/paste error. 2023-01-11 20:58:31 +08:00
Jia Tan 25035813d1 Tests: Fix type-limits warning in test_filter_flags.
This only occurs in test_filter_flags when the BCJ filters are not
configured and built. In this case, ARRAY_SIZE() returns 0 and causes a
type-limits warning with the loop variable since an unsigned number will
always be >= 0.
2023-01-11 20:42:29 +08:00
Lasse Collin 0b8fa310cf liblzma: CLMUL CRC64: Work around a bug in MSVC, second attempt.
This affects only 32-bit x86 builds. x86-64 is OK as is.

I still cannot easily test this myself. The reporter has tested
this and it passes the tests included in the CMake build and
performance is good: raw CRC64 is 2-3 times faster than the
C version of the slice-by-four method. (Note that liblzma doesn't
include a MSVC-compatible version of the 32-bit x86 assembly code
for the slice-by-four method.)

Thanks to Iouri Kharon for figuring out a fix, testing, and
benchmarking.
2023-01-10 22:15:55 +02:00
Jia Tan 765354b50c Tests: Fix unused function warning in test_block_header.
One of the global arrays of filters was only used in a test that
required both encoders and decoders to be configured in the build.
2023-01-11 01:18:50 +08:00
Jia Tan 7c23c05bef Tests: Fix unused function warning in test_index_hash.
test_index_hash does not use fill_index_hash() unless both encoders
and decoders are configured in the build.
2023-01-11 01:08:03 +08:00
Jia Tan 57464bb4eb CI/CD: Add 32-bit build and test steps to Ubuntu autotools runner.
If all goes well, Mac autotools and Linux and Mac CMake will be added
later for 32-bit builds.
2023-01-11 00:54:45 +08:00
Jia Tan 923eb689a4 CI/CD: Enables warnings as errors in autotool build.
This will help us catch warnings and potential bugs in builds that are
not often tested by us.
2023-01-11 00:51:01 +08:00
Jia Tan feae5528a3 CI/CD: Add -f argument to set CFLAGS in ci_build.sh.
For now, the suggested option is for -m32 only, but this can be updated
later if other flags are deemed useful.
2023-01-11 00:48:35 +08:00
Lasse Collin cfabb62a48 Revert "liblzma: CLMUL CRC64: Workaround a bug in MSVC (VS2015-2022)."
This reverts commit 36edc65ab4.

It was reported that it wasn't a good enough fix and MSVC
still produced (different kind of) bad code when building
for 32-bit x86 if optimizations are enabled.

Thanks to Iouri Kharon.
2023-01-10 12:47:16 +02:00
Lasse Collin 0b64215170 sysdefs.h: Don't include strings.h anymore.
On some platforms src/xz/suffix.c may need <strings.h> for
strcasecmp() but suffix.c includes the header when it needs it.

Unless there is an old system that otherwise supports enough C99
to build XZ Utils but doesn't have C89/C90-compatible <string.h>,
there should be no need to include <strings.h> in sysdefs.h.
2023-01-10 11:56:11 +02:00
Lasse Collin ec2fc39fe4 xz: Include <strings.h> in suffix.c if needed for strcasecmp().
SUSv2 and POSIX.1‐2017 declare only a few functions in <strings.h>.
Of these, strcasecmp() is used on some platforms in suffix.c.
Nothing else in the project needs <strings.h> (at least if
building on a modern system).

sysdefs.h currently includes <strings.h> if HAVE_STRINGS_H is
defined and suffix.c relied on this.

Note that dos/config.h doesn't #define HAVE_STRINGS_H even though
DJGPP does have strings.h. It isn't needed with DJGPP as strcasecmp()
is also in <string.h> in DJGPP.
2023-01-10 11:23:41 +02:00
Lasse Collin 7049c4a76c sysdefs.h: Fix a comment. 2023-01-10 10:05:13 +02:00
Lasse Collin 194a5fab69 sysdefs.h: Don't include memory.h anymore even if it were available.
It quite probably was never needed, that is, any system where memory.h
was required likely couldn't compile XZ Utils for other reasons anyway.

XZ Utils 5.2.6 and later source packages were generated using
Autoconf 2.71 which no longer defines HAVE_MEMORY_H. So the code
being removed is no longer used anyway.
2023-01-10 10:04:06 +02:00
Lasse Collin 5e34774c31 CMake: Fix appending to CMAKE_RC_FLAGS.
It's a string, not a list. It only worked when the variable was empty.

Thanks to Iouri Kharon.
2023-01-10 08:29:32 +02:00
Lasse Collin 6e652ceb18 Windows: Update INSTALL-MSVC.txt to recommend CMake over project files. 2023-01-10 00:35:09 +02:00
Lasse Collin 6b117d3b1f CMake: Fix windres issues again.
At least on some systems, GNU windres needs --use-temp-file
in addition to the \x20 hack to avoid spaces in the command line
argument. Hovever, that \x20 syntax is broken with llvm-windres
version 15.0.0 (results in "XZx20Utils") but luckily it works
with a regular space. Thus it is best to limit the workarounds
to GNU toolchain on Windows.
2023-01-09 23:41:25 +02:00
Lasse Collin 0c210ca7f4 Tests: test_filter_flags: Clean up minor issues.
Here are the list of the most significant issues addressed:
- Avoid using internal common.h header. It's not good to copy the
constants like this but common.h cannot be included for use outside
of liblzma. This is the quickest thing to do that could be fixed later.

- Omit the INIT_FILTER macro. Initialization should be done with just
regular designated initializers.

- Use start_offset = 257 for BCJ tests. It demonstrates that Filter
Flags encoder and decoder don't validate the options thoroughly.
257 is valid only for the x86 filter. This is a bit silly but
not a significant problem in practice because the encoder and
decoder initialization functions will catch bad alignment still.
Perhaps this should be fixed but it's not urgent and doesn't need
to be in 5.4.x.

- Various tweaks to comments such as filter id -> Filter ID
2023-01-09 20:35:20 +08:00
Jia Tan 5c9fdd3bf5 Tests: Refactors existing filter flags tests.
Converts the existing filter flags tests into tuktests.
2023-01-09 20:35:20 +08:00
Lasse Collin 36edc65ab4 liblzma: CLMUL CRC64: Workaround a bug in MSVC (VS2015-2022).
I haven't tested with MSVC myself and there doesn't seem to be
information about the problem online, so I'm relying on the bug report.

Thanks to Iouri Kharon for the bug report and the patch.
2023-01-09 12:22:05 +02:00
Lasse Collin 790a12a95a CMake: Fix a copypaste error in xzdec Windows resource file handling.
It was my mistake. Thanks to Iouri Kharon for the bug report.
2023-01-09 11:27:24 +02:00
Lasse Collin 0e1545fea3 Tests: tuktest.h: Support tuktest_malloc(0).
It's not needed in XZ Utils at least for now. It's good to support
it still because if such use is needed later, it wouldn't be
caught on GNU/Linux since malloc(0) from glibc returns non-NULL.
2023-01-08 00:32:29 +02:00
Lasse Collin 69d5d78c69 Update THANKS. 2023-01-08 00:24:23 +02:00
Lasse Collin dd38655f80 CMake: Update cmake_minimum_required from 3.13...3.16 to 3.13...3.25.
The changes listed on cmake-policies(7) for versions 3.17 to 3.25
shouldn't affect this project.
2023-01-08 00:20:42 +02:00
Lasse Collin a890a637be Update THANKS. 2023-01-08 00:20:42 +02:00
Lasse Collin 6e38e595dd CMake/Windows: Add resource files to xz.exe and xzdec.exe.
The command line tools cannot be built with MSVC for now but
they can be built with MinGW-w64.

Thanks to Iouri Kharon for the bug report and the original patch.
2023-01-08 00:20:42 +02:00
Lasse Collin 443dfebced CMake/Windows: Add a workaround for windres from GNU binutils.
Thanks to Iouri Kharon for the bug report and the original patch.
2023-01-08 00:20:42 +02:00
Lasse Collin ceb8050117 Build: Require that _mm_set_epi64x() is usable to enable CLMUL support.
VS2013 doesn't have _mm_set_epi64x() so this way CLMUL gets
disabled with VS2013.

Thanks to Iouri Kharon for the bug report.
2023-01-08 00:20:42 +02:00
Jia Tan 8d372bd940 CI/CD: Split CMake Linux and MacOS build phase to build and test.
The phase split was only done for Autotools before, so should also
apply to CMake.
2023-01-07 21:15:18 +08:00
Jia Tan 747c7f2b34 CI/CD: Reduce job runners to 4 instead of using matrix strategy.
The old version used too many runners that resulted in unnecessary
dependency downloads. Now, the runners are reused for the different
configurations for each OS and build system.
2023-01-07 21:15:16 +08:00
Jia Tan 4de35fd6b5 CI/CD: Add new -p (PHASE) argument to ci_build.sh
The new PHASE argument can be build, test, or all. all is the default.
This way, the CI/CD script can differentiate between the build and test
phases to make it easier to track down errors when they happen.
2023-01-07 21:15:14 +08:00
Jia Tan 6fd39664de
Merge pull request #7 from tukaani-project/tuktest_index_hash
Tuktest index hash
2023-01-07 00:10:50 +08:00
Lasse Collin fc0c788469 Tests: test_index_hash: Add an assert_uint_eq(). 2023-01-06 17:58:48 +02:00
Lasse Collin d550304f53 Tests: test_index_hash: Fix a memory leak. 2023-01-06 17:55:06 +02:00
Lasse Collin 02608f74ea Tests: test_index_hash: Don't treat pointers as booleans. 2023-01-06 17:53:03 +02:00
Lasse Collin 056766c860 Tests: test_index_hash: Fix a typo in a comment. 2023-01-06 17:51:41 +02:00
Lasse Collin 873e684028 Tests: test_index_hash: Avoid the variable name "index".
It can trigger warnings from -Wshadow on some systems.
2023-01-06 17:44:29 +02:00
Lasse Collin d1f24c3587 Tests: test_index_hash: Use the word "Record" instead of "entry". 2023-01-06 17:35:50 +02:00
Lasse Collin b93f7c5cbb Tests: test_index_hash: Tweak comments and style.
The words defined in the .xz file format specification
begin with capital letter to emphasize that they have
a specific meaning.
2023-01-06 17:35:05 +02:00
Lasse Collin c48b24fc06 Tests: test_index_hash: Use INDEX_INDICATOR constant instead of 0. 2023-01-06 17:17:37 +02:00
Jia Tan 78e0561dfe Style: Change #if !defined() to #ifndef in mythread.h. 2023-01-06 20:43:31 +08:00
Jia Tan e834e1e934 Build: Add missing stream_decoder_mt.c to .vcxproj files.
The line in the .vcxproj files for building with was missing in 5.4.0.
Thank to Hajin Jang for reporting the issue.
2023-01-06 20:42:18 +08:00
Jia Tan 84f9687cba liblzma: Remove common.h include from common/index.h.
common/index.h is needed by liblzma internally and tests. common.h will
include and define many things that are not needed by the tests. Also,
this prevents include order problems because common.h will redefine
LZMA_API resulting in a warning.
2023-01-05 20:57:25 +08:00
Lasse Collin 7657ce1c3c Update THANKS. 2023-01-04 23:16:26 +02:00
Lasse Collin aafd67fba0 Tests: Adjust style in test_compress.sh. 2023-01-04 18:40:28 +02:00
Jia Tan 52380678f4
Tests: Replace non portable shell parameter expansion
The shell parameter expansion using # and ## is not supported in
Solaris 10 Bourne shell (/bin/sh). Even though this is POSIX, it is not fully
portable, so we should avoid it.
2023-01-04 23:58:58 +08:00
Jia Tan d0eb345bb7 Translations: Add Korean translation of man pages.
Thanks to Seong-ho Cho
2023-01-03 21:02:38 +08:00
Jia Tan c4145978d9 Translations: Update the Esperanto translation. 2023-01-03 20:47:27 +08:00
Lasse Collin 4103a2e78a Bump version and soname for 5.5.0alpha.
5.5.0alpha won't be released, it's just to mark that
the branch is not for stable 5.4.x.

Once again there is no API/ABI stability for new features
in devel versions. The major soname won't be bumped even
if API/ABI of new features breaks between devel releases.
2023-01-02 17:20:47 +02:00
Lasse Collin 73c9e6d6b9 Build: Fix config.h comments. 2023-01-02 17:05:07 +02:00
Jia Tan bb740e3b11
Build: Only define HAVE_PROGRAM_INVOCATION_NAME if it is set to 1.
HAVE_DECL_PROGRAM_INVOCATION_NAME is renamed to
HAVE_PROGRAM_INVOCATION_NAME. Previously,
HAVE_DECL_PROGRAM_INVOCATION_NAME was always set when
building with autotools. CMake would only set this when it was 1, and the
dos/config.h did not define it. The new macro definition is consistent
across build systems.
2023-01-02 22:33:48 +08:00
Jia Tan 064cd385a7 Adds test_index_hash to .gitignore. 2023-01-02 22:20:04 +08:00
Jia Tan 3959162bae Tests: Creates test_index_hash.c
Tests all API functions exported from index_hash.h. Does not have a
dedicated test for lzma_index_hash_end.
2023-01-02 22:20:04 +08:00
Jia Tan f16e12d5e7 liblzma: Add NULL check to lzma_index_hash_append.
This is for consistency with lzma_index_append.
2023-01-02 22:20:04 +08:00
Jia Tan 203b008eb2 liblzma: Replaced hardcoded 0x0 index indicator byte with macro 2023-01-02 22:20:04 +08:00
Lasse Collin dfecda8752 Tests: test_check: Test corner cases of CLMUL CRC64. 2022-12-30 20:10:08 +02:00
Lasse Collin ce96bb2043 Tests: Clarify a comment in test_lzip_decoder.c. 2022-12-30 19:36:49 +02:00
Jia Tan 2fcba17fc4 xz: Includes <time.h> and <sys/time.h> conditionally in mytime.c.
Previously, mytime.c depended on mythread.h for <time.h> to be included.
2022-12-30 23:34:31 +08:00
Jia Tan f82294c831 liblzma: Includes sys/time.h conditionally in mythread
Previously, <sys/time.h> was always included, even if mythread only used
clock_gettime. <time.h> is still needed even if clock_gettime is not used
though because struct timespec is needed for mythread_condtime.
2022-12-30 23:34:31 +08:00
Jia Tan 74dae7d300 Build: No longer require HAVE_DECL_CLOCK_MONOTONIC to always be set.
Previously, if threading was enabled HAVE_DECL_CLOCK_MONOTONIC would always
be set to 0 or 1. However, this macro was needed in xz so if xz was not
built with threading and HAVE_DECL_CLOCK_MONOTONIC was not defined but
HAVE_CLOCK_GETTIME was, it caused a warning during build. Now,
HAVE_DECL_CLOCK_MONOTONIC has been renamed to HAVE_CLOCK_MONOTONIC and
will only be set if it is 1.
2022-12-30 23:34:31 +08:00
Jia Tan 7339e39dc0 Translations: Add Ukrainian translations of man pages.
Thanks to Yuri Chornoivan
2022-12-30 23:34:31 +08:00
Jia Tan 9f05c27a58 CI/CD: Create initial version of CI/CD workflow.
The CI/CD workflow will only execute on Ubuntu and MacOS latest version.
The workflow will attempt to build with autotools and CMake and execute
the tests. The workflow will run for all pull requests and pushes done
to the master branch.
2022-12-30 23:34:31 +08:00
Jia Tan 1275ebfba7 liblzma: Update documentation for lzma_filter_encoder. 2022-12-30 23:34:31 +08:00
Jia Tan 7c9ff5f166 Tests: Adds lzip decoder tests 2022-12-30 23:34:31 +08:00
Jia Cheong Tan 799ead162d Doxygen: Update .gitignore for generating docs for in source build.
In source builds are not recommended, but we should still ignore
the generated artifacts.
2022-12-30 23:34:31 +08:00
Jia Tan 5f7ce42a16 liblzma: Fix lzma_microlzma_encoder() return value.
Using return_if_error on lzma_lzma_lclppb_encode was improper because
return_if_error is expecting an lzma_ret value, but
lzma_lzma_lclppb_encode returns a boolean. This could result in
lzma_microlzma_encoder, which would be misleading for applications.
2022-12-30 23:34:31 +08:00
Jia Tan 8ace358d65 CMake: Update .gitignore for CMake artifacts from in source build.
In source builds are not recommended, but we can make it easier
by ignoring the generated artifacts from CMake.
2022-12-30 23:34:31 +08:00
Lasse Collin 8fd225a2c1 liblzma: Update authors list in arm64.c. 2022-12-16 18:30:02 +02:00
414 changed files with 46368 additions and 18400 deletions

24
.codespellrc Normal file
View File

@ -0,0 +1,24 @@
[codespell]
# Skip all translation files and a few other autogenerated files.
# The autotool files should have their typos fixed in the upstream, but
# until then we will blacklist them here.
skip = *.po,*.pot,./po4a/man,./doc/api,./configure,./autom4te.cache,./m4/libtool.m4,./build-aux/depcomp,./build-aux/ltmain.sh,./build-aux/config.guess,./build-aux/config.rpath,./m4/po.m4,./build-aux/config.sub
# Ignore false positive matching words. Ideally codespell would allow
# ignoring words for specific files, but that does not appear to be
# supported. Instead we need to hope we do not make these typos.
# Additionally, the ignored words must be specified lower-case even though
# some of the false positives only occurred upper-case.
# ANS - used as a variable name in xzmore.in.
# bu - groff syntax for creating a bullet list item, used in xz.1.
# te - groff syntax, used in xz.1.
# caf - command line options for tar example, used in xz.1.
ignore-words-list = ans,bu,te,caf
# Add extra dictionaries to help improvement comments, docs, etc.
builtin = clear,rare,informal,usage,names
# Always default to highest interactive level to avoid accidentally
# changing a false positive or picking the wrong replacement.
interactive = 3

23
.github/SECURITY.md vendored Normal file
View File

@ -0,0 +1,23 @@
# Security Policy
## Supported Versions
We provide security updates to the development branch and the stable
branches. Security patches for old releases are available on the
[project website](https://xz.tukaani.org/xz-utils/).
## Reporting a Vulnerability
If you discover a security vulnerability in this project, please
report it privately. **Do not disclose it as a public issue.** This gives
us time to work with you to fix the issue before public exposure, reducing
the chance that the exploit will be used before a patch is released.
You may submit a report by emailing us at
[xz@tukaani.org](mailto:xz@tukaani.org), or through
[Security Advisories](https://github.com/tukaani-project/xz/security/advisories/new).
While both options are available, we prefer email.
This project is maintained by a team of volunteers on a reasonable-effort
basis. As such, please give us 90 days to work on a fix before
public exposure.

160
.github/workflows/ci.yml vendored Normal file
View File

@ -0,0 +1,160 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# Author: Jia Tan
#
#############################################################################
name: CI
on:
# Triggers the workflow on push or pull request events but only for the master branch
push:
branches: [ master ]
pull_request:
branches: [ master ]
# Allows running workflow manually
workflow_dispatch:
jobs:
POSIX:
strategy:
matrix:
os: [ubuntu-latest, macos-latest]
build_system: [autotools, cmake]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 #v4.1.0
########################
# Install Dependencies #
########################
# Install Autotools on Linux
- name: Install Dependencies
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
sudo apt-get update
sudo apt-get install -y autoconf automake build-essential po4a autopoint gcc-multilib doxygen musl-tools
# Install Autotools on Mac
- name: Install Dependencies
if: ${{ matrix.os == 'macos-latest' && matrix.build_system == 'autotools' }}
run: brew install autoconf automake libtool po4a doxygen
# Install CMake on Linux
- name: Install Dependencies
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'cmake' }}
run: |
sudo apt-get update
sudo apt-get install -y build-essential cmake musl-tools
# Install CMake on Mac
- name: Install Dependencies
if: ${{ matrix.os == 'macos-latest' && matrix.build_system == 'cmake' }}
run: brew install cmake
##################
# Build and Test #
##################
# -b specifies the build system to use.
# -p specifies the phase (build or test) to help narrow down an error
# if one occurs.
#
# The first two builds/tests are only run on Autotools Linux and
# affect the CFLAGS. Resetting the CFLAGS requires clearing the
# config cache between runs, so the tests that require CFLAGS are
# done first.
- name: Build 32-bit
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: ./build-aux/ci_build.sh -b autotools -p build -f "-m32"
- name: Test 32-bit
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
./build-aux/ci_build.sh -b autotools -p test -f "-m32" -n 32_bit
cd ../xz_build && make distclean
# ifunc must be disabled for this test because __attribute__ ifunc is
# incompatible with -fsanitize=address.
#
# The sandbox must also be disabled because it will prevent access to
# the /proc/ filesystem on Linux, which is used by the sanitizer's
# instrumentation.
- name: Build with -fsanitize=address,undefined
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: ./build-aux/ci_build.sh -b autotools -p build -f "-fsanitize=address,undefined" -d ifunc,sandbox
- name: Test with -fsanitize=address,undefined
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: |
./build-aux/ci_build.sh -b autotools -p test -f "-fsanitize=address,undefined" -d ifunc,sandbox
cd ../xz_build && make distclean
# musl libc has some slight differences compared to glibc, including
# the lack of ifunc support. This tests if the ifunc detection
# functions properly since musl-gcc can compile with ifunc support,
# but will fail at runtime.
- name: Build with musl libc
if: ${{ matrix.os == 'ubuntu-latest'}}
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p build -m "/usr/bin/musl-gcc"
- name: Test with musl libc
if: ${{ matrix.os == 'ubuntu-latest'}}
run: |
./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p test -m "/usr/bin/musl-gcc"
- name: Clean up musl libc run
if: ${{ matrix.os == 'ubuntu-latest' && matrix.build_system == 'autotools' }}
run: cd ../xz_build && make distclean
- name: Build with full features
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p build
- name: Test with full features
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -p test -n full_features
- name: Build without encoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d encoders,shared -p build
- name: Test without encoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d encoders,shared -p test -n no_encoders
- name: Build without decoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d decoders,shared -p build
- name: Test without decoders
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d decoders,shared -p test -n no_decoders
- name: Build without threads
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d threads,shared -p build
- name: Test without threads
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d threads,shared -p test -n no_threads
- name: Build without BCJ filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d bcj,shared,nls -p build
- name: Test without BCJ filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d bcj,shared,nls -p test -n no_bcj
- name: Build without Delta filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d delta,shared,nls -p build
- name: Test without Delta filters
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d delta,shared,nls -p test -n no_delta
- name: Build without sha256 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,crc64 -d shared,nls -p build
- name: Test without sha256 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,crc64 -d shared,nls -p test -n no_sha256
- name: Build without crc64 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,sha256 -d shared,nls -p build
- name: Test without crc64 check
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -c crc32,sha256 -d shared,nls -p test -n no_crc64
- name: Build small
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d small -p build
- name: Test small
run: ./build-aux/ci_build.sh -b ${{ matrix.build_system }} -d small -p test -n small
# Attempt to upload the test logs as artifacts if any step has failed
- uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 #v4.0.0
if: ${{ failure() }}
with:
name: ${{ matrix.os }} ${{ matrix.build_system }} Test Logs
path: build-aux/artifacts

124
.github/workflows/windows-ci.yml vendored Normal file
View File

@ -0,0 +1,124 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# Author: Jia Tan
#
#############################################################################
name: Windows-CI
# Only run the Windows CI manually since it takes much longer than the others.
on: workflow_dispatch
jobs:
POSIX:
strategy:
matrix:
# Test different environments since the code may change between
# them and we want to ensure that we support all potential users.
# clang64 builds are currently broken when building static libraries
# due to a bug in ldd search path:
# https://github.com/llvm/llvm-project/issues/67779
# TODO - re-enable clang64 when this is resolved.
msys2_env: [mingw64, mingw32, ucrt64, msys]
build_system: [autotools, cmake]
# Set the shell to be msys2 as a default to avoid setting it for
# every individual run command.
defaults:
run:
shell: msys2 {0}
runs-on: windows-latest
steps:
#####################
# Setup Environment #
#####################
# Rely on the msys2 GitHub Action to set up the msys2 environment.
- name: Setup MSYS2
uses: msys2/setup-msys2@27b3aa77f672cb6b3054121cfd80c3d22ceebb1d #v2.20.1
with:
msystem: ${{ matrix.msys2_env }}
update: true
install: pactoys make
- name: Checkout code
# Need to explicitly set the shell here since we set the default
# shell as msys2 earlier. This avoids an extra msys2 dependency on
# git.
shell: powershell
# Avoid Windows line endings. Otherwise test_scripts.sh will fail
# because the expected output is stored in the test framework as a
# text file and will not match the output from xzgrep.
run: git config --global core.autocrlf false
- uses: actions/checkout@8ade135a41bc03ea155e62e844d188df1ea18608 #v4.1.0
########################
# Install Dependencies #
########################
# The pacman repository has a different naming scheme for default
# msys packages than the others. The pacboy tool allows installing
# the packages possible in matrix setup without a burdensome amount
# of ifs.
- name: Install Dependencies
if: ${{ matrix.msys2_env == 'msys' && matrix.build_system == 'autotools' }}
run: pacman --noconfirm -S --needed autotools base-devel doxygen gettext-devel gcc
- name: Install Dependencies
if: ${{ matrix.msys2_env != 'msys' && matrix.build_system == 'autotools' }}
run: pacboy --noconfirm -S --needed autotools:p toolchain:p doxygen:p
- name: Install Dependencies
if: ${{ matrix.msys2_env == 'msys' && matrix.build_system == 'cmake' }}
run: pacman --noconfirm -S --needed cmake base-devel gcc
- name: Install Dependencies
if: ${{ matrix.msys2_env != 'msys' && matrix.build_system == 'cmake' }}
run: pacboy --noconfirm -S --needed cmake:p toolchain:p
##################
# Build and Test #
##################
- name: Build with full features
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -p build
- name: Test with full features
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -p test -n full_features
- name: Build without threads
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d threads,shared -p build
- name: Test without threads
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d threads,shared -p test -n no_threads
- name: Build without encoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d encoders,shared -p build
- name: Test without encoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d encoders,shared -p test -n no_encoders
- name: Build without decoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d decoders,shared -p build
- name: Test without decoders
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -d decoders,shared -p test -n no_decoders
- name: Build with only crc32 check
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -c crc32 -d shared,nls -p build
- name: Test with only crc32 check
run: ./build-aux/ci_build.sh -a "--no-po4a" -b ${{ matrix.build_system }} -c crc32 -d shared,nls -p test -n crc32_only
###############
# Upload Logs #
###############
# Upload the test logs as artifacts if any step has failed.
- uses: actions/upload-artifact@c7d193f32edcb7bfad88892161225aeda64e9392 #v4.0.0
if: ${{ failure() }}
with:
name: ${{ matrix.msys2_env }} ${{ matrix.build_system }} Test Logs
path: build-aux/artifacts

35
.gitignore vendored
View File

@ -5,6 +5,7 @@
.deps
.libs
*.a
*.gcda
*.gcno
*.la
@ -24,6 +25,7 @@ Makefile.in
/libtool
/stamp-h1
build-aux/artifacts
build-aux/compile
build-aux/config.guess
build-aux/config.rpath
@ -36,6 +38,9 @@ build-aux/test-driver
coverage
/doc/internal
/doc/api
/src/liblzma/liblzma.pc
/src/lzmainfo/lzmainfo
/src/xz/xz
@ -57,8 +62,12 @@ coverage
/tests/test_block_header
/tests/test_check
/tests/test_filter_flags
/tests/test_filter_str
/tests/test_hardware
/tests/test_index
/tests/test_index_hash
/tests/test_lzip_decoder
/tests/test_microlzma
/tests/test_memlimit
/tests/test_stream_flags
/tests/test_vli
@ -77,3 +86,29 @@ coverage
/src/liblzma/api/Makefile
/src/lzmainfo/Makefile
/src/xzdec/Makefile
/CMakeCache.txt
/CMakeFiles
/CTestTestfile.cmake
/cmake_install.cmake
/DartConfiguration.tcl
/liblzma-config-version.cmake
/liblzma-config.cmake
/lzcat
/lzcat.1
/lzma
/lzma.1
/Testing
/tests_bin/
/unlzma
/unlzma.1
/unxz
/unxz.1
/xz
/xzcat
/xzcat.1
/xzdec
/windows/*/.vs
/windows/*/liblzma.vcxproj.user
/.vscode/

40
AUTHORS
View File

@ -9,15 +9,13 @@ Authors of XZ Utils
specifically the LZMA SDK <https://7-zip.org/sdk.html>. Without
this code, XZ Utils wouldn't exist.
The SHA-256 implementation in liblzma is based on the code found from
7-Zip <https://7-zip.org/>, which has a modified version of the SHA-256
code found from Crypto++ <https://www.cryptopp.com/>. The SHA-256 code
in Crypto++ was written by Kevin Springle and Wei Dai.
The SHA-256 implementation in liblzma is based on code written by
Wei Dai in Crypto++ Library <https://www.cryptopp.com/>.
Some scripts have been adapted from gzip. The original versions
were written by Jean-loup Gailly, Charles Levert, and Paul Eggert.
Andrew Dudman helped adapting the scripts and their man pages for
XZ Utils.
A few scripts have been adapted from GNU gzip. The original
versions were written by Jean-loup Gailly, Charles Levert, and
Paul Eggert. Andrew Dudman helped adapting the scripts and their
man pages for XZ Utils.
The initial version of the threaded .xz decompressor was written
by Sebastian Andrzej Siewior.
@ -25,15 +23,31 @@ Authors of XZ Utils
The initial version of the .lz (lzip) decoder was written
by Michał Górny.
CLMUL-accelerated CRC code was contributed by Ilya Kurdyukov.
Architecture-specific CRC optimizations were contributed by
Ilya Kurdyukov, Hans Jansen, and Chenxi Mao.
Other authors:
- Jonathan Nieder
- Joachim Henke
The GNU Autotools-based build system contains files from many authors,
which I'm not trying to list here.
Many people have contributed improvements or reported bugs.
Most of these people are mentioned in the file THANKS.
Several people have contributed fixes or reported bugs. Most of them
are mentioned in the file THANKS.
The translations of the command line tools and man pages have been
contributed by many people via the Translation Project:
- https://translationproject.org/domain/xz.html
- https://translationproject.org/domain/xz-man.html
The authors of the translated man pages are in the header comments
of the man page files. In the source package, the authors of the
translations are in po/*.po and po4a/*.po files.
Third-party code whose authors aren't listed here:
- GNU getopt_long() in the 'lib' directory is included for
platforms that don't have a usable getopt_long().
- The build system files from GNU Autoconf, GNU Automake,
GNU Libtool, GNU Gettext, Autoconf Archive, and related files.

File diff suppressed because it is too large Load Diff

105
COPYING
View File

@ -6,60 +6,95 @@ XZ Utils Licensing
is a rough summary of which licenses apply to which parts of this
package (but check the individual files to be sure!):
- liblzma is in the public domain.
- liblzma is under the BSD Zero Clause License (0BSD).
- xz, xzdec, and lzmadec command line tools are in the public
domain unless GNU getopt_long had to be compiled and linked
in from the lib directory. The getopt_long code is under
GNU LGPLv2.1+.
- The command line tools xz, xzdec, lzmadec, and lzmainfo are
under 0BSD except that, on systems that don't have a usable
getopt_long, GNU getopt_long is compiled and linked in from the
'lib' directory. The getopt_long code is under GNU LGPLv2.1+.
- The scripts to grep, diff, and view compressed files have been
adapted from gzip. These scripts and their documentation are
under GNU GPLv2+.
adapted from GNU gzip. These scripts (xzgrep, xzdiff, xzless,
and xzmore) are under GNU GPLv2+. The man pages of the scripts
are under 0BSD; they aren't based on the man pages of GNU gzip.
- All the documentation in the doc directory and most of the
XZ Utils specific documentation files in other directories
are in the public domain.
- Most of the XZ Utils specific documentation that is in
plain text files (like README, INSTALL, PACKAGERS, NEWS,
and ChangeLog) are under 0BSD unless stated otherwise in
the file itself. The files xz-file-format.txt and
lzma-file-format.xt are in the public domain but may
be distributed under the terms of 0BSD too.
- Translated messages are in the public domain.
- Doxygen-generated HTML version of the liblzma API documentation:
While Doxygen is under the GNU GPLv2, the license information
in Doxygen includes the following exception:
- The build system contains public domain files, and files that
are under GNU GPLv2+ or GNU GPLv3+. None of these files end up
in the binaries being built.
Documents produced by doxygen are derivative works
derived from the input used in their production;
they are not affected by this license.
- Test files and test code in the tests directory, and debugging
utilities in the debug directory are in the public domain.
Note: The JavaScript files (under the MIT license) have
been removed from the Doxygen output.
- The extra directory may contain public domain files, and files
that are under various free software licenses.
- The XZ logo (xz-logo.png) included in the Doxygen-generated
documentation is under the Creative Commons BY-SA 4.0 license.
You can do whatever you want with the files that have been put into
the public domain. If you find public domain legally problematic,
take the previous sentence as a license grant. If you still find
the lack of copyright legally problematic, you have too many
lawyers.
- Translated messages and man pages are under 0BSD except that
some old translations are in the public domain.
As usual, this software is provided "as is", without any warranty.
- Test files and test code in the 'tests' directory, and
debugging utilities in the 'debug' directory are under
the BSD Zero Clause License (0BSD).
If you copy significant amounts of public domain code from XZ Utils
- The GNU Autotools based build system contains files that are
under GNU GPLv2+, GNU GPLv3+, and a few permissive licenses.
These files don't affect the licensing of the binaries being
built.
- The extra directory contain files that are under various
free software licenses.
For the files under the BSD Zero Clause License (0BSD), if
a copyright notice is needed, the following is sufficient:
Copyright (C) The XZ Utils authors and contributors
If you copy significant amounts of 0BSD-licensed code from XZ Utils
into your project, acknowledging this somewhere in your software is
polite (especially if it is proprietary, non-free software), but
naturally it is not legally required. Here is an example of a good
notice to put into "about box" or into documentation:
it is not legally required by the license terms. Here is an example
of a good notice to put into "about box" or into documentation:
This software includes code from XZ Utils <https://tukaani.org/xz/>.
This software includes code from XZ Utils
<https://xz.tukaani.org/xz-utils/>.
The following license texts are included in the following files:
- COPYING.0BSD: BSD Zero Clause License
- COPYING.LGPLv2.1: GNU Lesser General Public License version 2.1
- COPYING.GPLv2: GNU General Public License version 2
- COPYING.GPLv3: GNU General Public License version 3
- COPYING.CC-BY-SA-4.0: Creative Commons Attribution-ShareAlike 4.0
International Public License
Note that the toolchain (compiler, linker etc.) may add some code
pieces that are copyrighted. Thus, it is possible that e.g. liblzma
binary wouldn't actually be in the public domain in its entirety
even though it contains no copyrighted code from the XZ Utils source
package.
A note about old XZ Utils releases:
If you have questions, don't hesitate to ask the author(s) for more
information.
XZ Utils releases 5.4.6 and older and 5.5.1alpha have a
significant amount of code put into the public domain and
that obviously remains so. The switch from public domain to
0BSD for newer releases was made in Febrary 2024 because
public domain has (real or perceived) legal ambiguities in
some jurisdictions.
There is very little *practical* difference between public
domain and 0BSD. The main difference likely is that one
shouldn't claim that 0BSD-licensed code is in the public
domain; 0BSD-licensed code is copyrighted but available under
an extremely permissive license. Neither 0BSD nor public domain
require retaining or reproducing author, copyright holder, or
license notices when distributing the software. (Compare to,
for example, BSD 2-Clause "Simplified" License which does have
such requirements.)
If you have questions, don't hesitate to ask for more information.
The contact information is in the README file.

11
COPYING.0BSD Normal file
View File

@ -0,0 +1,11 @@
Permission to use, copy, modify, and/or distribute this
software for any purpose with or without fee is hereby granted.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

427
COPYING.CC-BY-SA-4.0 Normal file
View File

@ -0,0 +1,427 @@
Attribution-ShareAlike 4.0 International
=======================================================================
Creative Commons Corporation ("Creative Commons") is not a law firm and
does not provide legal services or legal advice. Distribution of
Creative Commons public licenses does not create a lawyer-client or
other relationship. Creative Commons makes its licenses and related
information available on an "as-is" basis. Creative Commons gives no
warranties regarding its licenses, any material licensed under their
terms and conditions, or any related information. Creative Commons
disclaims all liability for damages resulting from their use to the
fullest extent possible.
Using Creative Commons Public Licenses
Creative Commons public licenses provide a standard set of terms and
conditions that creators and other rights holders may use to share
original works of authorship and other material subject to copyright
and certain other rights specified in the public license below. The
following considerations are for informational purposes only, are not
exhaustive, and do not form part of our licenses.
Considerations for licensors: Our public licenses are
intended for use by those authorized to give the public
permission to use material in ways otherwise restricted by
copyright and certain other rights. Our licenses are
irrevocable. Licensors should read and understand the terms
and conditions of the license they choose before applying it.
Licensors should also secure all rights necessary before
applying our licenses so that the public can reuse the
material as expected. Licensors should clearly mark any
material not subject to the license. This includes other CC-
licensed material, or material used under an exception or
limitation to copyright. More considerations for licensors:
wiki.creativecommons.org/Considerations_for_licensors
Considerations for the public: By using one of our public
licenses, a licensor grants the public permission to use the
licensed material under specified terms and conditions. If
the licensor's permission is not necessary for any reason--for
example, because of any applicable exception or limitation to
copyright--then that use is not regulated by the license. Our
licenses grant only permissions under copyright and certain
other rights that a licensor has authority to grant. Use of
the licensed material may still be restricted for other
reasons, including because others have copyright or other
rights in the material. A licensor may make special requests,
such as asking that all changes be marked or described.
Although not required by our licenses, you are encouraged to
respect those requests where reasonable. More considerations
for the public:
wiki.creativecommons.org/Considerations_for_licensees
=======================================================================
Creative Commons Attribution-ShareAlike 4.0 International Public
License
By exercising the Licensed Rights (defined below), You accept and agree
to be bound by the terms and conditions of this Creative Commons
Attribution-ShareAlike 4.0 International Public License ("Public
License"). To the extent this Public License may be interpreted as a
contract, You are granted the Licensed Rights in consideration of Your
acceptance of these terms and conditions, and the Licensor grants You
such rights in consideration of benefits the Licensor receives from
making the Licensed Material available under these terms and
conditions.
Section 1 -- Definitions.
a. Adapted Material means material subject to Copyright and Similar
Rights that is derived from or based upon the Licensed Material
and in which the Licensed Material is translated, altered,
arranged, transformed, or otherwise modified in a manner requiring
permission under the Copyright and Similar Rights held by the
Licensor. For purposes of this Public License, where the Licensed
Material is a musical work, performance, or sound recording,
Adapted Material is always produced where the Licensed Material is
synched in timed relation with a moving image.
b. Adapter's License means the license You apply to Your Copyright
and Similar Rights in Your contributions to Adapted Material in
accordance with the terms and conditions of this Public License.
c. BY-SA Compatible License means a license listed at
creativecommons.org/compatiblelicenses, approved by Creative
Commons as essentially the equivalent of this Public License.
d. Copyright and Similar Rights means copyright and/or similar rights
closely related to copyright including, without limitation,
performance, broadcast, sound recording, and Sui Generis Database
Rights, without regard to how the rights are labeled or
categorized. For purposes of this Public License, the rights
specified in Section 2(b)(1)-(2) are not Copyright and Similar
Rights.
e. Effective Technological Measures means those measures that, in the
absence of proper authority, may not be circumvented under laws
fulfilling obligations under Article 11 of the WIPO Copyright
Treaty adopted on December 20, 1996, and/or similar international
agreements.
f. Exceptions and Limitations means fair use, fair dealing, and/or
any other exception or limitation to Copyright and Similar Rights
that applies to Your use of the Licensed Material.
g. License Elements means the license attributes listed in the name
of a Creative Commons Public License. The License Elements of this
Public License are Attribution and ShareAlike.
h. Licensed Material means the artistic or literary work, database,
or other material to which the Licensor applied this Public
License.
i. Licensed Rights means the rights granted to You subject to the
terms and conditions of this Public License, which are limited to
all Copyright and Similar Rights that apply to Your use of the
Licensed Material and that the Licensor has authority to license.
j. Licensor means the individual(s) or entity(ies) granting rights
under this Public License.
k. Share means to provide material to the public by any means or
process that requires permission under the Licensed Rights, such
as reproduction, public display, public performance, distribution,
dissemination, communication, or importation, and to make material
available to the public including in ways that members of the
public may access the material from a place and at a time
individually chosen by them.
l. Sui Generis Database Rights means rights other than copyright
resulting from Directive 96/9/EC of the European Parliament and of
the Council of 11 March 1996 on the legal protection of databases,
as amended and/or succeeded, as well as other essentially
equivalent rights anywhere in the world.
m. You means the individual or entity exercising the Licensed Rights
under this Public License. Your has a corresponding meaning.
Section 2 -- Scope.
a. License grant.
1. Subject to the terms and conditions of this Public License,
the Licensor hereby grants You a worldwide, royalty-free,
non-sublicensable, non-exclusive, irrevocable license to
exercise the Licensed Rights in the Licensed Material to:
a. reproduce and Share the Licensed Material, in whole or
in part; and
b. produce, reproduce, and Share Adapted Material.
2. Exceptions and Limitations. For the avoidance of doubt, where
Exceptions and Limitations apply to Your use, this Public
License does not apply, and You do not need to comply with
its terms and conditions.
3. Term. The term of this Public License is specified in Section
6(a).
4. Media and formats; technical modifications allowed. The
Licensor authorizes You to exercise the Licensed Rights in
all media and formats whether now known or hereafter created,
and to make technical modifications necessary to do so. The
Licensor waives and/or agrees not to assert any right or
authority to forbid You from making technical modifications
necessary to exercise the Licensed Rights, including
technical modifications necessary to circumvent Effective
Technological Measures. For purposes of this Public License,
simply making modifications authorized by this Section 2(a)
(4) never produces Adapted Material.
5. Downstream recipients.
a. Offer from the Licensor -- Licensed Material. Every
recipient of the Licensed Material automatically
receives an offer from the Licensor to exercise the
Licensed Rights under the terms and conditions of this
Public License.
b. Additional offer from the Licensor -- Adapted Material.
Every recipient of Adapted Material from You
automatically receives an offer from the Licensor to
exercise the Licensed Rights in the Adapted Material
under the conditions of the Adapter's License You apply.
c. No downstream restrictions. You may not offer or impose
any additional or different terms or conditions on, or
apply any Effective Technological Measures to, the
Licensed Material if doing so restricts exercise of the
Licensed Rights by any recipient of the Licensed
Material.
6. No endorsement. Nothing in this Public License constitutes or
may be construed as permission to assert or imply that You
are, or that Your use of the Licensed Material is, connected
with, or sponsored, endorsed, or granted official status by,
the Licensor or others designated to receive attribution as
provided in Section 3(a)(1)(A)(i).
b. Other rights.
1. Moral rights, such as the right of integrity, are not
licensed under this Public License, nor are publicity,
privacy, and/or other similar personality rights; however, to
the extent possible, the Licensor waives and/or agrees not to
assert any such rights held by the Licensor to the limited
extent necessary to allow You to exercise the Licensed
Rights, but not otherwise.
2. Patent and trademark rights are not licensed under this
Public License.
3. To the extent possible, the Licensor waives any right to
collect royalties from You for the exercise of the Licensed
Rights, whether directly or through a collecting society
under any voluntary or waivable statutory or compulsory
licensing scheme. In all other cases the Licensor expressly
reserves any right to collect such royalties.
Section 3 -- License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the
following conditions.
a. Attribution.
1. If You Share the Licensed Material (including in modified
form), You must:
a. retain the following if it is supplied by the Licensor
with the Licensed Material:
i. identification of the creator(s) of the Licensed
Material and any others designated to receive
attribution, in any reasonable manner requested by
the Licensor (including by pseudonym if
designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of
warranties;
v. a URI or hyperlink to the Licensed Material to the
extent reasonably practicable;
b. indicate if You modified the Licensed Material and
retain an indication of any previous modifications; and
c. indicate the Licensed Material is licensed under this
Public License, and include the text of, or the URI or
hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any
reasonable manner based on the medium, means, and context in
which You Share the Licensed Material. For example, it may be
reasonable to satisfy the conditions by providing a URI or
hyperlink to a resource that includes the required
information.
3. If requested by the Licensor, You must remove any of the
information required by Section 3(a)(1)(A) to the extent
reasonably practicable.
b. ShareAlike.
In addition to the conditions in Section 3(a), if You Share
Adapted Material You produce, the following conditions also apply.
1. The Adapter's License You apply must be a Creative Commons
license with the same License Elements, this version or
later, or a BY-SA Compatible License.
2. You must include the text of, or the URI or hyperlink to, the
Adapter's License You apply. You may satisfy this condition
in any reasonable manner based on the medium, means, and
context in which You Share Adapted Material.
3. You may not offer or impose any additional or different terms
or conditions on, or apply any Effective Technological
Measures to, Adapted Material that restrict exercise of the
rights granted under the Adapter's License You apply.
Section 4 -- Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that
apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
to extract, reuse, reproduce, and Share all or a substantial
portion of the contents of the database;
b. if You include all or a substantial portion of the database
contents in a database in which You have Sui Generis Database
Rights, then the database in which You have Sui Generis Database
Rights (but not its individual contents) is Adapted Material,
including for purposes of Section 3(b); and
c. You must comply with the conditions in Section 3(a) if You Share
all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not
replace Your obligations under this Public License where the Licensed
Rights include other Copyright and Similar Rights.
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
c. The disclaimer of warranties and limitation of liability provided
above shall be interpreted in a manner that, to the extent
possible, most closely approximates an absolute disclaimer and
waiver of all liability.
Section 6 -- Term and Termination.
a. This Public License applies for the term of the Copyright and
Similar Rights licensed here. However, if You fail to comply with
this Public License, then Your rights under this Public License
terminate automatically.
b. Where Your right to use the Licensed Material has terminated under
Section 6(a), it reinstates:
1. automatically as of the date the violation is cured, provided
it is cured within 30 days of Your discovery of the
violation; or
2. upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any
right the Licensor may have to seek remedies for Your violations
of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the
Licensed Material under separate terms or conditions or stop
distributing the Licensed Material at any time; however, doing so
will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
License.
Section 7 -- Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different
terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the
Licensed Material not stated herein are separate from and
independent of the terms and conditions of this Public License.
Section 8 -- Interpretation.
a. For the avoidance of doubt, this Public License does not, and
shall not be interpreted to, reduce, limit, restrict, or impose
conditions on any use of the Licensed Material that could lawfully
be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License is
deemed unenforceable, it shall be automatically reformed to the
minimum extent necessary to make it enforceable. If the provision
cannot be reformed, it shall be severed from this Public License
without affecting the enforceability of the remaining terms and
conditions.
c. No term or condition of this Public License will be waived and no
failure to comply consented to unless expressly agreed to by the
Licensor.
d. Nothing in this Public License constitutes or may be interpreted
as a limitation upon, or waiver of, any privileges and immunities
that apply to the Licensor or You, including from the legal
processes of any jurisdiction or authority.
=======================================================================
Creative Commons is not a party to its public
licenses. Notwithstanding, Creative Commons may elect to apply one of
its public licenses to material it publishes and in those instances
will be considered the “Licensor.” The text of the Creative Commons
public licenses is dedicated to the public domain under the CC0 Public
Domain Dedication. Except for the limited purpose of indicating that
material is shared under a Creative Commons public license or as
otherwise permitted by the Creative Commons policies published at
creativecommons.org/policies, Creative Commons does not authorize the
use of the trademark "Creative Commons" or any other trademark or logo
of Creative Commons without its prior written consent including,
without limitation, in connection with any unauthorized modifications
to any of its public licenses or any other arrangements,
understandings, or agreements concerning use of licensed material. For
the avoidance of doubt, this paragraph does not form part of the
public licenses.
Creative Commons may be contacted at creativecommons.org.

View File

@ -1,7 +1,7 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
@ -645,7 +645,7 @@ the "copyright" line and a pointer to where the full notice is found.
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
along with this program. If not, see <https://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
@ -664,11 +664,11 @@ might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
<https://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
<https://www.gnu.org/licenses/why-not-lgpl.html>.

View File

@ -1,6 +1,6 @@
See the commit log in the git repository:
git clone https://git.tukaani.org/xz.git
git clone https://github.com/tukaani-project/xz
Note that "make dist" doesn't put this tiny file into the package.
Instead, the git commit log is used as ChangeLog. See dist-hook in

File diff suppressed because it is too large Load Diff

287
INSTALL
View File

@ -22,13 +22,16 @@ XZ Utils Installation
3. xzgrep and other scripts
3.1. Dependencies
3.2. PATH
4. Troubleshooting
4.1. "No C99 compiler was found."
4.2. "No POSIX conforming shell (sh) was found."
4.3. configure works but build fails at crc32_x86.S
4.4. Lots of warnings about symbol visibility
4.5. "make check" fails
4.6. liblzma.so (or similar) not found when running xz
4. Tests
4.1 Testing in parallel
4.2 Cross compiling
5. Troubleshooting
5.1. "No C99 compiler was found."
5.2. "No POSIX conforming shell (sh) was found."
5.3. configure works but build fails at crc32_x86.S
5.4. Lots of warnings about symbol visibility
5.5. "make check" fails
5.6. liblzma.so (or similar) not found when running xz
0. Preface
@ -100,7 +103,7 @@ XZ Utils Installation
XZ Utils doesn't have code to detect the amount of physical RAM and
number of CPU cores on MINIX 3.
See section 4.4 in this file about symbol visibility warnings (you
See section 5.4 in this file about symbol visibility warnings (you
may want to pass gl_cv_cc_visibility=no to configure).
@ -127,56 +130,50 @@ XZ Utils Installation
missing from PATH (/usr/xpg4/bin or /usr/xpg6/bin). Nowadays
/usr/xpg4/bin is added to the script PATH by default on Solaris
(see --enable-path-for-scripts=PREFIX in section 2), but old xz
releases needed extra steps. See sections 4.5 and 3.2 for more
releases needed extra steps. See sections 5.5 and 3.2 for more
information.
1.2.6. Tru64
If you try to use the native C compiler on Tru64 (passing CC=cc to
configure), you may need the workaround mention in section 4.1 in
configure), you may need the workaround mention in section 5.1 in
this file (pass also ac_cv_prog_cc_c99= to configure).
1.2.7. Windows
If it is enough to build liblzma (no command line tools):
The "windows" directory contains instructions for a few types
of builds:
- There is CMake support. It should be good enough to build
static liblzma or liblzma.dll with Visual Studio. The CMake
support may work with MinGW or MinGW-w64. Read the comment
in the beginning of CMakeLists.txt before running CMake!
- INSTALL-MinGW-w64_with_CMake.txt
Simple instructions how to build XZ Utils natively on
Windows using only CMake and a prebuilt toolchain
(GCC + MinGW-w64 or Clang/LLVM + MinGW-w64).
- There are Visual Studio project files under the "windows"
directory. See windows/INSTALL-MSVC.txt. In the future the
project files will be removed when CMake support is good
enough. Thus, please test the CMake version and help fix
possible issues.
- INSTALL-MinGW-w64_with_Autotools.txt
Native build under MSYS2 or cross-compilation from
GNU/Linux using a bash script that creates a .zip
and .7z archives of the binaries and documentation.
The related file README-Windows.txt is for the
resulting binary package.
To build also the command line tools:
- INSTALL-MSVC.txt
Building with MSVC / Visual Studio and CMake.
- MinGW-w64 + MSYS (32-bit and 64-bit x86): This is used
for building the official binary packages for Windows.
There is windows/build.bash to ease packaging XZ Utils with
MinGW(-w64) + MSYS into a redistributable .zip or .7z file.
See windows/INSTALL-MinGW.txt for more information.
- liblzma-crt-mixing.txt
Documentation what to take into account as a programmer
if liblzma.dll and the application don't use the same
CRT (MSVCRT or UCRT).
- MinGW + MSYS (32-bit x86): I haven't recently tested this.
Other choices:
- Cygwin 1.7.35 and later: NOTE that using XZ Utils >= 5.2.0
under Cygwin older than 1.7.35 can lead to DATA LOSS! If
you must use an old Cygwin version, stick to XZ Utils 5.0.x
which is safe under older Cygwin versions. You can check
the Cygwin version with the command "cygcheck -V".
- Cygwin: https://cygwin.com/
Building on Cygwin can be done like on many POSIX operating
systems. XZ Utils >= 5.2.0 isn't compatible with Cygwin older
than 1.7.35 (data loss!). 1.7.35 was released on 2015-03-04.
It may be possible to build liblzma with other toolchains too, but
that will probably require writing a separate makefile. Building
the command line tools with non-GNU toolchains will be harder than
building only liblzma.
Even if liblzma is built with MinGW(-w64), the resulting DLL can
be used by other compilers and linkers, including MSVC. See
windows/README-Windows.txt for details.
- MSYS2: https://www.msys2.org/
1.2.8. DOS
@ -356,20 +353,20 @@ XZ Utils Installation
with --docdir=DIR.
--disable-assembler
liblzma includes some assembler optimizations. Currently
there is only assembler code for CRC32 and CRC64 for
32-bit x86.
This disables CRC32 and CRC64 assembly code on
32-bit x86. This option currently does nothing
on other architectures (not even on x86-64).
All the assembler code in liblzma is position-independent
code, which is suitable for use in shared libraries and
position-independent executables. So far only i386
instructions are used, but the code is optimized for i686
class CPUs. If you are compiling liblzma exclusively for
The 32-bit x86 assembly is position-independent code
which is suitable for use in shared libraries and
position-independent executables. It uses only i386
instructions but the code is optimized for i686 class
CPUs. If you are compiling liblzma exclusively for
pre-i686 systems, you may want to disable the assembler
code.
--disable-clmul-crc
Disable the use carryless multiplication for CRC
Disable the use of carryless multiplication for CRC
calculation even if compiler support for it is detected.
The code uses runtime detection of SSSE3, SSE4.1, and
CLMUL instructions on x86. On 32-bit x86 this currently
@ -380,6 +377,16 @@ XZ Utils Installation
required extensions (-msse4.1 -mpclmul) then runtime
detection isn't used and the generic code is omitted.
--disable-arm64-crc32
Disable the use of the ARM64 CRC32 instruction extension
even if compiler support for it is detected. The code will
detect support for the instruction at runtime.
If using compiler options that unconditionally allow the
required extensions (-march=armv8-a+crc or -march=armv8.1-a
and later) then runtime detection isn't used and the
generic code is omitted.
--enable-unaligned-access
Allow liblzma to use unaligned memory access for 16-bit,
32-bit, and 64-bit loads and stores. This should be
@ -389,8 +396,43 @@ XZ Utils Installation
slow. This option shouldn't be used on systems that
rely on such emulation.
Unaligned access is enabled by default on x86, x86-64,
big endian PowerPC, some ARM, and some ARM64 systems.
Unaligned access is enabled by default on these:
- 32-bit x86
- 64-bit x86-64
- 32-bit big endian PowerPC
- 64-bit big endian PowerPC
- 64-bit little endian PowerPC
- some RISC-V [1]
- some 32-bit ARM [2]
- some 64-bit ARM64 [2] (NOTE: Autodetection bug
if using GCC -mstrict-align, see below.)
[1] Unaligned access is enabled by default if
configure sees that the C compiler
#defines __riscv_misaligned_fast.
[2] Unaligned access is enabled by default if
configure sees that the C compiler
#defines __ARM_FEATURE_UNALIGNED:
- ARMv7 + GCC or Clang: It works. The options
-munaligned-access and -mno-unaligned-access
affect this macro correctly.
- ARM64 + Clang: It works. The options
-munaligned-access, -mno-unaligned-access,
and -mstrict-align affect this macro correctly.
Clang >= 17 supports -mno-strict-align too.
- ARM64 + GCC: It partially works. The macro
is always #defined by GCC versions at least
up to 13.2, even when using -mstrict-align.
If building for strict-align ARM64, the
configure option --disable-unaligned-access
should be used if using a GCC version that has
this issue because otherwise the performance
may be degraded. It likely won't crash due to
how unaligned access is done in the C code.
--enable-unsafe-type-punning
This enables use of code like
@ -455,14 +497,16 @@ XZ Utils Installation
win95 Use Windows 95 compatible threads. This
is compatible with Windows XP and later
too. This is the default for 32-bit x86
Windows builds. The `win95' threading is
incompatible with --enable-small.
Windows builds. Unless the compiler
supports __attribute__((__constructor__)),
the 'win95' threading is incompatible with
--enable-small.
vista Use Windows Vista compatible threads. The
resulting binaries won't run on Windows XP
or older. This is the default for Windows
excluding 32-bit x86 builds (that is, on
x86-64 the default is `vista').
x86-64 the default is 'vista').
no Disable threading support. This is the
same as using --disable-threads.
@ -474,14 +518,25 @@ XZ Utils Installation
calls any liblzma functions from more than
one thread, something bad may happen.
--enable-ifunc
Use __attribute__((__ifunc__())) in liblzma. This is
enabled by default on GNU/Linux and FreeBSD.
The ifunc attribute is incompatible with
-fsanitize=address. --disable-ifunc must be used
if any -fsanitize= option is specified in CFLAGS.
--enable-sandbox=METHOD
There is limited sandboxing support in the xz tool. If
built with sandbox support, it's used automatically when
(de)compressing exactly one file to standard output and
the options --files or --files0 weren't used. This is a
common use case, for example, (de)compressing .tar.xz
files via GNU tar. The sandbox is also used for
single-file `xz --test' or `xz --list'.
There is limited sandboxing support in the xz and xzdec
tools. If built with sandbox support, xz uses it
automatically when (de)compressing exactly one file to
standard output when the options --files or --files0 aren't
used. This is a common use case, for example,
(de)compressing .tar.xz files via GNU tar. The sandbox is
also used for single-file 'xz --test' or 'xz --list'.
xzdec always uses the sandbox, except when more than one
file are decompressed. In this case it will enable the
sandbox for the last file that is decompressed.
Supported METHODs:
@ -493,7 +548,7 @@ XZ Utils Installation
no Disable sandboxing support.
capsicum
Use Capsicum (FreeBSD >= 10) for
Use Capsicum (FreeBSD >= 10.2) for
sandboxing. If no Capsicum support
is found, configure will give an error.
@ -501,10 +556,50 @@ XZ Utils Installation
sandboxing. If pledge(2) isn't found,
configure will give an error.
--enable-symbol-versions
Use symbol versioning for liblzma. This is enabled by
default on GNU/Linux, other GNU-based systems, and
FreeBSD.
landlock
Use Landlock (Linux >= 5.13) for
sandboxing. If no Landlock support
is found, configure will give an error.
--enable-symbol-versions[=VARIANT]
Use symbol versioning for liblzma shared library.
This is enabled by default on GNU/Linux (glibc only),
other GNU-based systems, and FreeBSD.
Symbol versioning is never used for static liblzma. This
option is ignored when not building a shared library.
Supported VARIANTs:
no Disable symbol versioning. This is the
same as using --disable-symbol-versions.
auto Autodetect between "no", "linux",
and "generic".
yes Autodetect between "linux" and
"generic". This forces symbol
versioning to be used when
building a shared library.
generic Generic version is the default for
FreeBSD and GNU/Linux on MicroBlaze.
This is also used on GNU/Linux when
building with NVIDIA HPC Compiler
because the compiler doesn't support
the features required for the "linux"
variant below.
linux Special version for GNU/Linux (glibc
only). This adds a few extra symbol
versions for compatibility with binaries
that have been linked against a liblzma
version that has been patched with
"xz-5.2.2-compat-libs.patch" from
RHEL/CentOS 7. That patch was used
by some build tools outside of
RHEL/CentOS 7 too.
--enable-debug
This enables the assert() macro and possibly some other
@ -596,7 +691,7 @@ XZ Utils Installation
a fallback xzdiff will use mkdir to securely create a temporary
directory. Having mktemp available is still recommended since the
mkdir fallback method isn't as robust as mktemp is. The original
mktemp can be found from <http://www.mktemp.org/>. On GNU, most will
mktemp can be found from <https://www.mktemp.org/>. On GNU, most will
use the mktemp program from GNU coreutils instead of the original
implementation. Both mktemp versions are fine.
@ -624,10 +719,54 @@ XZ Utils Installation
src/scripts/xz*.in
4. Troubleshooting
4. Tests
--------
The test framework can be built and run by executing "make check" in
the build directory. The tests are a mix of executables and POSIX
shell scripts (sh). All tests should pass if the default configuration
is used. Disabling features through the configure options may cause
some tests to be skipped. If any tests do not pass, see section 5.5.
4.1. Testing in parallel
The tests can be run in parallel using the "-j" make option on systems
that support it. For instance, "make -j4 check" will run up to four
tests simultaneously.
4.2. Cross compiling
The tests can be built without running them:
make check TESTS=
The TESTS variable is the list of tests you wish to run. Leaving it
empty will compile the tests without running any.
If the tests are copied to a target machine to execute, the test data
files in the directory tests/files must also be copied. The tests
search for the data files using the environment variable $srcdir,
expecting to find the data files under $srcdir/files/. If $srcdir
isn't set then it defaults to the current directory.
The shell script tests can be copied from the source directory to the
target machine to execute. In addition to the test files, these tests
will expect the following relative file paths to execute properly:
./create_compress_files
../config.h
../src/xz/xz
../src/xzdec/xzdec
../src/scripts/xzdiff
../src/scripts/xzgrep
5. Troubleshooting
------------------
4.1. "No C99 compiler was found."
5.1. "No C99 compiler was found."
You need a C99 compiler to build XZ Utils. If the configure script
cannot find a C99 compiler and you think you have such a compiler
@ -642,7 +781,7 @@ XZ Utils Installation
support enough C99.
4.2. "No POSIX conforming shell (sh) was found."
5.2. "No POSIX conforming shell (sh) was found."
xzgrep and other scripts need a shell that (roughly) conforms
to POSIX. The configure script tries to find such a shell. If
@ -652,7 +791,7 @@ XZ Utils Installation
this error by passing --disable-scripts to configure.
4.3. configure works but build fails at crc32_x86.S
5.3. configure works but build fails at crc32_x86.S
The easy fix is to pass --disable-assembler to the configure script.
@ -669,7 +808,7 @@ XZ Utils Installation
(see INSTALL.generic).
4.4. Lots of warnings about symbol visibility
5.4. Lots of warnings about symbol visibility
On some systems where symbol visibility isn't supported, GCC may
still accept the visibility options and attributes, which will make
@ -681,7 +820,7 @@ XZ Utils Installation
using --enable-werror.
4.5. "make check" fails
5.5. "make check" fails
If the other tests pass but test_scripts.sh fails, then the problem
is in the scripts in src/scripts. Comparing the contents of
@ -707,7 +846,7 @@ XZ Utils Installation
information.
4.6. liblzma.so (or similar) not found when running xz
5.6. liblzma.so (or similar) not found when running xz
If you installed the package with "make install" and get an error
about liblzma.so (or a similarly named file) being missing, try

View File

@ -1,9 +1,5 @@
##
## SPDX-License-Identifier: 0BSD
## Author: Lasse Collin
##
## This file has been put into the public domain.
## You can do whatever you want with this file.
##
# Use -n to prevent gzip from adding a timestamp to the .gz headers.
GZIP_ENV = -9n
@ -21,11 +17,11 @@ if COND_DOC
dist_doc_DATA = \
AUTHORS \
COPYING \
COPYING.0BSD \
COPYING.GPLv2 \
NEWS \
README \
THANKS \
TODO \
doc/faq.txt \
doc/history.txt \
doc/xz-file-format.txt \
@ -38,31 +34,42 @@ dist_examples_DATA = \
doc/examples/02_decompress.c \
doc/examples/03_compress_custom.c \
doc/examples/04_compress_easy_mt.c \
doc/examples/11_file_info.c \
doc/examples/Makefile
examplesolddir = $(docdir)/examples_old
dist_examplesold_DATA = \
doc/examples_old/xz_pipe_comp.c \
doc/examples_old/xz_pipe_decomp.c
# Install the Doxygen generated documentation if they were built.
install-data-local:
if test -d "$(srcdir)/doc/api" ; then \
$(MKDIR_P) "$(DESTDIR)$(docdir)/api" && \
$(INSTALL_DATA) "$(srcdir)"/doc/api/* \
"$(DESTDIR)$(docdir)/api"; \
fi
# Remove the Doxygen generated documentation when uninstalling.
uninstall-local:
rm -rf "$(DESTDIR)$(docdir)/api"
endif
EXTRA_DIST = \
po4a \
extra \
dos \
windows \
macosx \
cmake \
dos \
doxygen \
extra \
po4a \
windows \
CMakeLists.txt \
autogen.sh \
Doxyfile.in \
COPYING.CC-BY-SA-4.0 \
COPYING.GPLv2 \
COPYING.GPLv3 \
COPYING.LGPLv2.1 \
INSTALL.generic \
PACKAGERS \
TODO \
autogen.sh \
build-aux/manconv.sh \
build-aux/version.sh
build-aux/version.sh \
doc/xz-logo.png \
po/xz.pot-header
ACLOCAL_AMFLAGS = -I m4
@ -81,7 +88,7 @@ manfiles = \
dist-hook:
if test -d "$(srcdir)/.git" && type git > /dev/null 2>&1; then \
( cd "$(srcdir)" && git log --date=iso --stat \
6060f7dc76fd6c2a8a1f8e85d0e4d86bb78273e6^..HEAD ) \
b69da6d4bb6bb11fc0cf066920791990d2b22a06^..HEAD ) \
> "$(distdir)/ChangeLog"; \
fi
if type groff > /dev/null 2>&1 && type ps2pdf > /dev/null 2>&1; then \
@ -100,6 +107,11 @@ dist-hook:
> "$$dest/txt/$$BASE.txt"; \
done; \
fi
if test -d "$(srcdir)/doc/api" ; then \
$(MKDIR_P) "$(distdir)/doc/api" && \
$(INSTALL_DATA) "$(srcdir)"/doc/api/* \
"$(distdir)/doc/api"; \
fi
# This works with GNU tar and gives cleaner package than normal 'make dist'.
# This also ensures that the man page translations are up to date (dist-hook
@ -107,6 +119,7 @@ dist-hook:
mydist:
sh "$(srcdir)/src/liblzma/validate_map.sh"
cd "$(srcdir)/po4a" && sh update-po
cd "$(srcdir)/doxygen" && sh update-doxygen
VERSION=$(VERSION); \
if test -d "$(srcdir)/.git" && type git > /dev/null 2>&1; then \
SNAPSHOT=`cd "$(srcdir)" && git describe --abbrev=4 | cut -b2-`; \

623
NEWS
View File

@ -2,6 +2,586 @@
XZ Utils Release Notes
======================
5.6.1 (2024-03-09)
* liblzma: Fixed two bugs relating to GNU indirect function (IFUNC)
with GCC. The more serious bug caused a program linked with
liblzma to crash on start up if the flag -fprofile-generate was
used to build liblzma. The second bug caused liblzma to falsely
report an invalid write to Valgrind when loading liblzma.
* xz: Changed the messages for thread reduction due to memory
constraints to only appear under the highest verbosity level.
* Build:
- Fixed a build issue when the header file <linux/landlock.h>
was present on the system but the Landlock system calls were
not defined in <sys/syscall.h>.
- The CMake build now warns and disables NLS if both gettext
tools and pre-created .gmo files are missing. Previously,
this caused the CMake build to fail.
* Minor improvements to man pages.
* Minor improvements to tests.
5.6.0 (2024-02-24)
This bumps the minor version of liblzma because new features were
added. The API and ABI are still backward compatible with liblzma
5.4.x and 5.2.x and 5.0.x.
NOTE: As described in the NEWS for 5.5.2beta, the core components
are now under the BSD Zero Clause License (0BSD).
Since 5.5.2beta:
* liblzma:
- Disabled the branchless C variant in the LZMA decoder based
on the benchmark results from the community.
- Disabled x86-64 inline assembly on x32 to fix the build.
* Sandboxing support in xz:
- Landlock is now used even when xz needs to create files.
In this case the sandbox has to be more permissive than
when no files need to be created. A similar thing was
already in use with pledge(2) since 5.3.4alpha.
- Landlock and pledge(2) are now stricter when reading from
more than one input file and only writing to standard output.
- Added support for Landlock ABI version 4.
* CMake:
- Default to -O2 instead of -O3 with CMAKE_BUILD_TYPE=Release.
-O3 is not useful for speed and makes the code larger.
- Now builds lzmainfo and lzmadec.
- xzdiff, xzgrep, xzless, xzmore, and their symlinks are now
installed. The scripts are also tested during "make test".
- Added translation support for xz, lzmainfo, and the
man pages.
- Applied the symbol versioning workaround for MicroBlaze that
is used in the Autotools build.
- The general XZ Utils and liblzma API documentation is now
installed.
- The CMake component names were changed a little and several
were added. liblzma_Runtime and liblzma_Development are
unchanged.
- Minimum required CMake version is now 3.14. However,
translation support is disabled with CMake versions
older than 3.20.
- The CMake-based build is now close to feature parity with the
Autotools-based build. Most importantly a few tests aren't
run yet. Testing the CMake-based build on different operating
systems would be welcome now. See the comment at the top of
CMakeLists.txt.
* Fixed a bug in the Autotools feature test for ARM64 CRC32
instruction support for old versions of Clang. This did not
affect the CMake build.
* Windows:
- The build instructions in INSTALL and windows/INSTALL*.txt
were revised completely.
- windows/build-with-cmake.bat along with the instructions
in windows/INSTALL-MinGW-w64_with_CMake.txt should make
it very easy to build liblzma.dll and xz.exe on Windows
using CMake and MinGW-w64 with either GCC or Clang/LLVM.
- windows/build.bash was updated. It now works on MSYS2 and
on GNU/Linux (cross-compiling) to create a .zip and .7z
package for 32-bit and 64-bit x86 using GCC + MinGW-w64.
* The TODO file is no longer installed as part of the
documentation. The file is out of date and does not reflect
the actual tasks that will be completed in the future.
* Translations:
- Translated lzmainfo man pages are now installed. These
had been forgotten in earlier versions.
- Updated Croatian, Esperanto, German, Hungarian, Korean,
Polish, Romanian, Spanish, Swedish, Vietnamese, and Ukrainian
translations.
- Updated German, Korean, Romanian, and Ukrainian man page
translations.
* Added a few tests.
Summary of new features added in the 5.5.x development releases:
* liblzma:
- LZMA decoder: Speed optimizations to the C code and
added GCC & Clang compatible inline assembly for x86-64.
- Added lzma_mt_block_size() to recommend a Block size for
multithreaded encoding.
- Added CLMUL-based CRC32 on x86-64 and E2K with runtime
processor detection. Similar to CRC64, on 32-bit x86 it
isn't available unless --disable-assembler is used.
- Optimized the CRC32 calculation on ARM64 platforms using the
CRC32 instructions. Runtime detection for the instruction is
used on GNU/Linux, FreeBSD, Windows, and macOS. If the
compiler flags indicate unconditional CRC32 instruction
support (+crc) then the generic version is not built.
- Added definitions of mask values like
LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.
* xz:
- Multithreaded mode is now the default. This improves
compression speed and creates .xz files that can be
decompressed in multithreaded mode. The downsides are
increased memory usage and slightly worse compression ratio.
- Added a new command line option --filters to set the filter
chain using the liblzma filter string syntax.
- Added new command line options --filters1 ... --filters9 to
set additional filter chains using the liblzma filter string
syntax. The --block-list option now allows specifying filter
chains that were set using these new options.
- Ported the command line tools to Windows MSVC.
Visual Studio 2015 or later is required.
* Added lz4 support to xzdiff/xzcmp and xzgrep.
5.5.2beta (2024-02-14)
* Licensing change: The core components are now under the
BSD Zero Clause License (0BSD). In XZ Utils 5.4.6 and older
and 5.5.1alpha these components are in the public domain and
obviously remain so; the change affects the new releases only.
0BSD is an extremely permissive license which doesn't require
retaining or reproducing copyright or license notices when
distributing the code, thus in practice there is extremely
little difference to public domain.
* liblzma
- Significant speed optimizations to the LZMA decoder were
made. There are now three variants that can be chosen at
build time:
* Basic C version: This is a few percent faster than
5.4.x due to some new optimizations.
* Branchless C: This is currently the default on platforms
for which there is no assembly code. This should be a few
percent faster than the basic C version.
* x86-64 inline assembly. This works with GCC and Clang.
The default choice can currently be overridden by setting
LZMA_RANGE_DECODER_CONFIG in CPPFLAGS: 0 means the basic
version and 3 means that branchless C version.
- Optimized the CRC32 calculation on ARM64 platforms using the
CRC32 instructions. The instructions are optional in ARMv8.0
and are required in ARMv8.1 and later. Runtime detection for
the instruction is used on GNU/Linux, FreeBSD, Windows, and
macOS. If the compiler flags indicate unconditional CRC32
instruction support (+crc) then the generic version is not
built.
* Added lz4 support to xzdiff/xzcmp and xzgrep.
* Man pages of xzdiff/xzcmp, xzgrep, and xzmore were rewritten
to simplify licensing of the man page translations.
* Translations:
- Updated Chinese (simplified), German, Korean, Polish,
Romanian, Spanish, Swedish, and Ukrainian translations.
- Updated German, Korean, Romanian, and Ukrainian man page
translations.
* Small improvements to the tests.
* Added doc/examples/11_file_info.c. It was added to the Git
repository in 2017 but forgotten to be added into distribution
tarballs.
* Removed doc/examples_old. These were from 2012.
* Removed the macos/build.sh script. It had not been updated
since 2013.
5.5.1alpha (2024-01-26)
* Added a new filter for RISC-V binaries. The filter can be used
for 32-bit and 64-bit binaries with either little or big
endianness. In liblzma, the Filter ID is LZMA_FILTER_RISCV (0x0B)
and the xz option is --riscv. liblzma filter string syntax
recognizes this filter as "riscv".
* liblzma:
- Added lzma_mt_block_size() to recommend a Block size for
multithreaded encoding
- Added CLMUL-based CRC32 on x86-64 and E2K with runtime
processor detection. Similar to CRC64, on 32-bit x86 it
isn't available unless --disable-assembler is used.
- Implemented GNU indirect function (IFUNC) as a runtime
function dispatching method for CRC32 and CRC64 fast
implementations on x86. Only GNU/Linux (glibc) and FreeBSD
builds will use IFUNC, unless --enable-ifunc is specified to
configure.
- Added definitions of mask values like
LZMA_INDEX_CHECK_MASK_CRC32 to <lzma/index.h>.
- The XZ logo is now included in the Doxygen generated
documentation. It is licensed under Creative Commons
Attribution-ShareAlike 4.0.
* xz:
- Multithreaded mode is now the default. This improves
compression speed and creates .xz files that can be
decompressed multithreaded at the cost of increased memory
usage and slightly worse compression ratio.
- Added new command line option --filters to set the filter
chain using liblzma filter string syntax.
- Added new command line options --filters1 ... --filters9 to
set additional filter chains using liblzma filter string
syntax. The --block-list option now allows specifying filter
chains that were set using these new options.
- Added support for Linux Landlock as a sandboxing method.
- xzdec now supports pledge(2), Capsicum, and Linux Landlock as
sandboxing methods.
- Progress indicator time stats remain accurate after pausing
xz with SIGTSTP.
- Ported xz and xzdec to Windows MSVC. Visual Studio 2015 or
later is required.
* CMake Build:
- Supports pledge(2), Capsicum, and Linux Landlock sandboxing
methods.
- Replacement functions for getopt_long() are used on platforms
that do not have it.
* Enabled unaligned access by default on PowerPC64LE and on RISC-V
targets that define __riscv_misaligned_fast.
* Tests:
- Added two new fuzz targets to OSS-Fuzz.
- Implemented Continuous Integration (CI) testing using
GitHub Actions.
* Changed quoting style from `...' to '...' in all messages,
scripts, and documentation.
* Added basic Codespell support to help catch typo errors.
5.4.6 (2024-01-26)
* Fixed a bug involving internal function pointers in liblzma not
being initialized to NULL. The bug can only be triggered if
lzma_filters_update() is called on a LZMA1 encoder, so it does
not affect xz or any application known to us that uses liblzma.
* xz:
- Fixed a regression introduced in 5.4.2 that caused encoding
in the raw format to unnecessarily fail if --suffix was not
used. For instance, the following command no longer reports
that --suffix must be used:
echo foo | xz --format=raw --lzma2 | wc -c
- Fixed an issue on MinGW-w64 builds that prevented reading
from or writing to non-terminal character devices like NUL.
* Added a new test.
5.4.5 (2023-11-31)
* liblzma:
- Use __attribute__((__no_sanitize_address__)) to avoid address
sanitization with CRC64 CLMUL. It uses 16-byte-aligned reads
which can extend past the bounds of the input buffer and
inherently trigger address sanitization errors. This isn't
a bug.
- Fixed an assertion failure that could be triggered by a large
unpadded_size argument. It was verified that there was no
other bug than the assertion failure.
- Fixed a bug that prevented building with Windows Vista
threading when __attribute__((__constructor__)) is not
supported.
* xz now properly handles special files such as "con" or "nul" on
Windows. Before this fix, the following wrote "foo" to the
console and deleted the input file "con_xz":
echo foo | xz > con_xz
xz --suffix=_xz --decompress con_xz
* Build systems:
- Allow builds with Windows win95 threading and small mode when
__attribute__((__constructor__)) is supported.
- Added a new line to liblzma.pc for MSYS2 (Windows):
Cflags.private: -DLZMA_API_STATIC
When compiling code that will link against static liblzma,
the LZMA_API_STATIC macro needs to be defined on Windows.
- CMake specific changes:
* Fixed a bug that allowed CLOCK_MONOTONIC to be used even
if the check for it failed.
* Fixed a bug where configuring CMake multiple times
resulted in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC
not being set.
* Fixed the build with MinGW-w64-based Clang/LLVM 17.
llvm-windres now has more accurate GNU windres emulation
so the GNU windres workaround from 5.4.1 is needed with
llvm-windres version 17 too.
* The import library on Windows is now properly named
"liblzma.dll.a" instead of "libliblzma.dll.a"
* Fixed a bug causing the Ninja Generator to fail on
UNIX-like systems. This bug was introduced in 5.4.0.
* Added a new option to disable CLMUL CRC64.
* A module-definition (.def) file is now created when
building liblzma.dll with MinGW-w64.
* The pkg-config liblzma.pc file is now installed on all
builds except when using MSVC on Windows.
* Added large file support by default for platforms that
need it to handle files larger than 2 GiB. This includes
MinGW-w64, even 64-bit builds.
* Small fixes and improvements to the tests.
* Updated translations: Chinese (simplified) and Esperanto.
5.4.4 (2023-08-02)
* liblzma and xzdec can now build against WASI SDK when threading
support is disabled. xz and tests don't build yet.
* CMake:
- Fixed a bug preventing other projects from including liblzma
multiple times using find_package().
- Don't create broken symlinks in Cygwin and MSYS2 unless
supported by the environment. This prevented building for the
default MSYS2 environment. The problem was introduced in
xz 5.4.0.
* Documentation:
- Small improvements to man pages.
- Small improvements and typo fixes for liblzma API
documentation.
* Tests:
- Added a new section to INSTALL to describe basic test usage
and address recent questions about building the tests when
cross compiling.
- Small fixes and improvements to the tests.
* Translations:
- Fixed a mistake that caused one of the error messages to not
be translated. This only affected versions 5.4.2 and 5.4.3.
- Updated the Chinese (simplified), Croatian, Esperanto, German,
Korean, Polish, Romanian, Spanish, Swedish, Ukrainian, and
Vietnamese translations.
- Updated the German, Korean, Romanian, and Ukrainian man page
translations.
5.4.3 (2023-05-04)
* All fixes from 5.2.12
* Features in the CMake build can now be disabled as CMake cache
variables, similar to the Autotools build.
* Minor update to the Croatian translation.
5.4.2 (2023-03-18)
* All fixes from 5.2.11 that were not included in 5.4.1.
* If xz is built with support for the Capsicum sandbox but running
in an environment that doesn't support Capsicum, xz now runs
normally without sandboxing instead of exiting with an error.
* liblzma:
- Documentation was updated to improve the style, consistency,
and completeness of the liblzma API headers.
- The Doxygen-generated HTML documentation for the liblzma API
header files is now included in the source release and is
installed as part of "make install". All JavaScript is
removed to simplify license compliance and to reduce the
install size.
- Fixed a minor bug in lzma_str_from_filters() that produced
too many filters in the output string instead of reporting
an error if the input array had more than four filters. This
bug did not affect xz.
* Build systems:
- autogen.sh now invokes the doxygen tool via the new wrapper
script doxygen/update-doxygen, unless the command line option
--no-doxygen is used.
- Added microlzma_encoder.c and microlzma_decoder.c to the
VS project files for Windows and to the CMake build. These
should have been included in 5.3.2alpha.
* Tests:
- Added a test to the CMake build that was forgotten in the
previous release.
- Added and refactored a few tests.
* Translations:
- Updated the Brazilian Portuguese translation.
- Added Brazilian Portuguese man page translation.
5.4.1 (2023-01-11)
* liblzma:
- Fixed the return value of lzma_microlzma_encoder() if the
LZMA options lc/lp/pb are invalid. Invalid lc/lp/pb options
made the function return LZMA_STREAM_END without encoding
anything instead of returning LZMA_OPTIONS_ERROR.
- Windows / Visual Studio: Workaround a possible compiler bug
when targeting 32-bit x86 and compiling the CLMUL version of
the CRC64 code. The CLMUL code isn't enabled by the Windows
project files but it is in the CMake-based builds.
* Build systems:
- Windows-specific CMake changes:
* Don't try to enable CLMUL CRC64 code if _mm_set_epi64x()
isn't available. This fixes CMake-based build with Visual
Studio 2013.
* Created a workaround for a build failure with windres
from GNU binutils. It is used only when the C compiler
is GCC (not Clang). The workaround is incompatible
with llvm-windres, resulting in "XZx20Utils" instead
of "XZ Utils" in the resource file, but without the
workaround llvm-windres works correctly. See the
comment in CMakeLists.txt for details.
* Included the resource files in the xz and xzdec build
rules. Building the command line tools is still
experimental but possible with MinGW-w64.
- Visual Studio: Added stream_decoder_mt.c to the project
files. Now the threaded decompressor lzma_stream_decoder_mt()
gets built. CMake-based build wasn't affected.
- Updated windows/INSTALL-MSVC.txt to mention that CMake-based
build is now the preferred method with Visual Studio. The
project files will probably be removed after 5.4.x releases.
- Changes to #defines in config.h:
* HAVE_DECL_CLOCK_MONOTONIC was replaced by
HAVE_CLOCK_MONOTONIC. The old macro was always defined
in configure-generated config.h to either 0 or 1. The
new macro is defined (to 1) only if the declaration of
CLOCK_MONOTONIC is available. This matches the way most
other config.h macros work and makes things simpler with
other build systems.
* HAVE_DECL_PROGRAM_INVOCATION_NAME was replaced by
HAVE_PROGRAM_INVOCATION_NAME for the same reason.
* Tests:
- Fixed test script compatibility with ancient /bin/sh
versions. Now the five test_compress_* tests should
no longer fail on Solaris 10.
- Added and refactored a few tests.
* Translations:
- Updated the Catalan and Esperanto translations.
- Added Korean and Ukrainian man page translations.
5.4.0 (2022-12-13)
This bumps the minor version of liblzma because new features were
@ -559,6 +1139,47 @@ XZ Utils Release Notes
(FreeBSD >= 10).
5.2.12 (2023-05-04)
* Fixed a build system bug that prevented building liblzma as a
shared library when configured with --disable-threads. This bug
affected releases 5.2.6 to 5.2.11 and 5.4.0 to 5.4.2.
* Include <intrin.h> for Windows intrinsic functions where they are
needed. This fixed a bug that prevented building liblzma using
clang-cl on Windows.
* Minor update to the Croatian translation. The small change
applies to a string in both 5.2 and 5.4 branches.
5.2.11 (2023-03-18)
* Removed all possible cases of null pointer + 0. It is undefined
behavior in C99 and C17. This was detected by a sanitizer and had
not caused any known issues.
* Build systems:
- Added a workaround for building with GCC on MicroBlaze Linux.
GCC 12 on MicroBlaze doesn't support the __symver__ attribute
even though __has_attribute(__symver__) returns true. The
build is now done without the extra RHEL/CentOS 7 symbols
that were added in XZ Utils 5.2.7. The workaround only
applies to the Autotools build (not CMake).
- CMake: Ensure that the C compiler language is set to C99 or
a newer standard.
- CMake changes from XZ Utils 5.4.1:
* Added a workaround for a build failure with
windres from GNU binutils.
* Included the Windows resource files in the xz
and xzdec build rules.
5.2.10 (2022-12-13)
* xz: Don't modify argv[] when parsing the --memlimit* and
@ -830,7 +1451,7 @@ XZ Utils Release Notes
of the deprecated egrep and fgrep commands.
- Fixed parsing of the options -E, -F, -G, -P, and -X. The
problem occurred when multiple options were specied in
problem occurred when multiple options were specified in
a single argument, for example,
echo foo | xzgrep -Fe foo

View File

@ -44,6 +44,8 @@ Information to packagers of XZ Utils
lzmadec binary for compatibility with LZMA Utils
liblzma liblzma.so.*
liblzma-devel liblzma.so, liblzma.a, API headers
liblzma-doc Doxygen-generated liblzma API docs (HTML),
example programs
2. Package description
@ -109,20 +111,28 @@ Information to packagers of XZ Utils
This package includes the API headers, static library, and
other development files related to liblzma.
liblzma-doc:
liblzma API documentation in HTML and example usage
This package includes the Doxygen-generated liblzma API
HTML docs and example programs showing how to use liblzma.
3. License
----------
If the package manager supports a license field, you probably should
put GPLv2+ there (GNU GPL v2 or later). The interesting parts of
XZ Utils are in the public domain, but some less important files
ending up into the binary package are under GPLv2+. So it is simplest
to just say GPLv2+ if you cannot specify "public domain and GPLv2+".
XZ Utils are under the BSD Zero Clause License (0BSD), but some less
important files ending up into the binary package are under GPLv2+.
So it is simplest to just say GPLv2+ if you cannot specify
"BSD0 and GPLv2+".
If you split XZ Utils into multiple packages as described earlier
in this file, liblzma and liblzma-dev packages will contain only
public domain code (from XZ Utils at least; compiler or linker may
add some third-party code, which may be copyrighted).
0BSD-licensed code from XZ Utils (compiler or linker may add some
third-party code which may have other licenses).
4. configure options
@ -160,12 +170,13 @@ Information to packagers of XZ Utils
can be replaced with a symlink if your distro ships with shared
copies of the common license texts.
liblzma API is currently only documented using Doxygen tags in the
API headers. It hasn't been tested much how good results Doxygen
is able to make from the tags (e.g. Doxyfile might need tweaking,
the tagging may need to be improved etc.), so it might be simpler
to just let people read docs directly from the .h files for now,
and also save quite a bit in package size at the same time.
The Doxygen-generated documentation (HTML) for the liblzma API
headers is included in the source release and will be installed by
"make install" to $docdir/api. All JavaScript is removed to
simplify license compliance and to reduce the install size. If the
liblzma API documentation is not desired, either run configure with
--disable-doc or remove the doc/api directory before running
"make install".
6. Extra files

47
README
View File

@ -67,24 +67,27 @@ XZ Utils
1.1. Overall documentation
README This file
README This file
INSTALL.generic Generic install instructions for those not familiar
with packages using GNU Autotools
INSTALL Installation instructions specific to XZ Utils
PACKAGERS Information to packagers of XZ Utils
INSTALL.generic Generic install instructions for those not
familiar with packages using GNU Autotools
INSTALL Installation instructions specific to XZ Utils
PACKAGERS Information to packagers of XZ Utils
COPYING XZ Utils copyright and license information
COPYING.GPLv2 GNU General Public License version 2
COPYING.GPLv3 GNU General Public License version 3
COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1
COPYING XZ Utils copyright and license information
COPYING.0BSD BSD Zero Clause License
COPYING.GPLv2 GNU General Public License version 2
COPYING.GPLv3 GNU General Public License version 3
COPYING.LGPLv2.1 GNU Lesser General Public License version 2.1
COPYING.CC-BY-SA-4.0 Creative Commons Attribution-ShareAlike 4.0
International Public License
AUTHORS The main authors of XZ Utils
THANKS Incomplete list of people who have helped making
this software
NEWS User-visible changes between XZ Utils releases
ChangeLog Detailed list of changes (commit log)
TODO Known bugs and some sort of to-do list
AUTHORS The main authors of XZ Utils
THANKS Incomplete list of people who have helped making
this software
NEWS User-visible changes between XZ Utils releases
ChangeLog Detailed list of changes (commit log)
TODO Known bugs and some sort of to-do list
Note that only some of the above files are included in binary
packages.
@ -113,8 +116,8 @@ XZ Utils
Since the liblzma API has similarities to the zlib API, some people
may find it useful to read the zlib docs and tutorial too:
http://zlib.net/manual.html
http://zlib.net/zlib_how.html
https://zlib.net/manual.html
https://zlib.net/zlib_how.html
2. Version numbering
@ -281,13 +284,17 @@ XZ Utils
7-Zip and the p7zip port of 7-Zip support the .xz format starting
from the version 9.00alpha.
http://7-zip.org/
http://p7zip.sourceforge.net/
https://7-zip.org/
https://p7zip.sourceforge.net/
XZ Embedded is a limited implementation written for use in the Linux
kernel, but it is also suitable for other embedded use.
https://tukaani.org/xz/embedded.html
https://xz.tukaani.org/xz-embedded/
XZ for Java is a complete implementation written in pure Java.
https://xz.tukaani.org/xz-for-java/
6. Contact information

25
THANKS
View File

@ -5,6 +5,7 @@ Thanks
Some people have helped more, some less, but nevertheless everyone's help
has been important. :-) In alphabetical order:
- Mark Adler
- Kian-Meng Ang
- H. Peter Anvin
- Jeff Bastian
- Nelson H. F. Beebe
@ -19,10 +20,13 @@ has been important. :-) In alphabetical order:
- Jakub Bogusz
- Adam Borowski
- Maarten Bosmans
- Lukas Braune
- Benjamin Buch
- Trent W. Buck
- Kevin R. Bulgrien
- James Buren
- David Burklund
- Frank Busse
- Daniel Mealha Cabrita
- Milo Casagrande
- Marek Černocký
@ -30,6 +34,7 @@ has been important. :-) In alphabetical order:
- Vitaly Chikunov
- Antoine Cœur
- Gabi Davar
- İhsan Doğan
- Chris Donawa
- Andrew Dudman
- Markus Duft
@ -38,6 +43,7 @@ has been important. :-) In alphabetical order:
- Robert Elz
- Gilles Espinasse
- Denis Excoffier
- Vincent Fazio
- Michael Felt
- Michael Fox
- Mike Frysinger
@ -46,9 +52,11 @@ has been important. :-) In alphabetical order:
- Bjarni Ingi Gislason
- John Paul Adrian Glaubitz
- Bill Glessner
- Matthew Good
- Michał Górny
- Jason Gorski
- Juan Manuel Guerrero
- Gabriela Gutierrez
- Diederik de Haas
- Joachim Henke
- Christian Hesse
@ -56,19 +64,27 @@ has been important. :-) In alphabetical order:
- Peter Ivanov
- Nicholas Jackson
- Sam James
- Hajin Jang
- Hans Jansen
- Jouk Jansen
- Jun I Jin
- Kiyoshi Kanazawa
- Joona Kannisto
- Per Øyvind Karlsen
- Iouri Kharon
- Thomas Klausner
- Richard Koch
- Anton Kochkov
- Ville Koskinen
- Sergey Kosukhin
- Marcin Kowalczyk
- Jan Kratochvil
- Christian Kujau
- Stephan Kulow
- Ilya Kurdyukov
- Peter Lawler
- James M Leddy
- Kelvin Lee
- Vincent Lefevre
- Hin-Tak Leung
- Andraž 'ruskie' Levstik
@ -79,6 +95,7 @@ has been important. :-) In alphabetical order:
- Lorenzo De Liso
- H.J. Lu
- Bela Lubkin
- Chenxi Mao
- Gregory Margo
- Julien Marrec
- Ed Maste
@ -96,6 +113,7 @@ has been important. :-) In alphabetical order:
- Jonathan Nieder
- Andre Noll
- Peter O'Gorman
- Dimitri Papadopoulos Orfanos
- Daniel Packard
- Filip Palian
- Peter Pallinger
@ -118,6 +136,7 @@ has been important. :-) In alphabetical order:
- Torsten Rupp
- Stephen Sachs
- Jukka Salmi
- Agostino Sarubbo
- Alexandre Sauvé
- Benno Schulenberg
- Andreas Schwab
@ -129,13 +148,16 @@ has been important. :-) In alphabetical order:
- Brad Smith
- Bruce Stark
- Pippijn van Steenhoven
- Martin Storsjö
- Jonathan Stott
- Dan Stromberg
- Jia Tan
- Vincent Torri
- Alexey Tourbin
- Paul Townsend
- Mohammed Adnène Trojette
- Alexey Tourbin
- Taiki Tsunekawa
- Maksym Vatsyk
- Loganaden Velvindron
- Patrick J. Volkerding
- Martin Väth
@ -150,6 +172,7 @@ has been important. :-) In alphabetical order:
- Charles Wilson
- Lars Wirzenius
- Pilorz Wojciech
- Chien Wong
- Ryan Young
- Andreas Zieringer

4
TODO
View File

@ -24,10 +24,6 @@ Known bugs
tuklib_exit() doesn't block signals => EINTR is possible.
SIGTSTP is not handled. If xz is stopped, the estimated remaining
time and calculated (de)compression speed won't make sense in the
progress indicator (xz --verbose).
If liblzma has created threads and fork() gets called, liblzma
code will break in the child process unless it calls exec() and
doesn't touch liblzma.

View File

@ -1,12 +1,10 @@
#!/bin/sh
# SPDX-License-Identifier: 0BSD
###############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
###############################################################################
set -e -x
@ -19,15 +17,38 @@ ${AUTOCONF:-autoconf}
${AUTOHEADER:-autoheader}
${AUTOMAKE:-automake} -acf --foreign
# Generate the translated man pages if the "po4a" tool is available.
# Generate the translated man pages and the doxygen documentation if the
# "po4a" and "doxygen" tools are available.
# This is *NOT* done by "autoreconf -fi" or when "make" is run.
#
# Pass --no-po4a to this script to skip this step. It can be useful when
# you know that po4a isn't available and don't want autogen.sh to exit
# with non-zero exit status.
if test "x$1" != "x--no-po4a"; then
# Pass --no-po4a or --no-doxygen to this script to skip these steps.
# It can be useful when you know that po4a or doxygen aren't available and
# don't want autogen.sh to exit with non-zero exit status.
generate_po4a="y"
generate_doxygen="y"
for arg in "$@"
do
case $arg in
"--no-po4a")
generate_po4a="n"
;;
"--no-doxygen")
generate_doxygen="n"
;;
esac
done
if test "$generate_po4a" != "n"; then
cd po4a
sh update-po
cd ..
fi
if test "$generate_doxygen" != "n"; then
cd doxygen
sh update-doxygen
cd ..
fi
exit 0

287
build-aux/ci_build.sh Executable file
View File

@ -0,0 +1,287 @@
#!/bin/bash
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# Script meant to be used for Continuous Integration automation for POSIX
# systems. On GitHub, this is used by Ubuntu and MacOS builds.
#
#############################################################################
#
# Author: Jia Tan
#
#############################################################################
set -e
USAGE="Usage: $0
-a [autogen flags]
-b [autotools|cmake]
-c [crc32|crc64|sha256]
-d [encoders|decoders|bcj|delta|threads|shared|nls|small|ifunc|clmul|sandbox]
-f [CFLAGS]
-l [destdir]
-m [compiler]
-n [ARTIFACTS_DIR_NAME]
-p [all|build|test]
-s [srcdir]"
# Absolute path of script directory
ABS_DIR=$(cd -- "$(dirname -- "$0")" && pwd)
# Default CLI option values
AUTOGEN_FLAGS=""
BUILD_SYSTEM="autotools"
CHECK_TYPE="crc32,crc64,sha256"
BCJ="y"
DELTA="y"
ENCODERS="y"
DECODERS="y"
THREADS="y"
SHARED="y"
NATIVE_LANG_SUPPORT="y"
SMALL="n"
IFUNC="y"
CLMUL="y"
SANDBOX="y"
SRC_DIR="$ABS_DIR/../"
DEST_DIR="$SRC_DIR/../xz_build"
PHASE="all"
ARTIFACTS_DIR_NAME="output"
###################
# Parse arguments #
###################
while getopts a:b:c:d:l:m:n:s:p:f:h opt; do
# b option can have either value "autotools" OR "cmake"
case ${opt} in
h)
echo "$USAGE"
exit 0
;;
a)
AUTOGEN_FLAGS="$OPTARG"
;;
b)
case "$OPTARG" in
autotools) ;;
cmake) ;;
*) echo "Invalid build system: $OPTARG"; exit 1;;
esac
BUILD_SYSTEM="$OPTARG"
;;
c) CHECK_TYPE="$OPTARG"
;;
# d options can be a comma separated list of things to disable at
# configure time
d)
for disable_arg in $(echo "$OPTARG" | sed "s/,/ /g"); do
case "$disable_arg" in
encoders) ENCODERS="n" ;;
decoders) DECODERS="n" ;;
bcj) BCJ="n" ;;
delta) DELTA="n" ;;
threads) THREADS="n" ;;
shared) SHARED="n";;
nls) NATIVE_LANG_SUPPORT="n";;
small) SMALL="y";;
ifunc) IFUNC="n";;
clmul) CLMUL="n";;
sandbox) SANDBOX="n";;
*) echo "Invalid disable value: $disable_arg"; exit 1 ;;
esac
done
;;
l) DEST_DIR="$OPTARG"
;;
m)
CC="$OPTARG"
export CC
;;
n) ARTIFACTS_DIR_NAME="$OPTARG"
;;
s) SRC_DIR="$OPTARG"
;;
p) PHASE="$OPTARG"
;;
f)
CFLAGS="$OPTARG"
export CFLAGS
;;
esac
done
####################
# Helper Functions #
####################
# These two functions essentially implement the ternary "?" operator.
add_extra_option() {
# First argument is option value ("y" or "n")
# Second argument is option to set if "y"
# Third argument is option to set if "n"
if [ "$1" = "y" ]
then
EXTRA_OPTIONS="$EXTRA_OPTIONS $2"
else
EXTRA_OPTIONS="$EXTRA_OPTIONS $3"
fi
}
add_to_filter_list() {
# First argument is option value ("y" or "n")
# Second argument is option to set if "y"
if [ "$1" = "y" ]
then
FILTER_LIST="$FILTER_LIST$2"
fi
}
###############
# Build Phase #
###############
if [ "$PHASE" = "all" ] || [ "$PHASE" = "build" ]
then
# Checksum options should be specified differently based on the
# build system. It must be calculated here since we won't know
# the build system used until all args have been parsed.
# Autotools - comma separated
# CMake - semi-colon separated
if [ "$BUILD_SYSTEM" = "autotools" ]
then
SEP=","
else
SEP=";"
fi
CHECK_TYPE_TEMP=""
for crc in $(echo "$CHECK_TYPE" | sed "s/,/ /g"); do
case "$crc" in
# Remove "crc32" from cmake build, if specified.
crc32)
if [ "$BUILD_SYSTEM" = "cmake" ]
then
continue
fi
;;
crc64) ;;
sha256) ;;
*) echo "Invalid check type: $crc"; exit 1 ;;
esac
CHECK_TYPE_TEMP="$CHECK_TYPE_TEMP$SEP$crc"
done
# Remove the first character from $CHECK_TYPE_TEMP since it will
# always be the delimiter.
CHECK_TYPE="${CHECK_TYPE_TEMP:1}"
FILTER_LIST="lzma1$SEP"lzma2
# Build based on arguments
mkdir -p "$DEST_DIR"
# Generate configure option values
EXTRA_OPTIONS=""
case $BUILD_SYSTEM in
autotools)
cd "$SRC_DIR"
# Run autogen.sh script if not already run
if [ ! -f configure ]
then
./autogen.sh "$AUTOGEN_FLAGS"
fi
cd "$DEST_DIR"
add_to_filter_list "$BCJ" ",x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv"
add_to_filter_list "$DELTA" ",delta"
add_extra_option "$ENCODERS" "--enable-encoders=$FILTER_LIST" "--disable-encoders"
add_extra_option "$DECODERS" "--enable-decoders=$FILTER_LIST" "--disable-decoders"
add_extra_option "$THREADS" "" "--disable-threads"
add_extra_option "$SHARED" "" "--disable-shared"
add_extra_option "$NATIVE_LANG_SUPPORT" "" "--disable-nls"
add_extra_option "$SMALL" "--enable-small" ""
add_extra_option "$IFUNC" "" "--disable-ifunc"
add_extra_option "$CLMUL" "" "--disable-clmul-crc"
add_extra_option "$SANDBOX" "" "--enable-sandbox=no"
# Run configure script
"$SRC_DIR"/configure --enable-werror --enable-checks="$CHECK_TYPE" $EXTRA_OPTIONS --config-cache
# Build the project
make
;;
cmake)
cd "$DEST_DIR"
add_to_filter_list "$BCJ" ";x86;powerpc;ia64;arm;armthumb;arm64;sparc;riscv"
add_to_filter_list "$DELTA" ";delta"
add_extra_option "$THREADS" "-DENABLE_THREADS=ON" "-DENABLE_THREADS=OFF"
# Disable MicroLZMA if encoders are not configured.
add_extra_option "$ENCODERS" "-DENCODERS=$FILTER_LIST" "-DENCODERS= -DMICROLZMA_ENCODER=OFF"
# Disable MicroLZMA and lzip decoders if decoders are not configured.
add_extra_option "$DECODERS" "-DDECODERS=$FILTER_LIST" "-DDECODERS= -DMICROLZMA_DECODER=OFF -DLZIP_DECODER=OFF"
# CMake disables the shared library by default.
add_extra_option "$SHARED" "-DBUILD_SHARED_LIBS=ON" ""
add_extra_option "$SMALL" "-DHAVE_SMALL=ON" ""
if test -n "$CC" ; then
EXTRA_OPTIONS="$EXTRA_OPTIONS -DCMAKE_C_COMPILER=$CC"
fi
# Remove old cache file to clear previous settings.
rm -f "CMakeCache.txt"
cmake "$SRC_DIR/CMakeLists.txt" -B "$DEST_DIR" $EXTRA_OPTIONS -DADDITIONAL_CHECK_TYPES="$CHECK_TYPE" -G "Unix Makefiles"
cmake --build "$DEST_DIR"
;;
esac
fi
##############
# Test Phase #
##############
if [ "$PHASE" = "all" ] || [ "$PHASE" = "test" ]
then
case $BUILD_SYSTEM in
autotools)
cd "$DEST_DIR"
# If the tests fail, copy the test logs into the artifacts folder
if make check
then
:
else
mkdir -p "$SRC_DIR/build-aux/artifacts/$ARTIFACTS_DIR_NAME"
cp ./tests/*.log "$SRC_DIR/build-aux/artifacts/$ARTIFACTS_DIR_NAME"
exit 1
fi
;;
cmake)
cd "$DEST_DIR"
if make test
then
:
else
mkdir -p "$SRC_DIR/build-aux/artifacts/$ARTIFACTS_DIR_NAME"
cp ./Testing/Temporary/*.log "$SRC_DIR/build-aux/artifacts/$ARTIFACTS_DIR_NAME"
exit 1
fi
;;
esac
fi

View File

@ -1,5 +1,6 @@
#!/bin/sh
#
# SPDX-License-Identifier: 0BSD
###############################################################################
#
# Wrapper for GNU groff to convert man pages to a few formats
@ -17,9 +18,6 @@
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
###############################################################################
FORMAT=$1
@ -38,10 +36,10 @@ s/^\\.PD\$/.PD $PD/"
case $FORMAT in
ascii)
groff -t -mandoc -Tascii | col -bx
groff -t -mandoc -Tascii -P-c | col -bx
;;
utf8)
groff -t -mandoc -Tutf8 | col -bx
groff -t -mandoc -Tutf8 -P-c | col -bx
;;
ps)
sed "$SED_PD" | groff -dpaper=$PAPER -t -mandoc \

View File

@ -1,5 +1,6 @@
#!/bin/sh
#
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# Get the version string from version.h and print it out without
@ -9,9 +10,6 @@
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
sed -n 's/LZMA_VERSION_STABILITY_ALPHA/alpha/

View File

@ -0,0 +1,25 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# remove-ordinals.cmake
#
# Removes the ordinal numbers from a DEF file that has been created by
# GNU ld or LLVM lld option --output-def (when creating a Windows DLL).
# This should be equivalent: sed 's/ \+@ *[0-9]\+//'
#
# Usage:
#
# cmake -DINPUT_FILE=infile.def.in \
# -DOUTPUT_FILE=outfile.def \
# -P remove-ordinals.cmake
#
#############################################################################
#
# Author: Lasse Collin
#
#############################################################################
file(READ "${INPUT_FILE}" STR)
string(REGEX REPLACE " +@ *[0-9]+" "" STR "${STR}")
file(WRITE "${OUTPUT_FILE}" "${STR}")

View File

@ -1,11 +1,12 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_common.cmake - common functions and macros for tuklib_*.cmake files
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
function(tuklib_add_definitions TARGET_OR_ALL DEFINITIONS)
# DEFINITIONS may be an empty string/list but it's fine here. There is

View File

@ -1,11 +1,12 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_cpucores.cmake - see tuklib_cpucores.m4 for description and comments
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(CheckCSourceCompiles)

View File

@ -1,11 +1,12 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_integer.cmake - see tuklib_integer.m4 for description and comments
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(TestBigEndian)
@ -59,29 +60,67 @@ function(tuklib_integer TARGET_OR_ALL)
endif()
endif()
# 16-bit and 32-bit unaligned access is fast on x86(-64),
# big endian PowerPC, and usually on 32/64-bit ARM too.
# There are others too and ARM could be a false match.
# Guess that unaligned access is fast on these archs:
# - 32/64-bit x86 / x86-64
# - 32/64-bit big endian PowerPC
# - 64-bit little endian PowerPC
# - Some 32-bit ARM
# - Some 64-bit ARM64 (AArch64)
# - Some 32/64-bit RISC-V
#
# Guess the default value for the option.
# CMake's ability to give info about the target arch seems bad.
# The the same arch can have different name depending on the OS.
#
# FIXME: The regex is based on guessing, not on factual information!
#
# NOTE: Compared to the Autoconf test, this lacks the GCC/Clang test
# on ARM and always assumes that unaligned is fast on ARM.
# CMake doesn't provide a standardized/normalized list of processor arch
# names. For example, x86-64 may be "x86_64" (Linux), "AMD64" (Windows),
# or even "EM64T" (64-bit WinXP).
set(FAST_UNALIGNED_GUESS OFF)
if(CMAKE_SYSTEM_PROCESSOR MATCHES
"[Xx3456]86|^[Xx]64|^[Aa][Mm][Dd]64|^[Aa][Rr][Mm]|^aarch|^powerpc|^ppc")
if(NOT WORDS_BIGENDIAN OR
NOT CMAKE_SYSTEM_PROCESSOR MATCHES "^powerpc|^ppc")
set(FAST_UNALIGNED_GUESS ON)
string(TOLOWER "${CMAKE_SYSTEM_PROCESSOR}" PROCESSOR)
# There is no ^ in the first regex branch to allow "i" at the beginning
# so it can match "i386" to "i786", and "x86_64".
if(PROCESSOR MATCHES "[x34567]86|^x64|^amd64|^em64t")
set(FAST_UNALIGNED_GUESS ON)
elseif(PROCESSOR MATCHES "^powerpc|^ppc")
if(WORDS_BIGENDIAN OR PROCESSOR MATCHES "64")
set(FAST_UNALIGNED_GUESS ON)
endif()
elseif(PROCESSOR MATCHES "^arm|^aarch64|^riscv")
# On 32-bit and 64-bit ARM, GCC and Clang
# #define __ARM_FEATURE_UNALIGNED if
# unaligned access is supported.
#
# Exception: GCC at least up to 13.2.0
# defines it even when using -mstrict-align
# so in that case this autodetection goes wrong.
# Most of the time -mstrict-align isn't used so it
# shouldn't be a common problem in practice. See:
# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111555
#
# RISC-V C API Specification says that if
# __riscv_misaligned_fast is defined then
# unaligned access is known to be fast.
#
# MSVC is handled as a special case: We assume that
# 32/64-bit ARM supports fast unaligned access.
# If MSVC gets RISC-V support then this will assume
# fast unaligned access on RISC-V too.
check_c_source_compiles("
#if !defined(__ARM_FEATURE_UNALIGNED) \
&& !defined(__riscv_misaligned_fast) \
&& !defined(_MSC_VER)
compile error
#endif
int main(void) { return 0; }
"
TUKLIB_FAST_UNALIGNED_DEFINED_BY_PREPROCESSOR)
if(TUKLIB_FAST_UNALIGNED_DEFINED_BY_PREPROCESSOR)
set(FAST_UNALIGNED_GUESS ON)
endif()
endif()
option(TUKLIB_FAST_UNALIGNED_ACCESS
"Enable if the system supports *fast* unaligned memory access \
with 16-bit and 32-bit integers."
with 16-bit, 32-bit, and 64-bit integers."
"${FAST_UNALIGNED_GUESS}")
tuklib_add_definition_if("${TARGET_OR_ALL}" TUKLIB_FAST_UNALIGNED_ACCESS)

View File

@ -0,0 +1,53 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_large_file_support.cmake
#
# If off_t is less than 64 bits by default and -D_FILE_OFFSET_BITS=64
# makes off_t become 64-bit, the CMake option LARGE_FILE_SUPPORT is
# provided (ON by default) and -D_FILE_OFFSET_BITS=64 is added to
# the compile definitions if LARGE_FILE_SUPPORT is ON.
#
# Author: Lasse Collin
#
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(CheckCSourceCompiles)
function(tuklib_large_file_support TARGET_OR_ALL)
# MSVC must be handled specially in the C code.
if(MSVC)
return()
endif()
set(TUKLIB_LARGE_FILE_SUPPORT_TEST
"#include <sys/types.h>
int foo[sizeof(off_t) >= 8 ? 1 : -1];
int main(void) { return 0; }")
check_c_source_compiles("${TUKLIB_LARGE_FILE_SUPPORT_TEST}"
TUKLIB_LARGE_FILE_SUPPORT_BY_DEFAULT)
if(NOT TUKLIB_LARGE_FILE_SUPPORT_BY_DEFAULT)
cmake_push_check_state()
# This needs -D.
list(APPEND CMAKE_REQUIRED_DEFINITIONS "-D_FILE_OFFSET_BITS=64")
check_c_source_compiles("${TUKLIB_LARGE_FILE_SUPPORT_TEST}"
TUKLIB_LARGE_FILE_SUPPORT_WITH_FOB64)
cmake_pop_check_state()
endif()
if(TUKLIB_LARGE_FILE_SUPPORT_WITH_FOB64)
# Show the option only when _FILE_OFFSET_BITS=64 affects sizeof(off_t).
option(LARGE_FILE_SUPPORT
"Use -D_FILE_OFFSET_BITS=64 to support files larger than 2 GiB."
ON)
if(LARGE_FILE_SUPPORT)
# This must not use -D.
tuklib_add_definitions("${TARGET_OR_ALL}" "_FILE_OFFSET_BITS=64")
endif()
endif()
endfunction()

View File

@ -1,11 +1,12 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_mbstr.cmake - see tuklib_mbstr.m4 for description and comments
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(CheckSymbolExists)

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_physmem.cmake - see tuklib_physmem.m4 for description and comments
#
@ -6,9 +9,7 @@
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(CheckCSourceCompiles)

View File

@ -1,11 +1,12 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# tuklib_progname.cmake - see tuklib_progname.m4 for description and comments
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
include("${CMAKE_CURRENT_LIST_DIR}/tuklib_common.cmake")
include(CheckSymbolExists)
@ -13,7 +14,6 @@ include(CheckSymbolExists)
function(tuklib_progname TARGET_OR_ALL)
# NOTE: This glibc extension requires _GNU_SOURCE.
check_symbol_exists(program_invocation_name errno.h
HAVE_DECL_PROGRAM_INVOCATION_NAME)
tuklib_add_definition_if("${TARGET_OR_ALL}"
HAVE_DECL_PROGRAM_INVOCATION_NAME)
HAVE_PROGRAM_INVOCATION_NAME)
tuklib_add_definition_if("${TARGET_OR_ALL}" HAVE_PROGRAM_INVOCATION_NAME)
endfunction()

View File

@ -1,12 +1,11 @@
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
# SPDX-License-Identifier: 0BSD
###############################################################################
#
# Author: Lasse Collin
# Process this file with autoconf to produce a configure script.
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
###############################################################################
@ -18,7 +17,7 @@
AC_PREREQ([2.69])
AC_INIT([XZ Utils], m4_esyscmd([/bin/sh build-aux/version.sh]),
[xz@tukaani.org], [xz], [https://tukaani.org/xz/])
[xz@tukaani.org], [xz], [https://xz.tukaani.org/xz-utils/])
AC_CONFIG_SRCDIR([src/liblzma/common/common.h])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_MACRO_DIR([m4])
@ -79,8 +78,8 @@ fi
# Filters #
###########
m4_define([SUPPORTED_FILTERS], [lzma1,lzma2,delta,x86,powerpc,ia64,arm,armthumb,arm64,sparc])dnl
m4_define([SIMPLE_FILTERS], [x86,powerpc,ia64,arm,armthumb,arm64,sparc])
m4_define([SUPPORTED_FILTERS], [lzma1,lzma2,delta,x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv])dnl
m4_define([SIMPLE_FILTERS], [x86,powerpc,ia64,arm,armthumb,arm64,sparc,riscv])
m4_define([LZ_FILTERS], [lzma1,lzma2])
m4_foreach([NAME], [SUPPORTED_FILTERS],
@ -310,7 +309,7 @@ case $enable_microlzma in
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([--enable-microlzma accepts only `yes' or `no'.])
AC_MSG_ERROR([--enable-microlzma accepts only 'yes' or 'no'.])
;;
esac
AM_CONDITIONAL(COND_MICROLZMA, test "x$enable_microlzma" = xyes)
@ -364,7 +363,7 @@ case $enable_assembler in
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([--enable-assembler accepts only `yes', `no', or `x86' (32-bit).])
AC_MSG_ERROR([--enable-assembler accepts only 'yes', 'no', or 'x86' (32-bit).])
;;
esac
AM_CONDITIONAL(COND_ASM_X86, test "x$enable_assembler" = xx86)
@ -380,6 +379,16 @@ AC_ARG_ENABLE([clmul-crc], AS_HELP_STRING([--disable-clmul-crc],
[], [enable_clmul_crc=yes])
############################
# ARM64 CRC32 Instructions #
############################
AC_ARG_ENABLE([arm64-crc32], AS_HELP_STRING([--disable-arm64-crc32],
[Do not use ARM64 CRC32 instructions even if support for it
is detected.]),
[], [enable_arm64_crc32=yes])
#####################
# Size optimization #
#####################
@ -393,7 +402,7 @@ if test "x$enable_small" = xyes; then
AC_DEFINE([HAVE_SMALL], [1], [Define to 1 if optimizing for size.])
elif test "x$enable_small" != xno; then
AC_MSG_RESULT([])
AC_MSG_ERROR([--enable-small accepts only `yes' or `no'])
AC_MSG_ERROR([--enable-small accepts only 'yes' or 'no'])
fi
AC_MSG_RESULT([$enable_small])
AM_CONDITIONAL(COND_SMALL, test "x$enable_small" = xyes)
@ -405,8 +414,8 @@ AM_CONDITIONAL(COND_SMALL, test "x$enable_small" = xyes)
AC_MSG_CHECKING([if threading support is wanted])
AC_ARG_ENABLE([threads], AS_HELP_STRING([--enable-threads=METHOD],
[Supported METHODS are `yes', `no', `posix', `win95', and
`vista'. The default is `yes'. Using `no' together with
[Supported METHODS are 'yes', 'no', 'posix', 'win95', and
'vista'. The default is 'yes'. Using 'no' together with
--enable-small makes liblzma thread unsafe.]),
[], [enable_threads=yes])
@ -433,18 +442,10 @@ case $enable_threads in
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([--enable-threads only accepts `yes', `no', `posix', `win95', or `vista'])
AC_MSG_ERROR([--enable-threads only accepts 'yes', 'no', 'posix', 'win95', or 'vista'])
;;
esac
# The Win95 threading lacks thread-safe one-time initialization function.
# It's better to disallow it instead of allowing threaded but thread-unsafe
# build.
if test "x$enable_small$enable_threads" = xyeswin95; then
AC_MSG_ERROR([--enable-threads=win95 and --enable-small cannot be
used at the same time])
fi
# We use the actual result a little later.
@ -527,20 +528,24 @@ AM_CONDITIONAL([COND_DOC], [test x$enable_doc != xno])
AC_MSG_CHECKING([if sandboxing should be used])
AC_ARG_ENABLE([sandbox], [AS_HELP_STRING([--enable-sandbox=METHOD],
[Sandboxing METHOD can be
`auto', `no', `capsicum', or `pledge'.
The default is `auto' which enables sandboxing if
'auto', 'no', 'capsicum', 'pledge', or 'landlock'.
The default is 'auto' which enables sandboxing if
a supported sandboxing method is found.])],
[], [enable_sandbox=auto])
case $enable_sandbox in
auto)
case $enable_xzdec-$enable_xz-$enable_sandbox in
no-no-*)
enable_sandbox=no
AC_MSG_RESULT([no, --disable-xz and --disable-xzdec was used])
;;
*-*-auto)
AC_MSG_RESULT([maybe (autodetect)])
;;
no | capsicum | pledge)
*-*-no | *-*-capsicum | *-*-pledge | *-*-landlock)
AC_MSG_RESULT([$enable_sandbox])
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([--enable-sandbox only accepts `auto', `no', `capsicum', or `pledge'.])
AC_MSG_ERROR([--enable-sandbox only accepts 'auto', 'no', 'capsicum', 'pledge', or 'landlock'.])
;;
esac
@ -596,12 +601,15 @@ echo "Initializing Automake:"
# We don't use "subdir-objects" yet because it breaks "make distclean" when
# dependencies are enabled (as of Automake 1.14.1) due to this bug:
# http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17354
# https://debbugs.gnu.org/cgi/bugreport.cgi?bug=17354
# The -Wno-unsupported is used to silence warnings about missing
# "subdir-objects".
AM_INIT_AUTOMAKE([1.12 foreign tar-v7 filename-length-max=99 -Wno-unsupported])
AC_PROG_LN_S
dnl # Autoconf >= 2.70 warns that AC_PROG_CC_C99 is obsolete. However,
dnl # we have to keep using AC_PROG_CC_C99 instead of AC_PROG_CC
dnl # as long as we try to be compatible with Autoconf 2.69.
AC_PROG_CC_C99
if test x$ac_cv_prog_cc_c99 = xno ; then
AC_MSG_ERROR([No C99 compiler was found.])
@ -611,8 +619,8 @@ AM_PROG_CC_C_O
AM_PROG_AS
AC_USE_SYSTEM_EXTENSIONS
case $enable_threads in
posix)
AS_CASE([$enable_threads],
[posix], [
echo
echo "POSIX threading support:"
AX_PTHREAD([:]) dnl We don't need the HAVE_PTHREAD macro.
@ -627,32 +635,24 @@ case $enable_threads in
AC_DEFINE([MYTHREAD_POSIX], [1],
[Define to 1 when using POSIX threads (pthreads).])
# These are nice to have but not mandatory.
#
# FIXME: xz uses clock_gettime if it is available and can do
# it even when threading is disabled. Moving this outside
# of pthread detection may be undesirable because then
# liblzma may get linked against librt even when librt isn't
# needed by liblzma.
# This is nice to have but not mandatory.
OLD_CFLAGS=$CFLAGS
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
AC_SEARCH_LIBS([clock_gettime], [rt])
AC_CHECK_FUNCS([clock_gettime pthread_condattr_setclock])
AC_CHECK_DECLS([CLOCK_MONOTONIC], [], [], [[#include <time.h>]])
AC_CHECK_FUNCS([pthread_condattr_setclock])
CFLAGS=$OLD_CFLAGS
;;
win95)
],
[win95], [
AC_DEFINE([MYTHREAD_WIN95], [1], [Define to 1 when using
Windows 95 (and thus XP) compatible threads.
This avoids use of features that were added in
Windows Vista.])
;;
vista)
],
[vista], [
AC_DEFINE([MYTHREAD_VISTA], [1], [Define to 1 when using
Windows Vista compatible threads. This uses
features that are not available on Windows XP.])
;;
esac
]
)
AM_CONDITIONAL([COND_THREADS], [test "x$enable_threads" != xno])
echo
@ -705,57 +705,84 @@ fi
# used when creating a static library.
#
# Libtool always uses -DPIC when building shared libraries by default and
# doesn't use it for static libs by default. This can be overriden with
# doesn't use it for static libs by default. This can be overridden with
# --with-pic and --without-pic though. As long as neither --with-pic nor
# --without-pic is used then we can use #ifdef PIC to detect if the file is
# being built for a shared library.
if test "x$enable_symbol_versions" = xno ; then
AS_IF([test "x$enable_symbol_versions" = xno], [
enable_symbol_versions=no
AC_MSG_RESULT([no])
elif test "x$enable_shared" = xno ; then
], [test "x$enable_shared" = xno], [
enable_symbol_versions=no
AC_MSG_RESULT([no (not building a shared library)])
else
case $host_os in
linux*)
case "$pic_mode-$enable_static" in
default-*)
# Use symvers if PIC is defined.
have_symbol_versions_linux=2
;;
*-no)
# Not building static library.
# Use symvers unconditionally.
have_symbol_versions_linux=1
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([
], [
# "yes" means that symbol version are to be used but we need to
# autodetect which variant to use.
if test "x$enable_symbol_versions" = xyes ; then
case "$host_cpu-$host_os" in
microblaze*)
# GCC 12 on MicroBlaze doesn't support
# __symver__ attribute. It's simplest and
# safest to use the generic version on that
# platform since then only the linker script
# is needed. The RHEL/CentOS 7 compatibility
# symbols don't matter on MicroBlaze.
enable_symbol_versions=generic
;;
*-linux*)
# NVIDIA HPC Compiler doesn't support symbol
# versioning but the linker script can still
# be used.
AC_EGREP_CPP([use_generic_symbol_versioning],
[#ifdef __NVCOMPILER
use_generic_symbol_versioning
#endif],
[enable_symbol_versions=generic],
[enable_symbol_versions=linux])
;;
*)
enable_symbol_versions=generic
;;
esac
fi
if test "x$enable_symbol_versions" = xlinux ; then
case "$pic_mode-$enable_static" in
default-*)
# Use symvers if PIC is defined.
have_symbol_versions_linux=2
;;
*-no)
# Not building static library.
# Use symvers unconditionally.
have_symbol_versions_linux=1
;;
*)
AC_MSG_RESULT([])
AC_MSG_ERROR([
On GNU/Linux, building both shared and static library at the same time
is not supported if --with-pic or --without-pic is used.
Use either --disable-shared or --disable-static to build one type
of library at a time. If both types are needed, build one at a time,
possibly picking only src/liblzma/.libs/liblzma.a from the static build.])
;;
esac
enable_symbol_versions=linux
AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX],
[$have_symbol_versions_linux],
[Define to 1 to if GNU/Linux-specific details
are unconditionally wanted for symbol
versioning. Define to 2 to if these are wanted
only if also PIC is defined (allows building
both shared and static liblzma at the same
time with Libtool if neither --with-pic nor
--without-pic is used). This define must be
used together with liblzma_linux.map.])
;;
*)
enable_symbol_versions=generic
;;
esac
;;
esac
AC_DEFINE_UNQUOTED([HAVE_SYMBOL_VERSIONS_LINUX],
[$have_symbol_versions_linux],
[Define to 1 to if GNU/Linux-specific details
are unconditionally wanted for symbol
versioning. Define to 2 to if these are wanted
only if also PIC is defined (allows building
both shared and static liblzma at the same
time with Libtool if neither --with-pic nor
--without-pic is used). This define must be
used together with liblzma_linux.map.])
elif test "x$enable_symbol_versions" != xgeneric ; then
AC_MSG_RESULT([])
AC_MSG_ERROR([unknown symbol versioning variant '$enable_symbol_versions'])
fi
AC_MSG_RESULT([yes ($enable_symbol_versions)])
fi
])
AM_CONDITIONAL([COND_SYMVERS_LINUX],
[test "x$enable_symbol_versions" = xlinux])
@ -826,17 +853,22 @@ AC_C_BIGENDIAN
# __attribute__((__constructor__)) can be used for one-time initializations.
# Use -Werror because some compilers accept unknown attributes and just
# give a warning. If it works this should give no warnings, even
# clang -Weverything should be fine.
# dnl This doesn't need AC_LANG_SOURCE, minimal code is enough.
# give a warning.
#
# FIXME? Unfortunately -Werror can cause trouble if CFLAGS contains options
# that produce warnings for unrelated reasons. For example, GCC and Clang
# support -Wunused-macros which will warn about "#define _GNU_SOURCE 1"
# which will be among the #defines that Autoconf inserts to the beginning of
# the test program. There seems to be no nice way to prevent Autoconf from
# inserting the any defines to the test program.
AC_MSG_CHECKING([if __attribute__((__constructor__)) can be used])
have_func_attribute_constructor=no
OLD_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -Werror"
AC_COMPILE_IFELSE([
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
__attribute__((__constructor__))
static void my_constructor_func(void) { return; }
], [
]])], [
AC_DEFINE([HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR], [1],
[Define to 1 if __attribute__((__constructor__))
is supported for functions.])
@ -847,6 +879,99 @@ AC_COMPILE_IFELSE([
])
CFLAGS="$OLD_CFLAGS"
# The Win95 threading lacks a thread-safe one-time initialization function.
# The one-time initialization is needed for crc32_small.c and crc64_small.c
# create the CRC tables. So if small mode is enabled, the threading mode is
# win95, and the compiler does not support attribute constructor, then we
# would end up with a multithreaded build that is thread-unsafe. As a
# result this configuration is not allowed.
if test "x$enable_small$enable_threads$have_func_attribute_constructor" \
= xyeswin95no; then
AC_MSG_ERROR([
--enable-threads=win95 and --enable-small cannot be used
at the same time with a compiler that doesn't support
__attribute__((__constructor__))])
fi
# __attribute__((__ifunc__())) can be used to choose between different
# implementations of the same function at runtime. This is slightly more
# efficient than using __attribute__((__constructor__)) and setting
# a function pointer.
AC_ARG_ENABLE([ifunc], [AS_HELP_STRING([--enable-ifunc],
[Use __attribute__((__ifunc__())). Enabled by default on
GNU/Linux (glibc) and FreeBSD.])],
[], [enable_ifunc=auto])
# When enable_ifunc is 'auto', allow the use of __attribute__((__ifunc__()))
# if compiler support is detected and we are building for GNU/Linux (glibc)
# or FreeBSD. uClibc and musl don't support ifunc in their dynamic linkers
# but some compilers still accept the attribute when compiling for these
# C libraries, which results in broken binaries. That's why we need to
# check which libc is being used.
if test "x$enable_ifunc" = xauto ; then
OLD_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -Werror"
AC_MSG_CHECKING([if __attribute__((__ifunc__())) can be used])
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
/*
* Force a compilation error when not using glibc on Linux
* or if we are not using FreeBSD. uClibc will define
* __GLIBC__ but does not support ifunc, so we must have
* an extra check to disable with uClibc.
*/
#if defined(__linux__)
# include <features.h>
# if !defined(__GLIBC__) || defined(__UCLIBC__)
compile error
# endif
#elif !defined(__FreeBSD__)
compile error
#endif
static void func(void) { return; }
/*
* The attribute __no_profile_instrument_function__ is
* needed with GCC to prevent improper instrumentation in
* the ifunc resolver.
*/
__attribute__((__no_profile_instrument_function__))
static void (*resolve_func (void)) (void) { return func; }
void func_ifunc (void)
__attribute__((__ifunc__("resolve_func")));
/*
* 'clang -Wall' incorrectly warns that resolve_func is
* unused (-Wunused-function). Correct assembly output is
* still produced. This problem exists at least in Clang
* versions 4 to 17. The following silences the bogus warning:
*/
void make_clang_quiet(void);
void make_clang_quiet(void) { resolve_func()(); }
]])], [
enable_ifunc=yes
], [
enable_ifunc=no
])
AC_MSG_RESULT([$enable_ifunc])
CFLAGS="$OLD_CFLAGS"
fi
if test "x$enable_ifunc" = xyes ; then
AC_DEFINE([HAVE_FUNC_ATTRIBUTE_IFUNC], [1],
[Define to 1 if __attribute__((__ifunc__()))
is supported for functions.])
# ifunc explicitly does not work with -fsanitize=address.
# If configured, it will result in a liblzma build that will fail
# when liblzma is loaded at runtime (when the ifunc resolver
# executes).
AS_CASE([$CFLAGS], [*-fsanitize=*], [AC_MSG_ERROR([
CFLAGS contains '-fsanitize=' which is incompatible with ifunc.
Use --disable-ifunc when using '-fsanitize'.])])
fi
###############################################################################
# Checks for library functions.
@ -855,6 +980,22 @@ CFLAGS="$OLD_CFLAGS"
# Gnulib replacements as needed
gl_GETOPT
# If clock_gettime() is available, liblzma with pthreads may use it, and
# xz may use it even when threading support is disabled. In XZ Utils 5.4.x
# and older, configure checked for clock_gettime() only when using pthreads.
# This way non-threaded builds of liblzma didn't get a useless dependency on
# librt which further had a dependency on libpthread. Avoiding these was
# useful when a small build was needed, for example, for initramfs use.
#
# The above reasoning is thoroughly obsolete: On GNU/Linux, librt hasn't
# been needed for clock_gettime() since glibc 2.17 (2012-12-25).
# Solaris 10 needs librt but Solaris 11 doesn't anymore.
AC_SEARCH_LIBS([clock_gettime], [rt])
AC_CHECK_FUNCS([clock_gettime])
AC_CHECK_DECL([CLOCK_MONOTONIC], [AC_DEFINE([HAVE_CLOCK_MONOTONIC], [1],
[Define to 1 if 'CLOCK_MONOTONIC' is declared in <time.h>.])], [],
[[#include <time.h>]])
# Find the best function to set timestamps.
AC_CHECK_FUNCS([futimens futimes futimesat utimes _futime utime], [break])
@ -884,7 +1025,7 @@ TUKLIB_MBSTR
sha256_header_found=no
sha256_type_found=no
sha256_func_found=no
if test "x$enable_external_sha256" = "xyes"; then
AS_IF([test "x$enable_external_sha256" = "xyes"], [
# Test for Common Crypto before others, because Darwin has sha256.h
# too and we don't want to use that, because on older versions it
# uses OpenSSL functions, whose SHA256_Init is not guaranteed to
@ -914,7 +1055,7 @@ if test "x$enable_external_sha256" = "xyes"; then
[sha256_func_found=yes ; break])
fi
fi
fi
])
AM_CONDITIONAL([COND_INTERNAL_SHA256], [test "x$sha256_func_found" = xno])
if test "x$enable_external_sha256$sha256_func_found" = xyesno; then
AC_MSG_ERROR([--enable-external-sha256 was specified but no supported external SHA-256 implementation was found])
@ -935,6 +1076,8 @@ AC_CHECK_DECL([_mm_movemask_epi8],
# For faster CRC on 32/64-bit x86 and E2K (see also crc64_fast.c):
#
# - Check for the CLMUL intrinsic _mm_clmulepi64_si128 in <immintrin.h>.
# Check also for _mm_set_epi64x for consistency with CMake build
# where it's needed to disable CLMUL with VS2013.
#
# - Check that __attribute__((__target__("ssse3,sse4.1,pclmul"))) works
# together with _mm_clmulepi64_si128 from <immintrin.h>. The attribute
@ -945,9 +1088,9 @@ AC_CHECK_DECL([_mm_movemask_epi8],
# If everything above is supported, runtime detection will be used to keep the
# binaries working on systems that don't support the required extensions.
AC_MSG_CHECKING([if _mm_clmulepi64_si128 is usable])
if test "x$enable_clmul_crc" = xno ; then
AS_IF([test "x$enable_clmul_crc" = xno], [
AC_MSG_RESULT([no, --disable-clmul-crc was used])
else
], [
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
#include <immintrin.h>
@ -962,31 +1105,133 @@ else
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
__attribute__((__target__("ssse3,sse4.1,pclmul")))
#endif
__m128i my_clmul(__m128i a, __m128i b)
__m128i my_clmul(__m128i a)
{
const __m128i b = _mm_set_epi64x(1, 2);
return _mm_clmulepi64_si128(a, b, 0);
}
]])], [
AC_DEFINE([HAVE_USABLE_CLMUL], [1],
[Define to 1 if _mm_clmulepi64_si128 is usable.
[Define to 1 if _mm_set_epi64x and
_mm_clmulepi64_si128 are usable.
See configure.ac for details.])
AC_MSG_RESULT([yes])
enable_clmul_crc=yes
], [
AC_MSG_RESULT([no])
enable_clmul_crc=no
])
fi
AC_MSG_RESULT([$enable_clmul_crc])
])
# ARM64 C Language Extensions define CRC32 functions in arm_acle.h.
# These are supported by at least GCC and Clang which both need
# __attribute__((__target__("+crc"))), unless the needed compiler flags
# are used to support the CRC instruction.
AC_MSG_CHECKING([if ARM64 CRC32 instruction is usable])
AS_IF([test "x$enable_arm64_crc32" = xno], [
AC_MSG_RESULT([no, --disable-arm64-crc32 was used])
], [
# Set -Werror here because some versions of Clang (14 and older)
# do not report the unsupported __attribute__((__target__("+crc")))
# or __crc32d() as an error, only as a warning. This does not need
# to be done with CMake because tests will attempt to link and the
# error will be reported then.
OLD_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS -Werror"
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
#include <arm_acle.h>
#include <stdint.h>
#if (defined(__GNUC__) || defined(__clang__)) && !defined(__EDG__)
__attribute__((__target__("+crc")))
#endif
uint32_t my_crc(uint32_t a, uint64_t b)
{
return __crc32d(a, b);
}
]])], [
AC_DEFINE([HAVE_ARM64_CRC32], [1],
[Define to 1 if ARM64 CRC32 instruction is supported.
See configure.ac for details.])
enable_arm64_crc32=yes
], [
enable_arm64_crc32=no
])
AC_MSG_RESULT([$enable_arm64_crc32])
CFLAGS="$OLD_CFLAGS"
])
# Check for ARM64 CRC32 instruction runtime detection.
# getauxval() is supported on Linux, elf_aux_info() on FreeBSD, and
# sysctlbyname("hw.optional.armv8_crc32", ...) is supported on Darwin
# (macOS, iOS, etc.). Note that sysctlbyname() is supported on FreeBSD,
# NetBSD, and possibly others too but the string is specific to Apple OSes.
# The C code is responsible for checking defined(__APPLE__) before using
# sysctlbyname("hw.optional.armv8_crc32", ...).
AS_IF([test "x$enable_arm64_crc32" = xyes], [
AC_CHECK_FUNCS([getauxval elf_aux_info sysctlbyname])
])
# Check for sandbox support. If one is found, set enable_sandbox=found.
case $enable_sandbox in
auto | capsicum)
AX_CHECK_CAPSICUM([enable_sandbox=found], [:])
;;
esac
case $enable_sandbox in
auto | pledge)
AC_CHECK_FUNCS([pledge], [enable_sandbox=found ; break])
;;
esac
#
# About -fsanitize: Of our three sandbox methods, only Landlock is
# incompatible with -fsanitize. FreeBSD 13.2 with Capsicum was tested with
# -fsanitize=address,undefined and had no issues. OpenBSD (as of version
# 7.4) has minimal support for process instrumentation. OpenBSD does not
# distribute the additional libraries needed (libasan, libubsan, etc.) with
# GCC or Clang needed for runtime sanitization support and instead only
# support -fsanitize-minimal-runtime for minimal undefined behavior
# sanitization. This minimal support is compatible with our use of the
# Pledge sandbox. So only Landlock will result in a build that cannot
# compress or decompress a single file to standard out.
AS_CASE([$enable_sandbox],
[auto | capsicum], [
AC_CHECK_FUNCS([cap_rights_limit], [enable_sandbox=found])
]
)
AS_CASE([$enable_sandbox],
[auto | pledge], [
AC_CHECK_FUNCS([pledge], [enable_sandbox=found])
]
)
AS_CASE([$enable_sandbox],
[auto | landlock], [
AC_MSG_CHECKING([if Linux Landlock is usable])
# A compile check is done here because some systems have
# linux/landlock.h, but do not have the syscalls defined
# in order to actually use Linux Landlock.
AC_COMPILE_IFELSE([AC_LANG_SOURCE([[
#include <linux/landlock.h>
#include <sys/syscall.h>
#include <sys/prctl.h>
void my_sandbox(void)
{
(void)prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0);
(void)SYS_landlock_create_ruleset;
(void)SYS_landlock_restrict_self;
(void)LANDLOCK_CREATE_RULESET_VERSION;
return;
}
]])], [
enable_sandbox=found
AS_CASE([$CFLAGS], [*-fsanitize=*], [AC_MSG_ERROR([
CFLAGS contains '-fsanitize=' which is incompatible with the Landlock
sandboxing. Use --disable-sandbox when using '-fsanitize'.])])
AC_DEFINE([HAVE_LINUX_LANDLOCK], [1],
[Define to 1 if Linux Landlock is supported.
See configure.ac for details.])
AC_MSG_RESULT([yes])
], [
AC_MSG_RESULT([no])
])
]
)
# If a specific sandboxing method was explicitly requested and it wasn't
# found, give an error.
@ -1015,19 +1260,21 @@ if test -n "$CFLAG_VISIBILITY" && test "$is_w32" = no; then
AM_CFLAGS="$AM_CFLAGS $CFLAG_VISIBILITY"
fi
if test "$GCC" = yes ; then
AS_IF([test "$GCC" = yes], [
# Enable as much warnings as possible. These commented warnings won't
# work for this package though:
# * -Wunreachable-code breaks several assert(0) cases, which are
# backed up with "return LZMA_PROG_ERROR".
# * -Wcast-qual would break various things where we need a non-const
# pointer although we don't modify anything through it.
# * -Wcast-align breaks optimized CRC32 and CRC64 implementation
# on some architectures (not on x86), where this warning is bogus,
# because we take care of correct alignment.
# * -Winline, -Wdisabled-optimization, -Wunsafe-loop-optimizations
# don't seem so useful here; at least the last one gives some
# warnings which are not bugs.
# * -Wconversion still shows too many warnings.
#
# The flags before the empty line are for GCC and many of them
# are supported by Clang too. The flags after the empty line are
# for Clang.
for NEW_FLAG in \
-Wall \
-Wextra \
@ -1035,27 +1282,49 @@ if test "$GCC" = yes ; then
-Wformat=2 \
-Winit-self \
-Wmissing-include-dirs \
-Wstrict-aliasing \
-Wshift-overflow=2 \
-Wstrict-overflow=3 \
-Walloc-zero \
-Wduplicated-cond \
-Wfloat-equal \
-Wundef \
-Wshadow \
-Wpointer-arith \
-Wbad-function-cast \
-Wwrite-strings \
-Wdate-time \
-Wsign-conversion \
-Wfloat-conversion \
-Wlogical-op \
-Waggregate-return \
-Wstrict-prototypes \
-Wold-style-definition \
-Wmissing-prototypes \
-Wmissing-declarations \
-Wmissing-noreturn \
-Wredundant-decls
-Wredundant-decls \
\
-Wc99-compat \
-Wc11-extensions \
-Wc2x-compat \
-Wc2x-extensions \
-Wpre-c2x-compat \
-Warray-bounds-pointer-arithmetic \
-Wassign-enum \
-Wconditional-uninitialized \
-Wdocumentation \
-Wduplicate-enum \
-Wempty-translation-unit \
-Wflexible-array-extensions \
-Wmissing-variable-declarations \
-Wnewline-eof \
-Wshift-sign-overflow \
-Wstring-conversion
do
AC_MSG_CHECKING([if $CC accepts $NEW_FLAG])
OLD_CFLAGS="$CFLAGS"
CFLAGS="$CFLAGS $NEW_FLAG -Werror"
AC_COMPILE_IFELSE([AC_LANG_SOURCE(
[void foo(void); void foo(void) { }])], [
[[void foo(void); void foo(void) { }]])], [
AM_CFLAGS="$AM_CFLAGS $NEW_FLAG"
AC_MSG_RESULT([yes])
], [
@ -1071,7 +1340,7 @@ if test "$GCC" = yes ; then
if test "x$enable_werror" = "xyes"; then
AM_CFLAGS="$AM_CFLAGS -Werror"
fi
fi
])
###############################################################################
@ -1092,7 +1361,6 @@ xz=`echo xz | sed "$program_transform_name"`
AC_SUBST([xz])
AC_CONFIG_FILES([
Doxyfile
Makefile
po/Makefile.in
lib/Makefile

View File

@ -1,9 +1,5 @@
##
## SPDX-License-Identifier: 0BSD
## Author: Lasse Collin
##
## This file has been put into the public domain.
## You can do whatever you want with this file.
##
EXTRA_DIST = \
translation.bash

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file crc32.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file full_flush.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file hex2bin.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file known_sizes.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file memusage.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file repeat.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file sync_flush.c
@ -5,9 +7,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include "sysdefs.h"

View File

@ -1,4 +1,5 @@
#!/bin/bash
# SPDX-License-Identifier: 0BSD
###############################################################################
#
@ -20,9 +21,6 @@
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
###############################################################################
set -e

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file 01_compress_easy.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdbool.h>
@ -27,7 +26,7 @@ show_usage_and_exit(const char *argv0)
{
fprintf(stderr, "Usage: %s PRESET < INFILE > OUTFILE\n"
"PRESET is a number 0-9 and can optionally be "
"followed by `e' to indicate extreme preset\n",
"followed by 'e' to indicate extreme preset\n",
argv0);
exit(EXIT_FAILURE);
}

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file 02_decompress.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdbool.h>

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file 03_compress_custom.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdbool.h>

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file 04_compress_easy_mt.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdbool.h>

View File

@ -1,3 +1,5 @@
// SPDX-License-Identifier: 0BSD
///////////////////////////////////////////////////////////////////////////////
//
/// \file 11_file_info.c
@ -9,9 +11,6 @@
//
// Author: Lasse Collin
//
// This file has been put into the public domain.
// You can do whatever you want with this file.
//
///////////////////////////////////////////////////////////////////////////////
#include <stdbool.h>
@ -33,7 +32,7 @@ print_file_size(lzma_stream *strm, FILE *infile, const char *filename)
// large files on 32-bit systems (POSIX versions fseeko() and
// ftello() can support large files).
if (fseek(infile, 0, SEEK_END)) {
fprintf(stderr, "Error seeking the file `%s': %s\n",
fprintf(stderr, "Error seeking the file '%s': %s\n",
filename, strerror(errno));
return false;
}
@ -80,7 +79,7 @@ print_file_size(lzma_stream *strm, FILE *infile, const char *filename)
if (ferror(infile)) {
fprintf(stderr,
"Error reading from `%s': %s\n",
"Error reading from '%s': %s\n",
filename, strerror(errno));
return false;
}
@ -104,7 +103,7 @@ print_file_size(lzma_stream *strm, FILE *infile, const char *filename)
// switch fseeko() or lseek().
if (fseek(infile, (long)(strm->seek_pos), SEEK_SET)) {
fprintf(stderr, "Error seeking the "
"file `%s': %s\n",
"file '%s': %s\n",
filename, strerror(errno));
return false;
}
@ -133,24 +132,24 @@ print_file_size(lzma_stream *strm, FILE *infile, const char *filename)
case LZMA_FORMAT_ERROR:
// .xz magic bytes weren't found.
fprintf(stderr, "The file `%s' is not "
fprintf(stderr, "The file '%s' is not "
"in the .xz format\n", filename);
return false;
case LZMA_OPTIONS_ERROR:
fprintf(stderr, "The file `%s' has .xz headers that "
fprintf(stderr, "The file '%s' has .xz headers that "
"are not supported by this liblzma "
"version\n", filename);
return false;
case LZMA_DATA_ERROR:
fprintf(stderr, "The file `%s' is corrupt\n",
fprintf(stderr, "The file '%s' is corrupt\n",
filename);
return false;
case LZMA_MEM_ERROR:
fprintf(stderr, "Memory allocation failed when "
"decoding the file `%s'\n", filename);
"decoding the file '%s'\n", filename);
return false;
// LZMA_MEMLIMIT_ERROR shouldn't happen because we used
@ -183,7 +182,7 @@ main(int argc, char **argv)
FILE *infile = fopen(argv[i], "rb");
if (infile == NULL) {
fprintf(stderr, "Cannot open the file `%s': %s\n",
fprintf(stderr, "Cannot open the file '%s': %s\n",
argv[i], strerror(errno));
success = false;
}

View File

@ -1,9 +1,5 @@
#
# SPDX-License-Identifier: 0BSD
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
CC = c99
CFLAGS = -g

View File

@ -1,127 +0,0 @@
/*
* xz_pipe_comp.c
* A simple example of pipe-only xz compressor implementation.
* version: 2010-07-12 - by Daniel Mealha Cabrita
* Not copyrighted -- provided to the public domain.
*
* Compiling:
* Link with liblzma. GCC example:
* $ gcc -llzma xz_pipe_comp.c -o xz_pipe_comp
*
* Usage example:
* $ cat some_file | ./xz_pipe_comp > some_file.xz
*/
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <stdbool.h>
#include <lzma.h>
/* COMPRESSION SETTINGS */
/* analogous to xz CLI options: -0 to -9 */
#define COMPRESSION_LEVEL 6
/* boolean setting, analogous to xz CLI option: -e */
#define COMPRESSION_EXTREME true
/* see: /usr/include/lzma/check.h LZMA_CHECK_* */
#define INTEGRITY_CHECK LZMA_CHECK_CRC64
/* read/write buffer sizes */
#define IN_BUF_MAX 4096
#define OUT_BUF_MAX 4096
/* error codes */
#define RET_OK 0
#define RET_ERROR_INIT 1
#define RET_ERROR_INPUT 2
#define RET_ERROR_OUTPUT 3
#define RET_ERROR_COMPRESSION 4
/* note: in_file and out_file must be open already */
int xz_compress (FILE *in_file, FILE *out_file)
{
uint32_t preset = COMPRESSION_LEVEL | (COMPRESSION_EXTREME ? LZMA_PRESET_EXTREME : 0);
lzma_check check = INTEGRITY_CHECK;
lzma_stream strm = LZMA_STREAM_INIT; /* alloc and init lzma_stream struct */
uint8_t in_buf [IN_BUF_MAX];
uint8_t out_buf [OUT_BUF_MAX];
size_t in_len; /* length of useful data in in_buf */
size_t out_len; /* length of useful data in out_buf */
bool in_finished = false;
bool out_finished = false;
lzma_action action;
lzma_ret ret_xz;
int ret;
ret = RET_OK;
/* initialize xz encoder */
ret_xz = lzma_easy_encoder (&strm, preset, check);
if (ret_xz != LZMA_OK) {
fprintf (stderr, "lzma_easy_encoder error: %d\n", (int) ret_xz);
return RET_ERROR_INIT;
}
while ((! in_finished) && (! out_finished)) {
/* read incoming data */
in_len = fread (in_buf, 1, IN_BUF_MAX, in_file);
if (feof (in_file)) {
in_finished = true;
}
if (ferror (in_file)) {
in_finished = true;
ret = RET_ERROR_INPUT;
}
strm.next_in = in_buf;
strm.avail_in = in_len;
/* if no more data from in_buf, flushes the
internal xz buffers and closes the xz data
with LZMA_FINISH */
action = in_finished ? LZMA_FINISH : LZMA_RUN;
/* loop until there's no pending compressed output */
do {
/* out_buf is clean at this point */
strm.next_out = out_buf;
strm.avail_out = OUT_BUF_MAX;
/* compress data */
ret_xz = lzma_code (&strm, action);
if ((ret_xz != LZMA_OK) && (ret_xz != LZMA_STREAM_END)) {
fprintf (stderr, "lzma_code error: %d\n", (int) ret_xz);
out_finished = true;
ret = RET_ERROR_COMPRESSION;
} else {
/* write compressed data */
out_len = OUT_BUF_MAX - strm.avail_out;
fwrite (out_buf, 1, out_len, out_file);
if (ferror (out_file)) {
out_finished = true;
ret = RET_ERROR_OUTPUT;
}
}
} while (strm.avail_out == 0);
}
lzma_end (&strm);
return ret;
}
int main ()
{
int ret;
ret = xz_compress (stdin, stdout);
return ret;
}

View File

@ -1,123 +0,0 @@
/*
* xz_pipe_decomp.c
* A simple example of pipe-only xz decompressor implementation.
* version: 2012-06-14 - by Daniel Mealha Cabrita
* Not copyrighted -- provided to the public domain.
*
* Compiling:
* Link with liblzma. GCC example:
* $ gcc -llzma xz_pipe_decomp.c -o xz_pipe_decomp
*
* Usage example:
* $ cat some_file.xz | ./xz_pipe_decomp > some_file
*/
#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <stdbool.h>
#include <lzma.h>
/* read/write buffer sizes */
#define IN_BUF_MAX 4096
#define OUT_BUF_MAX 4096
/* error codes */
#define RET_OK 0
#define RET_ERROR_INIT 1
#define RET_ERROR_INPUT 2
#define RET_ERROR_OUTPUT 3
#define RET_ERROR_DECOMPRESSION 4
/* note: in_file and out_file must be open already */
int xz_decompress (FILE *in_file, FILE *out_file)
{
lzma_stream strm = LZMA_STREAM_INIT; /* alloc and init lzma_stream struct */
const uint32_t flags = LZMA_TELL_UNSUPPORTED_CHECK | LZMA_CONCATENATED;
const uint64_t memory_limit = UINT64_MAX; /* no memory limit */
uint8_t in_buf [IN_BUF_MAX];
uint8_t out_buf [OUT_BUF_MAX];
size_t in_len; /* length of useful data in in_buf */
size_t out_len; /* length of useful data in out_buf */
bool in_finished = false;
bool out_finished = false;
lzma_action action;
lzma_ret ret_xz;
int ret;
ret = RET_OK;
/* initialize xz decoder */
ret_xz = lzma_stream_decoder (&strm, memory_limit, flags);
if (ret_xz != LZMA_OK) {
fprintf (stderr, "lzma_stream_decoder error: %d\n", (int) ret_xz);
return RET_ERROR_INIT;
}
while ((! in_finished) && (! out_finished)) {
/* read incoming data */
in_len = fread (in_buf, 1, IN_BUF_MAX, in_file);
if (feof (in_file)) {
in_finished = true;
}
if (ferror (in_file)) {
in_finished = true;
ret = RET_ERROR_INPUT;
}
strm.next_in = in_buf;
strm.avail_in = in_len;
/* if no more data from in_buf, flushes the
internal xz buffers and closes the decompressed data
with LZMA_FINISH */
action = in_finished ? LZMA_FINISH : LZMA_RUN;
/* loop until there's no pending decompressed output */
do {
/* out_buf is clean at this point */
strm.next_out = out_buf;
strm.avail_out = OUT_BUF_MAX;
/* decompress data */
ret_xz = lzma_code (&strm, action);
if ((ret_xz != LZMA_OK) && (ret_xz != LZMA_STREAM_END)) {
fprintf (stderr, "lzma_code error: %d\n", (int) ret_xz);
out_finished = true;
ret = RET_ERROR_DECOMPRESSION;
} else {
/* write decompressed data */
out_len = OUT_BUF_MAX - strm.avail_out;
fwrite (out_buf, 1, out_len, out_file);
if (ferror (out_file)) {
out_finished = true;
ret = RET_ERROR_OUTPUT;
}
}
} while (strm.avail_out == 0);
}
/* Bug fix (2012-06-14): If no errors were detected, check
that the last lzma_code() call returned LZMA_STREAM_END.
If not, the file is probably truncated. */
if ((ret == RET_OK) && (ret_xz != LZMA_STREAM_END)) {
fprintf (stderr, "Input truncated or corrupt\n");
ret = RET_ERROR_DECOMPRESSION;
}
lzma_end (&strm);
return ret;
}
int main ()
{
int ret;
ret = xz_decompress (stdin, stdout);
return ret;
}

View File

@ -240,5 +240,5 @@ A: Give --enable-small to the configure script. Use also appropriate
If the result is still too big, take a look at XZ Embedded. It is
a separate project, which provides a limited but significantly
smaller XZ decoder implementation than XZ Utils. You can find it
at <https://tukaani.org/xz/embedded.html>.
at <https://xz.tukaani.org/xz-embedded/>.

View File

@ -40,11 +40,11 @@ The .lzma File Format
0.2. Changes
Last modified: 2022-07-13 21:00+0300
Last modified: 2024-01-16 18:00+0800
Compared to the previous version (2011-04-12 11:55+0300)
the section 1.1.3 was modified to allow End of Payload Marker
with a known Uncompressed Size.
Compared to the previous version (2022-07-13 21:00+0300)
the section 2 was modified to change links from http to
https and to update XZ links.
1. File Format
@ -157,17 +157,17 @@ The .lzma File Format
2. References
LZMA SDK - The original LZMA implementation
http://7-zip.org/sdk.html
https://7-zip.org/sdk.html
7-Zip
http://7-zip.org/
https://7-zip.org/
LZMA Utils - LZMA adapted to POSIX-like systems
http://tukaani.org/lzma/
https://tukaani.org/lzma/
XZ Utils - The next generation of LZMA Utils
http://tukaani.org/xz/
https://xz.tukaani.org/xz-utils/
The .xz file format - The successor of the .lzma format
http://tukaani.org/xz/xz-file-format.txt
https://xz.tukaani.org/format/xz-file-format.txt

View File

@ -2,7 +2,7 @@
The .xz File Format
===================
Version 1.1.0 (2022-12-11)
Version 1.2.0 (2024-01-19)
0. Preface
@ -81,18 +81,22 @@ Version 1.1.0 (2022-12-11)
0.2. Getting the Latest Version
The latest official version of this document can be downloaded
from <http://tukaani.org/xz/xz-file-format.txt>.
from <https://xz.tukaani.org/format/xz-file-format.txt>.
Specific versions of this document have a filename
xz-file-format-X.Y.Z.txt where X.Y.Z is the version number.
For example, the version 1.0.0 of this document is available
at <http://tukaani.org/xz/xz-file-format-1.0.0.txt>.
at <https://xz.tukaani.org/format/xz-file-format-1.0.0.txt>.
0.3. Version History
Version Date Description
1.2.0 2024-01-19 Added RISC-V filter and updated URLs in
Sections 0.2 and 7. The URL of this
specification was changed.
1.1.0 2022-12-11 Added ARM64 filter and clarified 32-bit
ARM endianness in Section 5.3.2,
language improvements in Section 5.4
@ -923,6 +927,7 @@ Version 1.1.0 (2022-12-11)
0x08 2 bytes ARM Thumb filter [1]
0x09 4 bytes SPARC filter
0x0A 4 bytes ARM64 filter [2]
0x0B 2 bytes RISC-V filter
[1] These are for little endian instruction encoding.
This must not be confused with data endianness.
@ -1136,30 +1141,30 @@ Version 1.1.0 (2022-12-11)
7. References
LZMA SDK - The original LZMA implementation
http://7-zip.org/sdk.html
https://7-zip.org/sdk.html
LZMA Utils - LZMA adapted to POSIX-like systems
http://tukaani.org/lzma/
https://tukaani.org/lzma/
XZ Utils - The next generation of LZMA Utils
http://tukaani.org/xz/
https://xz.tukaani.org/xz-utils/
[RFC-1952]
GZIP file format specification version 4.3
http://www.ietf.org/rfc/rfc1952.txt
https://www.ietf.org/rfc/rfc1952.txt
- Notation of byte boxes in section "2.1. Overall conventions"
[RFC-2119]
Key words for use in RFCs to Indicate Requirement Levels
http://www.ietf.org/rfc/rfc2119.txt
https://www.ietf.org/rfc/rfc2119.txt
[GNU-tar]
GNU tar 1.21 manual
http://www.gnu.org/software/tar/manual/html_node/Blocking-Factor.html
GNU tar 1.35 manual
https://www.gnu.org/software/tar/manual/html_node/Blocking-Factor.html
- Node 9.4.2 "Blocking Factor", paragraph that begins
"gzip will complain about trailing garbage"
- Note that this URL points to the latest version of the
manual, and may some day not contain the note which is in
1.21. For the exact version of the manual, download GNU
tar 1.21: ftp://ftp.gnu.org/pub/gnu/tar/tar-1.21.tar.gz
1.35. For the exact version of the manual, download GNU
tar 1.35: ftp://ftp.gnu.org/pub/gnu/tar/tar-1.35.tar.gz

BIN
doc/xz-logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.6 KiB

View File

@ -17,7 +17,7 @@ Introduction
Getting and Installing DJGPP
You may use <http://www.delorie.com/djgpp/zip-picker.html> to help
You may use <https://www.delorie.com/djgpp/zip-picker.html> to help
deciding what to download. If you are only interested in building
XZ Utils, the zip-picker may list files that you don't strictly
need. However, using the zip-picker can still be worth it to get a
@ -25,7 +25,7 @@ Getting and Installing DJGPP
from readme.1st too).
For a more manual method, first select a mirror from
<http://www.delorie.com/djgpp/getting.html> and go the
<https://www.delorie.com/djgpp/getting.html> and go the
subdirectory named "current". You need the following files:
unzip32.exe (if you don't already have a LFN-capable unzipper)

View File

@ -1,12 +1,11 @@
# SPDX-License-Identifier: 0BSD
###############################################################################
#
# Makefile to build XZ Utils using DJGPP
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
###############################################################################
# For debugging, set comment "#define NDEBUG 1" from config.h to enable

View File

@ -1,3 +1,5 @@
/* SPDX-License-Identifier: 0BSD */
/* How many MiB of RAM to assume if the real amount cannot be determined. */
#define ASSUME_RAM 32
@ -115,14 +117,14 @@
/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1
/* Define to 1 if you have the `utimes' function. */
/* Define to 1 if you have the 'utimes' function. */
#define HAVE_UTIMES 1
/* Define to 1 or 0, depending whether the compiler supports simple visibility
declarations. */
#define HAVE_VISIBILITY 0
/* Define to 1 if the system has the type `_Bool'. */
/* Define to 1 if the system has the type '_Bool'. */
#define HAVE__BOOL 1
/* Define to 1 if the GNU C extension __builtin_assume_aligned is supported.
@ -143,9 +145,9 @@
#define PACKAGE_NAME "XZ Utils"
/* Define to the home page for this package. */
#define PACKAGE_URL "https://tukaani.org/xz/"
#define PACKAGE_URL "https://xz.tukaani.org/xz-utils/"
/* The size of `size_t', as computed by sizeof. */
/* The size of 'size_t', as computed by sizeof. */
#define SIZEOF_SIZE_T 4
/* Define to 1 if the system supports fast unaligned access to 16-bit and

2687
doxygen/Doxyfile Normal file

File diff suppressed because it is too large Load Diff

13
doxygen/footer.html Normal file
View File

@ -0,0 +1,13 @@
<hr class="footer"/>
<p style="text-align: right;padding-right: 12px;">
XZ logo &copy; 2023 by Jia Tan is licensed under
<a href="COPYING.CC-BY-SA-4.0"
rel="license"
style="display:inline-block;">
CC BY-SA 4.0
</a>
</p>
</body>
</html>

109
doxygen/update-doxygen Executable file
View File

@ -0,0 +1,109 @@
#!/bin/sh
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# Updates the Doxygen generated documentation files in the source tree.
# If the doxygen command is not installed, it will exit with an error.
# This script can generate Doxygen documentation for all source files or for
# just liblzma API header files.
#
# It is recommended to use this script to update the Doxygen-generated HTML
# files since this will include the package version in the output and,
# in case of liblzma API docs, strip JavaScript files from the output.
#
#############################################################################
#
# Authors: Jia Tan
# Lasse Collin
#
#############################################################################
set -e
if type doxygen > /dev/null 2>&1; then
:
else
echo "doxygen/update-doxygen: 'doxygen' command not found." >&2
echo "doxygen/update-doxygen: Skipping Doxygen docs generation." >&2
exit 1
fi
if test ! -f Doxyfile; then
cd `dirname "$0"` || exit 1
if test ! -f Doxyfile; then
echo "doxygen/update-doxygen: Cannot find Doxyfile" >&2
exit 1
fi
fi
# Get the package version so that it can be included in the generated docs.
PACKAGE_VERSION=`cd .. && sh build-aux/version.sh` || exit 1
# If no arguments are specified, default to generating liblzma API header
# documentation only.
case $1 in
'' | api)
# Remove old documentation before re-generating the new.
rm -rf ../doc/api
# Generate the HTML documentation by preparing the Doxyfile
# in stdin and piping the result to the doxygen command.
# With Doxygen, the last assignment of a value to a tag will
# override any earlier assignment. So, we can use this
# feature to override the tags that need to change between
# "api" and "internal" modes.
(
cat Doxyfile
echo "PROJECT_NUMBER = $PACKAGE_VERSION"
) | doxygen -
# As of Doxygen 1.8.0 - 1.9.6 and the Doxyfile options we use,
# the output is good without any JavaScript. Unfortunately
# Doxygen doesn't have an option to disable JavaScript usage
# completely so we strip it away with the hack below.
#
# Omitting the JavaScript code avoids some license hassle
# as jquery.js is fairly big, it contains more than jQuery
# itself, and doesn't include the actual license text (it
# only refers to the MIT license by name).
echo "Stripping JavaScript from Doxygen output..."
for F in ../doc/api/*.html
do
sed 's/<script [^>]*><\/script>//g
s/onclick="[^"]*"//g' \
"$F" > ../doc/api/tmp
mv -f ../doc/api/tmp "$F"
done
rm -f ../doc/api/*.js
;;
internal)
# The docs from internal aren't for distribution so
# the JavaScript files aren't an issue here.
rm -rf ../doc/internal
(
cat Doxyfile
echo "PROJECT_NUMBER = $PACKAGE_VERSION"
echo 'PROJECT_NAME = "XZ Utils"'
echo 'STRIP_FROM_PATH = ../src'
echo 'INPUT = ../src'
echo 'HTML_OUTPUT = internal'
echo 'EXTRACT_PRIVATE = YES'
echo 'EXTRACT_STATIC = YES'
echo 'EXTRACT_LOCAL_CLASSES = YES'
echo 'SEARCHENGINE = YES'
) | doxygen -
;;
*)
echo "doxygen/update-doxygen: Error: mode argument '$1'" \
"is not supported." >&2
echo "doxygen/update-doxygen: Supported modes:" >&2
echo "doxygen/update-doxygen: - 'api' (default):" \
"liblzma API docs into doc/api" >&2
echo "doxygen/update-doxygen: - 'internal':"\
"internal docs into doc/internal" >&2
exit 1
;;
esac

View File

@ -1,5 +1,6 @@
#!/bin/bash
#
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# 7z2lzma.bash is very primitive .7z to .lzma converter. The input file must
@ -17,9 +18,6 @@
#
# Author: Lasse Collin <lasse.collin@tukaani.org>
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
#
#############################################################################
# You can use 7z or 7za, both will work.

View File

@ -1,3 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
scanlzma, scan for lzma compressed data in stdin and echo it to stdout.
Copyright (C) 2006 Timo Lindfors

View File

@ -1,3 +1,5 @@
## SPDX-License-Identifier: GPL-2.0-or-later
##
## Copyright (C) 2004-2007 Free Software Foundation, Inc.
##
@ -21,7 +23,17 @@ libgnu_a_SOURCES =
libgnu_a_DEPENDENCIES = $(LIBOBJS)
libgnu_a_LIBADD = $(LIBOBJS)
EXTRA_DIST = getopt.in.h getopt.c getopt1.c getopt_int.h
EXTRA_DIST = \
getopt.in.h \
getopt.c \
getopt1.c \
getopt_int.h \
getopt-cdefs.h \
getopt-core.h \
getopt-ext.h \
getopt-pfx-core.h \
getopt-pfx-ext.h
BUILT_SOURCES = $(GETOPT_H)
MOSTLYCLEANFILES = getopt.h getopt.h-t

72
lib/getopt-cdefs.h Normal file
View File

@ -0,0 +1,72 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* getopt-on-non-glibc compatibility macros.
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of gnulib.
Unlike most of the getopt implementation, it is NOT shared
with the GNU C Library.
This file is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as
published by the Free Software Foundation; either version 2.1 of the
License, or (at your option) any later version.
This file is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_CDEFS_H
#define _GETOPT_CDEFS_H 1
/* This header should not be used directly; include getopt.h or
unistd.h instead. It does not have a protective #error, because
the guard macro for getopt.h in gnulib is not fixed. */
/* getopt-core.h and getopt-ext.h are shared with GNU libc, and expect
a number of the internal macros supplied to GNU libc's headers by
sys/cdefs.h. Provide fallback definitions for all of them. */
#ifdef HAVE_SYS_CDEFS_H
# include <sys/cdefs.h>
#endif
#ifndef __BEGIN_DECLS
# ifdef __cplusplus
# define __BEGIN_DECLS extern "C" {
# else
# define __BEGIN_DECLS /* nothing */
# endif
#endif
#ifndef __END_DECLS
# ifdef __cplusplus
# define __END_DECLS }
# else
# define __END_DECLS /* nothing */
# endif
#endif
#ifndef __GNUC_PREREQ
# if defined __GNUC__ && defined __GNUC_VERSION__
# define __GNUC_PREREQ(maj, min) \
((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min))
# else
# define __GNUC_PREREQ(maj, min) 0
# endif
#endif
#ifndef __THROW
# if defined __cplusplus && (__GNUC_PREREQ (2,8) || __clang_major__ >= 4)
# if __cplusplus >= 201103L
# define __THROW noexcept (true)
# else
# define __THROW throw ()
# endif
# else
# define __THROW
# endif
#endif
#endif /* _GETOPT_CDEFS_H */

98
lib/getopt-core.h Normal file
View File

@ -0,0 +1,98 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* Declarations for getopt (basic, portable features only).
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of the GNU C Library and is also part of gnulib.
Patches to this file should be submitted to both projects.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
<https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_CORE_H
#define _GETOPT_CORE_H 1
/* This header should not be used directly; include getopt.h or
unistd.h instead. Unlike most bits headers, it does not have
a protective #error, because the guard macro for getopt.h in
gnulib is not fixed. */
__BEGIN_DECLS
/* For communication from 'getopt' to the caller.
When 'getopt' finds an option that takes an argument,
the argument value is returned here.
Also, when 'ordering' is RETURN_IN_ORDER,
each non-option ARGV-element is returned here. */
extern char *optarg;
/* Index in ARGV of the next element to be scanned.
This is used for communication to and from the caller
and for communication between successive calls to 'getopt'.
On entry to 'getopt', zero means this is the first call; initialize.
When 'getopt' returns -1, this is the index of the first of the
non-option elements that the caller should itself scan.
Otherwise, 'optind' communicates from one call to the next
how much of ARGV has been scanned so far. */
extern int optind;
/* Callers store zero here to inhibit the error message 'getopt' prints
for unrecognized options. */
extern int opterr;
/* Set to an option character which was unrecognized. */
extern int optopt;
/* Get definitions and prototypes for functions to process the
arguments in ARGV (ARGC of them, minus the program name) for
options given in OPTS.
Return the option character from OPTS just read. Return -1 when
there are no more options. For unrecognized options, or options
missing arguments, 'optopt' is set to the option letter, and '?' is
returned.
The OPTS string is a list of characters which are recognized option
letters, optionally followed by colons, specifying that that letter
takes an argument, to be placed in 'optarg'.
If a letter in OPTS is followed by two colons, its argument is
optional. This behavior is specific to the GNU 'getopt'.
The argument '--' causes premature termination of argument
scanning, explicitly telling 'getopt' that there are no more
options.
If OPTS begins with '-', then non-option arguments are treated as
arguments to the option '\1'. This behavior is specific to the GNU
'getopt'. If OPTS begins with '+', or POSIXLY_CORRECT is set in
the environment, then do not permute arguments.
For standards compliance, the 'argv' argument has the type
char *const *, but this is inaccurate; if argument permutation is
enabled, the argv array (not the strings it points to) must be
writable. */
extern int getopt (int ___argc, char *const *___argv, const char *__shortopts)
__THROW _GL_ARG_NONNULL ((2, 3));
__END_DECLS
#endif /* _GETOPT_CORE_H */

79
lib/getopt-ext.h Normal file
View File

@ -0,0 +1,79 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* Declarations for getopt (GNU extensions).
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of the GNU C Library and is also part of gnulib.
Patches to this file should be submitted to both projects.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
<https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_EXT_H
#define _GETOPT_EXT_H 1
/* This header should not be used directly; include getopt.h instead.
Unlike most bits headers, it does not have a protective #error,
because the guard macro for getopt.h in gnulib is not fixed. */
__BEGIN_DECLS
/* Describe the long-named options requested by the application.
The LONG_OPTIONS argument to getopt_long or getopt_long_only is a vector
of 'struct option' terminated by an element containing a name which is
zero.
The field 'has_arg' is:
no_argument (or 0) if the option does not take an argument,
required_argument (or 1) if the option requires an argument,
optional_argument (or 2) if the option takes an optional argument.
If the field 'flag' is not NULL, it points to a variable that is set
to the value given in the field 'val' when the option is found, but
left unchanged if the option is not found.
To have a long-named option do something other than set an 'int' to
a compiled-in constant, such as set a value from 'optarg', set the
option's 'flag' field to zero and its 'val' field to a nonzero
value (the equivalent single-letter option character, if there is
one). For long options that have a zero 'flag' field, 'getopt'
returns the contents of the 'val' field. */
struct option
{
const char *name;
/* has_arg can't be an enum because some compilers complain about
type mismatches in all the code that assumes it is an int. */
int has_arg;
int *flag;
int val;
};
/* Names for the values of the 'has_arg' field of 'struct option'. */
#define no_argument 0
#define required_argument 1
#define optional_argument 2
extern int getopt_long (int ___argc, char *__getopt_argv_const *___argv,
const char *__shortopts,
const struct option *__longopts, int *__longind)
__THROW _GL_ARG_NONNULL ((2, 3));
extern int getopt_long_only (int ___argc, char *__getopt_argv_const *___argv,
const char *__shortopts,
const struct option *__longopts, int *__longind)
__THROW _GL_ARG_NONNULL ((2, 3));
__END_DECLS
#endif /* _GETOPT_EXT_H */

68
lib/getopt-pfx-core.h Normal file
View File

@ -0,0 +1,68 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* getopt (basic, portable features) gnulib wrapper header.
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of gnulib.
Unlike most of the getopt implementation, it is NOT shared
with the GNU C Library.
This file is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as
published by the Free Software Foundation; either version 2.1 of the
License, or (at your option) any later version.
This file is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_PFX_CORE_H
#define _GETOPT_PFX_CORE_H 1
/* This header should not be used directly; include getopt.h or
unistd.h instead. It does not have a protective #error, because
the guard macro for getopt.h in gnulib is not fixed. */
/* Standalone applications should #define __GETOPT_PREFIX to an
identifier that prefixes the external functions and variables
defined in getopt-core.h and getopt-ext.h. Systematically
rename identifiers so that they do not collide with the system
functions and variables. Renaming avoids problems with some
compilers and linkers. */
#ifdef __GETOPT_PREFIX
# ifndef __GETOPT_ID
# define __GETOPT_CONCAT(x, y) x ## y
# define __GETOPT_XCONCAT(x, y) __GETOPT_CONCAT (x, y)
# define __GETOPT_ID(y) __GETOPT_XCONCAT (__GETOPT_PREFIX, y)
# endif
# undef getopt
# undef optarg
# undef opterr
# undef optind
# undef optopt
# define getopt __GETOPT_ID (getopt)
# define optarg __GETOPT_ID (optarg)
# define opterr __GETOPT_ID (opterr)
# define optind __GETOPT_ID (optind)
# define optopt __GETOPT_ID (optopt)
/* Work around a problem on macOS, which declares getopt with a
trailing __DARWIN_ALIAS(getopt) that would expand to something like
__asm("_" "rpl_getopt" "$UNIX2003") were it not for the following
hack to suppress the macOS declaration <https://bugs.gnu.org/40205>. */
# ifdef __APPLE__
# define _GETOPT
# endif
/* The system's getopt.h may have already included getopt-core.h to
declare the unprefixed identifiers. Undef _GETOPT_CORE_H so that
getopt-core.h declares them with prefixes. */
# undef _GETOPT_CORE_H
#endif
#include <getopt-core.h>
#endif /* _GETOPT_PFX_CORE_H */

72
lib/getopt-pfx-ext.h Normal file
View File

@ -0,0 +1,72 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* getopt (GNU extensions) gnulib wrapper header.
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of gnulib.
Unlike most of the getopt implementation, it is NOT shared
with the GNU C Library.
This file is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as
published by the Free Software Foundation; either version 2.1 of the
License, or (at your option) any later version.
This file is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_PFX_EXT_H
#define _GETOPT_PFX_EXT_H 1
/* This header should not be used directly; include getopt.h instead.
It does not have a protective #error, because the guard macro for
getopt.h in gnulib is not fixed. */
/* Standalone applications should #define __GETOPT_PREFIX to an
identifier that prefixes the external functions and variables
defined in getopt-core.h and getopt-ext.h. Systematically
rename identifiers so that they do not collide with the system
functions and variables. Renaming avoids problems with some
compilers and linkers. */
#ifdef __GETOPT_PREFIX
# ifndef __GETOPT_ID
# define __GETOPT_CONCAT(x, y) x ## y
# define __GETOPT_XCONCAT(x, y) __GETOPT_CONCAT (x, y)
# define __GETOPT_ID(y) __GETOPT_XCONCAT (__GETOPT_PREFIX, y)
# endif
# undef getopt_long
# undef getopt_long_only
# undef option
# undef _getopt_internal
# define getopt_long __GETOPT_ID (getopt_long)
# define getopt_long_only __GETOPT_ID (getopt_long_only)
# define option __GETOPT_ID (option)
# define _getopt_internal __GETOPT_ID (getopt_internal)
/* The system's getopt.h may have already included getopt-ext.h to
declare the unprefixed identifiers. Undef _GETOPT_EXT_H so that
getopt-ext.h declares them with prefixes. */
# undef _GETOPT_EXT_H
#endif
/* Standalone applications get correct prototypes for getopt_long and
getopt_long_only; they declare "char **argv". For backward
compatibility with old applications, if __GETOPT_PREFIX is not
defined, we supply GNU-libc-compatible, but incorrect, prototypes
using "char *const *argv". (GNU libc is stuck with the incorrect
prototypes, as they are baked into older versions of LSB.) */
#ifndef __getopt_argv_const
# if defined __GETOPT_PREFIX
# define __getopt_argv_const /* empty */
# else
# define __getopt_argv_const const
# endif
#endif
#include <getopt-ext.h>
#endif /* _GETOPT_PFX_EXT_H */

File diff suppressed because it is too large Load Diff

View File

@ -1,27 +1,28 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* Declarations for getopt.
Copyright (C) 1989-1994,1996-1999,2001,2003,2004,2005,2006,2007
Free Software Foundation, Inc.
This file is part of the GNU C Library.
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of gnulib.
Unlike most of the getopt implementation, it is NOT shared
with the GNU C Library, which supplies a different version of
this file.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation; either version 2.1, or (at your option)
any later version.
This file is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as
published by the Free Software Foundation; either version 2.1 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
This file is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along
with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_H
#ifndef __need_getopt
# define _GETOPT_H 1
#endif
#define _GETOPT_H 1
/* Standalone applications should #define __GETOPT_PREFIX to an
identifier that prefixes the external functions and variables
@ -31,196 +32,29 @@
identifiers so that they do not collide with the system functions
and variables. Renaming avoids problems with some compilers and
linkers. */
#if defined __GETOPT_PREFIX && !defined __need_getopt
#if defined __GETOPT_PREFIX
# include <stdlib.h>
# include <stdio.h>
# include <unistd.h>
# undef __need_getopt
# undef getopt
# undef getopt_long
# undef getopt_long_only
# undef optarg
# undef opterr
# undef optind
# undef optopt
# define __GETOPT_CONCAT(x, y) x ## y
# define __GETOPT_XCONCAT(x, y) __GETOPT_CONCAT (x, y)
# define __GETOPT_ID(y) __GETOPT_XCONCAT (__GETOPT_PREFIX, y)
# define getopt __GETOPT_ID (getopt)
# define getopt_long __GETOPT_ID (getopt_long)
# define getopt_long_only __GETOPT_ID (getopt_long_only)
# define optarg __GETOPT_ID (optarg)
# define opterr __GETOPT_ID (opterr)
# define optind __GETOPT_ID (optind)
# define optopt __GETOPT_ID (optopt)
# ifndef _MSC_VER
# include <unistd.h>
# endif
#endif
/* Standalone applications get correct prototypes for getopt_long and
getopt_long_only; they declare "char **argv". libc uses prototypes
with "char *const *argv" that are incorrect because getopt_long and
getopt_long_only can permute argv; this is required for backward
compatibility (e.g., for LSB 2.0.1).
This used to be `#if defined __GETOPT_PREFIX && !defined __need_getopt',
but it caused redefinition warnings if both unistd.h and getopt.h were
included, since unistd.h includes getopt.h having previously defined
__need_getopt.
The only place where __getopt_argv_const is used is in definitions
of getopt_long and getopt_long_only below, but these are visible
only if __need_getopt is not defined, so it is quite safe to rewrite
the conditional as follows:
*/
#if !defined __need_getopt
# if defined __GETOPT_PREFIX
# define __getopt_argv_const /* empty */
/* From Gnulib's lib/arg-nonnull.h: */
/* _GL_ARG_NONNULL((n,...,m)) tells the compiler and static analyzer tools
that the values passed as arguments n, ..., m must be non-NULL pointers.
n = 1 stands for the first argument, n = 2 for the second argument etc. */
#ifndef _GL_ARG_NONNULL
# if __GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 3) || defined __clang__
# define _GL_ARG_NONNULL(params) __attribute__ ((__nonnull__ params))
# else
# define __getopt_argv_const const
# define _GL_ARG_NONNULL(params)
# endif
#endif
/* If __GNU_LIBRARY__ is not already defined, either we are being used
standalone, or this is the first header included in the source file.
If we are being used with glibc, we need to include <features.h>, but
that does not exist if we are standalone. So: if __GNU_LIBRARY__ is
not defined, include <ctype.h>, which will pull in <features.h> for us
if it's from glibc. (Why ctype.h? It's guaranteed to exist and it
doesn't flood the namespace with stuff the way some other headers do.) */
#if !defined __GNU_LIBRARY__
# include <ctype.h>
#endif
#include <getopt-cdefs.h>
#include <getopt-pfx-core.h>
#include <getopt-pfx-ext.h>
#ifndef __THROW
# ifndef __GNUC_PREREQ
# define __GNUC_PREREQ(maj, min) (0)
# endif
# if defined __cplusplus && __GNUC_PREREQ (2,8)
# define __THROW throw ()
# else
# define __THROW
# endif
#endif
#ifdef __cplusplus
extern "C" {
#endif
/* For communication from `getopt' to the caller.
When `getopt' finds an option that takes an argument,
the argument value is returned here.
Also, when `ordering' is RETURN_IN_ORDER,
each non-option ARGV-element is returned here. */
extern char *optarg;
/* Index in ARGV of the next element to be scanned.
This is used for communication to and from the caller
and for communication between successive calls to `getopt'.
On entry to `getopt', zero means this is the first call; initialize.
When `getopt' returns -1, this is the index of the first of the
non-option elements that the caller should itself scan.
Otherwise, `optind' communicates from one call to the next
how much of ARGV has been scanned so far. */
extern int optind;
/* Callers store zero here to inhibit the error message `getopt' prints
for unrecognized options. */
extern int opterr;
/* Set to an option character which was unrecognized. */
extern int optopt;
#ifndef __need_getopt
/* Describe the long-named options requested by the application.
The LONG_OPTIONS argument to getopt_long or getopt_long_only is a vector
of `struct option' terminated by an element containing a name which is
zero.
The field `has_arg' is:
no_argument (or 0) if the option does not take an argument,
required_argument (or 1) if the option requires an argument,
optional_argument (or 2) if the option takes an optional argument.
If the field `flag' is not NULL, it points to a variable that is set
to the value given in the field `val' when the option is found, but
left unchanged if the option is not found.
To have a long-named option do something other than set an `int' to
a compiled-in constant, such as set a value from `optarg', set the
option's `flag' field to zero and its `val' field to a nonzero
value (the equivalent single-letter option character, if there is
one). For long options that have a zero `flag' field, `getopt'
returns the contents of the `val' field. */
struct option
{
const char *name;
/* has_arg can't be an enum because some compilers complain about
type mismatches in all the code that assumes it is an int. */
int has_arg;
int *flag;
int val;
};
/* Names for the values of the `has_arg' field of `struct option'. */
# define no_argument 0
# define required_argument 1
# define optional_argument 2
#endif /* need getopt */
/* Get definitions and prototypes for functions to process the
arguments in ARGV (ARGC of them, minus the program name) for
options given in OPTS.
Return the option character from OPTS just read. Return -1 when
there are no more options. For unrecognized options, or options
missing arguments, `optopt' is set to the option letter, and '?' is
returned.
The OPTS string is a list of characters which are recognized option
letters, optionally followed by colons, specifying that that letter
takes an argument, to be placed in `optarg'.
If a letter in OPTS is followed by two colons, its argument is
optional. This behavior is specific to the GNU `getopt'.
The argument `--' causes premature termination of argument
scanning, explicitly telling `getopt' that there are no more
options.
If OPTS begins with `-', then non-option arguments are treated as
arguments to the option '\1'. This behavior is specific to the GNU
`getopt'. If OPTS begins with `+', or POSIXLY_CORRECT is set in
the environment, then do not permute arguments. */
extern int getopt (int ___argc, char *const *___argv, const char *__shortopts)
__THROW;
#ifndef __need_getopt
extern int getopt_long (int ___argc, char *__getopt_argv_const *___argv,
const char *__shortopts,
const struct option *__longopts, int *__longind)
__THROW;
extern int getopt_long_only (int ___argc, char *__getopt_argv_const *___argv,
const char *__shortopts,
const struct option *__longopts, int *__longind)
__THROW;
#endif
#ifdef __cplusplus
}
#endif
/* Make sure we later can get all the definitions and declarations. */
#undef __need_getopt
#endif /* getopt.h */
#endif /* _GETOPT_H */

View File

@ -1,42 +1,33 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* getopt_long and getopt_long_only entry points for GNU getopt.
Copyright (C) 1987,88,89,90,91,92,93,94,96,97,98,2004,2006
Free Software Foundation, Inc.
This file is part of the GNU C Library.
Copyright (C) 1987-2023 Free Software Foundation, Inc.
This file is part of the GNU C Library and is also part of gnulib.
Patches to this file should be submitted to both projects.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation; either version 2.1, or (at your option)
any later version.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Lesser General Public License for more details.
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along
with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
<https://www.gnu.org/licenses/>. */
#ifdef _LIBC
# include <getopt.h>
#else
# include <config.h>
# include "getopt.h"
#ifndef _LIBC
# ifdef HAVE_CONFIG_H
# include <config.h>
# endif
#endif
#include "getopt.h"
#include "getopt_int.h"
#include <stdio.h>
/* This needs to come after some library #include
to get __GNU_LIBRARY__ defined. */
#ifdef __GNU_LIBRARY__
#include <stdlib.h>
#endif
#ifndef NULL
#define NULL 0
#endif
int
getopt_long (int argc, char *__getopt_argv_const *argv, const char *options,
const struct option *long_options, int *opt_index)
@ -51,7 +42,7 @@ _getopt_long_r (int argc, char **argv, const char *options,
struct _getopt_data *d)
{
return _getopt_internal_r (argc, argv, options, long_options, opt_index,
0, 0, d);
0, d, 0);
}
/* Like getopt_long, but '-' as well as '--' can indicate a long option.
@ -74,13 +65,14 @@ _getopt_long_only_r (int argc, char **argv, const char *options,
struct _getopt_data *d)
{
return _getopt_internal_r (argc, argv, options, long_options, opt_index,
1, 0, d);
1, d, 0);
}
#ifdef TEST
#include <stdio.h>
#include <stdlib.h>
int
main (int argc, char **argv)
@ -92,7 +84,7 @@ main (int argc, char **argv)
{
int this_option_optind = optind ? optind : 1;
int option_index = 0;
static struct option long_options[] =
static const struct option long_options[] =
{
{"add", 1, 0, 0},
{"append", 0, 0, 0},
@ -142,11 +134,11 @@ main (int argc, char **argv)
break;
case 'c':
printf ("option c with value `%s'\n", optarg);
printf ("option c with value '%s'\n", optarg);
break;
case 'd':
printf ("option d with value `%s'\n", optarg);
printf ("option d with value '%s'\n", optarg);
break;
case '?':

View File

@ -1,34 +1,64 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
/* Internal declarations for getopt.
Copyright (C) 1989-1994,1996-1999,2001,2003,2004
Free Software Foundation, Inc.
This file is part of the GNU C Library.
Copyright (C) 1989-2023 Free Software Foundation, Inc.
This file is part of the GNU C Library and is also part of gnulib.
Patches to this file should be submitted to both projects.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. */
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
<https://www.gnu.org/licenses/>. */
#ifndef _GETOPT_INT_H
#define _GETOPT_INT_H 1
#include <getopt.h>
extern int _getopt_internal (int ___argc, char **___argv,
const char *__shortopts,
const struct option *__longopts, int *__longind,
const struct option *__longopts, int *__longind,
int __long_only, int __posixly_correct);
/* Reentrant versions which can handle parsing multiple argument
vectors at the same time. */
/* Describe how to deal with options that follow non-option ARGV-elements.
REQUIRE_ORDER means don't recognize them as options; stop option
processing when the first non-option is seen. This is what POSIX
specifies should happen.
PERMUTE means permute the contents of ARGV as we scan, so that
eventually all the non-options are at the end. This allows options
to be given in any order, even with programs that were not written
to expect this.
RETURN_IN_ORDER is an option available to programs that were
written to expect options and other ARGV-elements in any order
and that care about the ordering of the two. We describe each
non-option ARGV-element as if it were the argument of an option
with character code 1.
The special argument '--' forces an end of option-scanning regardless
of the value of 'ordering'. In the case of RETURN_IN_ORDER, only
'--' can cause 'getopt' to return -1 with 'optind' != ARGC. */
enum __ord
{
REQUIRE_ORDER, PERMUTE, RETURN_IN_ORDER
};
/* Data type for reentrant functions. */
struct _getopt_data
{
@ -53,58 +83,17 @@ struct _getopt_data
by advancing to the next ARGV-element. */
char *__nextchar;
/* Describe how to deal with options that follow non-option ARGV-elements.
If the caller did not specify anything,
the default is REQUIRE_ORDER if the environment variable
POSIXLY_CORRECT is defined, PERMUTE otherwise.
REQUIRE_ORDER means don't recognize them as options;
stop option processing when the first non-option is seen.
This is what Unix does.
This mode of operation is selected by either setting the environment
variable POSIXLY_CORRECT, or using `+' as the first character
of the list of option characters, or by calling getopt.
PERMUTE is the default. We permute the contents of ARGV as we
scan, so that eventually all the non-options are at the end.
This allows options to be given in any order, even with programs
that were not written to expect this.
RETURN_IN_ORDER is an option available to programs that were
written to expect options and other ARGV-elements in any order
and that care about the ordering of the two. We describe each
non-option ARGV-element as if it were the argument of an option
with character code 1. Using `-' as the first character of the
list of option characters selects this mode of operation.
The special argument `--' forces an end of option-scanning regardless
of the value of `ordering'. In the case of RETURN_IN_ORDER, only
`--' can cause `getopt' to return -1 with `optind' != ARGC. */
enum
{
REQUIRE_ORDER, PERMUTE, RETURN_IN_ORDER
} __ordering;
/* If the POSIXLY_CORRECT environment variable is set
or getopt was called. */
int __posixly_correct;
/* See __ord above. */
enum __ord __ordering;
/* Handle permutation of arguments. */
/* Describe the part of ARGV that contains non-options that have
been skipped. `first_nonopt' is the index in ARGV of the first
of them; `last_nonopt' is the index after the last of them. */
been skipped. 'first_nonopt' is the index in ARGV of the first
of them; 'last_nonopt' is the index after the last of them. */
int __first_nonopt;
int __last_nonopt;
#if defined _LIBC && defined USE_NONOPTION_FLAGS
int __nonoption_flags_max_len;
int __nonoption_flags_len;
# endif
};
/* The initializer is necessary to set OPTIND and OPTERR to their
@ -114,8 +103,8 @@ struct _getopt_data
extern int _getopt_internal_r (int ___argc, char **___argv,
const char *__shortopts,
const struct option *__longopts, int *__longind,
int __long_only, int __posixly_correct,
struct _getopt_data *__data);
int __long_only, struct _getopt_data *__data,
int __posixly_correct);
extern int _getopt_long_r (int ___argc, char **___argv,
const char *__shortopts,

1
m4/.gitignore vendored
View File

@ -1,3 +1,4 @@
build-to-host.m4
codeset.m4
extern-inline.m4
fcntl-o.m4

View File

@ -1,85 +0,0 @@
# -*- Autoconf -*-
# SYNOPSIS
#
# AX_CHECK_CAPSICUM([ACTION-IF-FOUND[, ACTION-IF-NOT-FOUND]])
#
# DESCRIPTION
#
# This macro searches for an installed Capsicum header and library,
# and if found:
# - AC_DEFINE([HAVE_CAPSICUM]) is called.
# - AC_DEFINE([HAVE_SYS_CAPSICUM_H]) is called if <sys/capsicum.h>
# is present (otherwise <sys/capability.h> must be used).
# - CAPSICUM_LIB is set to the -l option needed to link Capsicum support,
# and AC_SUBST([CAPSICUM_LIB]) is called.
# - The shell commands in ACTION-IF-FOUND are run. The default
# ACTION-IF-FOUND prepends ${CAPSICUM_LIB} into LIBS. If you don't
# want to modify LIBS and don't need to run any other commands either,
# use a colon as ACTION-IF-FOUND.
#
# If Capsicum support isn't found:
# - The shell commands in ACTION-IF-NOT-FOUND are run. The default
# ACTION-IF-NOT-FOUND calls AC_MSG_WARN to print a warning that
# Capsicum support wasn't found.
#
# You should use autoheader to include a definition for the symbols above
# in a config.h file.
#
# Sample usage in a C/C++ source is as follows:
#
# #ifdef HAVE_CAPSICUM
# # ifdef HAVE_SYS_CAPSICUM_H
# # include <sys/capsicum.h>
# # else
# # include <sys/capability.h>
# # endif
# #endif /* HAVE_CAPSICUM */
#
# LICENSE
#
# Copyright (c) 2014 Google Inc.
# Copyright (c) 2015 Lasse Collin <lasse.collin@tukaani.org>
#
# Copying and distribution of this file, with or without modification,
# are permitted in any medium without royalty provided the copyright
# notice and this notice are preserved. This file is offered as-is,
# without any warranty.
#serial 2
AC_DEFUN([AX_CHECK_CAPSICUM], [
# On FreeBSD >= 11.x and Linux, Capsicum is uses <sys/capsicum.h>.
# If this header is found, it is assumed to be the right one.
capsicum_header_found=no
AC_CHECK_HEADERS([sys/capsicum.h], [capsicum_header_found=yes])
if test "$capsicum_header_found" = no ; then
# On FreeBSD 10.x Capsicum uses <sys/capability.h>. Such a header exists
# on Linux too but it describes POSIX.1e capabilities. Look for the
# declaration of cap_rights_limit to check if <sys/capability.h> is
# a Capsicum header.
AC_CHECK_DECL([cap_rights_limit], [capsicum_header_found=yes], [],
[#include <sys/capability.h>])
fi
capsicum_lib_found=no
CAPSICUM_LIB=
if test "$capsicum_header_found" = yes ; then
AC_LANG_PUSH([C])
# FreeBSD >= 10.x has Capsicum functions in libc.
AC_LINK_IFELSE([AC_LANG_CALL([], [cap_rights_limit])],
[capsicum_lib_found=yes], [])
# Linux has Capsicum functions in libcaprights.
AC_CHECK_LIB([caprights], [cap_rights_limit],
[CAPSICUM_LIB=-lcaprights
capsicum_lib_found=yes], [])
AC_LANG_POP([C])
fi
AC_SUBST([CAPSICUM_LIB])
if test "$capsicum_lib_found" = yes ; then
AC_DEFINE([HAVE_CAPSICUM], [1], [Define to 1 if Capsicum is available.])
m4_default([$1], [LIBS="${CAPSICUM_LIB} $LIBS"])
else
m4_default([$2], [AC_MSG_WARN([Capsicum support not found])])
fi])

View File

@ -1,3 +1,5 @@
dnl SPDX-License-Identifier: GPL-3.0-or-later WITH Autoconf-exception-macro
# ===========================================================================
# https://www.gnu.org/software/autoconf-archive/ax_pthread.html
# ===========================================================================

View File

@ -1,71 +1,72 @@
# getopt.m4 serial 14 (modified version)
dnl Copyright (C) 2002-2006, 2008 Free Software Foundation, Inc.
dnl SPDX-License-Identifier: FSFULLR
# getopt.m4 serial 49 (modified version)
dnl Copyright (C) 2002-2006, 2008-2023 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
# The getopt module assume you want GNU getopt, with getopt_long etc,
# rather than vanilla POSIX getopt. This means your code should
# always include <getopt.h> for the getopt prototypes.
# This version has been modified to reduce complexity since we only need
# GNU getopt_long and do not care about replacing getopt.
AC_DEFUN([gl_GETOPT_SUBSTITUTE],
# Check for a POSIX compliant getopt function with GNU extensions (such as
# options with optional arguments) and the functions getopt_long,
# getopt_long_only.
AC_DEFUN([gl_FUNC_GETOPT_GNU],
[
AC_LIBOBJ([getopt])
AC_LIBOBJ([getopt1])
gl_GETOPT_SUBSTITUTE_HEADER
])
AC_REQUIRE([gl_GETOPT_CHECK_HEADERS])
AC_DEFUN([gl_GETOPT_SUBSTITUTE_HEADER],
[
GETOPT_H=getopt.h
AC_DEFINE([__GETOPT_PREFIX], [[rpl_]],
[Define to rpl_ if the getopt replacement functions and variables
should be used.])
AC_SUBST([GETOPT_H])
if test -n "$gl_replace_getopt"; then
gl_GETOPT_SUBSTITUTE
fi
])
AC_DEFUN([gl_GETOPT_CHECK_HEADERS],
[
if test -z "$GETOPT_H"; then
AC_CHECK_HEADERS([getopt.h], [], [GETOPT_H=getopt.h])
gl_replace_getopt=
if test -z "$gl_replace_getopt"; then
AC_CHECK_HEADERS([getopt.h], [], [gl_replace_getopt=yes])
fi
if test -z "$GETOPT_H"; then
AC_CHECK_FUNCS([getopt_long], [], [GETOPT_H=getopt.h])
if test -z "$gl_replace_getopt"; then
AC_CHECK_FUNCS([getopt_long], [], [gl_replace_getopt=yes])
fi
dnl BSD getopt_long uses a way to reset option processing, that is different
dnl from GNU and Solaris (which copied the GNU behavior). We support both
dnl GNU and BSD style resetting of getopt_long(), so there's no need to use
dnl GNU getopt_long() on BSD due to different resetting style.
dnl
dnl With getopt_long(), some BSD versions have a bug in handling optional
dnl arguments. This bug appears only if the environment variable
dnl POSIXLY_CORRECT has been set, so it shouldn't be too bad in most
dnl cases; probably most don't have that variable set. But if we actually
dnl hit this bug, it is a real problem due to our heavy use of optional
dnl arguments.
dnl
dnl According to CVS logs, the bug was introduced in OpenBSD in 2003-09-22
dnl and copied to FreeBSD in 2004-02-24. It was fixed in both in 2006-09-22,
dnl so the affected versions shouldn't be popular anymore anyway. NetBSD
dnl never had this bug. TODO: What about Darwin and others?
if test -z "$GETOPT_H"; then
if test -z "$gl_replace_getopt"; then
AC_CHECK_DECL([optreset],
[AC_DEFINE([HAVE_OPTRESET], 1,
[Define to 1 if getopt.h declares extern int optreset.])],
[], [#include <getopt.h>])
fi
dnl Solaris 10 getopt doesn't handle `+' as a leading character in an
dnl option string (as of 2005-05-05). We don't use that feature, so this
dnl POSIX 2008 does not specify leading '+' behavior, but see
dnl http://austingroupbugs.net/view.php?id=191 for a recommendation on
dnl the next version of POSIX. We don't use that feature, so this
dnl is not a problem for us. Thus, the respective test was removed here.
dnl Checks for getopt handling '-' as a leading character in an option
dnl string were removed, since we also don't use that feature.
])
AC_DEFUN([gl_GETOPT_IFELSE],
AC_DEFUN([gl_GETOPT_SUBSTITUTE],
[
AC_REQUIRE([gl_GETOPT_CHECK_HEADERS])
AS_IF([test -n "$GETOPT_H"], [$1], [$2])
AC_LIBOBJ([getopt])
AC_LIBOBJ([getopt1])
AC_CHECK_HEADERS_ONCE([sys/cdefs.h])
AC_DEFINE([__GETOPT_PREFIX], [[rpl_]],
[Define to rpl_ if the getopt replacement functions and variables
should be used.])
GETOPT_H=getopt.h
AC_SUBST([GETOPT_H])
])
AC_DEFUN([gl_GETOPT], [gl_GETOPT_IFELSE([gl_GETOPT_SUBSTITUTE])])
AC_DEFUN([gl_GETOPT], [gl_FUNC_GETOPT_GNU])

View File

@ -1,3 +1,5 @@
dnl SPDX-License-Identifier: FSFULLR
# Find a POSIX-conforming shell.
# Copyright (C) 2007-2008 Free Software Foundation, Inc.

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# SYNOPSIS
#
@ -7,13 +10,11 @@
#
# Common checks for tuklib.
#
# COPYING
#############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
#############################################################################
AC_DEFUN_ONCE([TUKLIB_COMMON], [
AC_REQUIRE([AC_CANONICAL_HOST])

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# SYNOPSIS
#
@ -17,13 +20,11 @@
# GetSystemInfo() is used on Cygwin)
# - pstat_getdynamic(): HP-UX
#
# COPYING
#############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
#############################################################################
AC_DEFUN_ONCE([TUKLIB_CPUCORES], [
AC_REQUIRE([TUKLIB_COMMON])

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# SYNOPSIS
#
@ -8,16 +11,14 @@
# Checks for tuklib_integer.h:
# - Endianness
# - Does the compiler or the operating system provide byte swapping macros
# - Does the hardware support fast unaligned access to 16-bit
# and 32-bit integers
# - Does the hardware support fast unaligned access to 16-bit, 32-bit,
# and 64-bit integers
#
# COPYING
#############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
#############################################################################
AC_DEFUN_ONCE([TUKLIB_INTEGER], [
AC_REQUIRE([TUKLIB_COMMON])
@ -65,22 +66,42 @@ AC_MSG_CHECKING([if unaligned memory access should be used])
AC_ARG_ENABLE([unaligned-access], AS_HELP_STRING([--enable-unaligned-access],
[Enable if the system supports *fast* unaligned memory access
with 16-bit, 32-bit, and 64-bit integers. By default,
this is enabled only on x86, x86_64, big endian PowerPC,
and some ARM systems.]),
this is enabled on x86, x86-64,
32/64-bit big endian PowerPC,
64-bit little endian PowerPC,
and some ARM, ARM64, and RISC-V systems.]),
[], [enable_unaligned_access=auto])
if test "x$enable_unaligned_access" = xauto ; then
# TODO: There may be other architectures, on which unaligned access
# is OK.
# NOTE: There might be other architectures on which unaligned access
# is fast.
case $host_cpu in
i?86|x86_64|powerpc|powerpc64)
i?86|x86_64|powerpc|powerpc64|powerpc64le)
enable_unaligned_access=yes
;;
arm*|aarch64*)
arm*|aarch64*|riscv*)
# On 32-bit and 64-bit ARM, GCC and Clang
# #define __ARM_FEATURE_UNALIGNED if
# unaligned access is supported.
#
# Exception: GCC at least up to 13.2.0
# defines it even when using -mstrict-align
# so in that case this autodetection goes wrong.
# Most of the time -mstrict-align isn't used so it
# shouldn't be a common problem in practice. See:
# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111555
#
# RISC-V C API Specification says that if
# __riscv_misaligned_fast is defined then
# unaligned access is known to be fast.
#
# MSVC is handled as a special case: We assume that
# 32/64-bit ARM supports fast unaligned access.
# If MSVC gets RISC-V support then this will assume
# fast unaligned access on RISC-V too.
AC_COMPILE_IFELSE([AC_LANG_SOURCE([
#ifndef __ARM_FEATURE_UNALIGNED
#if !defined(__ARM_FEATURE_UNALIGNED) \
&& !defined(__riscv_misaligned_fast) \
&& !defined(_MSC_VER)
compile error
#endif
int main(void) { return 0; }

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# SYNOPSIS
#
@ -15,13 +18,11 @@
# functions, but each function is put into a separate .c file so
# that it is possible to pick only what is strictly needed.
#
# COPYING
#############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
#############################################################################
AC_DEFUN_ONCE([TUKLIB_MBSTR], [
AC_REQUIRE([TUKLIB_COMMON])

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# SYNOPSIS
#
@ -29,13 +32,11 @@
# - sysinfo() works on Linux/dietlibc and probably on other Linux
# systems whose libc may lack sysconf().
#
# COPYING
#############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
#############################################################################
AC_DEFUN_ONCE([TUKLIB_PHYSMEM], [
AC_REQUIRE([TUKLIB_COMMON])

View File

@ -1,3 +1,6 @@
# SPDX-License-Identifier: 0BSD
#############################################################################
#
# SYNOPSIS
#
@ -11,15 +14,16 @@
# This .m4 file is needed allow this module to use glibc's
# program_invocation_name.
#
# COPYING
#############################################################################
#
# Author: Lasse Collin
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
# Author: Lasse Collin
#
#############################################################################
AC_DEFUN_ONCE([TUKLIB_PROGNAME], [
AC_REQUIRE([TUKLIB_COMMON])
AC_CHECK_DECLS([program_invocation_name], [], [], [#include <errno.h>])
AC_CHECK_DECL([program_invocation_name], [AC_DEFINE(
[HAVE_PROGRAM_INVOCATION_NAME], [1],
[Define to 1 if 'program_invocation_name' is declared in <errno.h>.])],
[], [#include <errno.h>])
])dnl

View File

@ -1,5 +1,7 @@
# visibility.m4 serial 6
dnl Copyright (C) 2005, 2008, 2010-2020 Free Software Foundation, Inc.
dnl SPDX-License-Identifier: FSFULLR
# visibility.m4 serial 8
dnl Copyright (C) 2005, 2008, 2010-2023 Free Software Foundation, Inc.
dnl This file is free software; the Free Software Foundation
dnl gives unlimited permission to copy and/or distribute it,
dnl with or without modifications, as long as this notice is preserved.
@ -58,6 +60,11 @@ AC_DEFUN([gl_VISIBILITY],
extern __attribute__((__visibility__("default"))) int exportedvar;
extern __attribute__((__visibility__("hidden"))) int hiddenfunc (void);
extern __attribute__((__visibility__("default"))) int exportedfunc (void);
void dummyfunc (void);
int hiddenvar;
int exportedvar;
int hiddenfunc (void) { return 51; }
int exportedfunc (void) { return 1225736919; }
void dummyfunc (void) {}
]],
[[]])],

View File

@ -1,113 +0,0 @@
#!/bin/sh
###############################################################################
# Author: Anders F Björklund <afb@users.sourceforge.net>
#
# This file has been put into the public domain.
# You can do whatever you want with this file.
###############################################################################
mkdir -p Root
mkdir -p Resources
# Abort immediately if something goes wrong.
set -e
GCC="gcc-4.2"
SDK="/Developer/SDKs/MacOSX10.5.sdk"
MDT="10.5"
GTT=i686-apple-darwin9
ARCHES1="-arch ppc -arch ppc64 -arch i386 -arch x86_64"
ARCHES2="-arch ppc -arch i386"
PKGFORMAT="10.5" # xar
# avoid "unknown required load command: 0x80000022" from linking on Snow Leopard
uname -r | grep ^1 >/dev/null && LDFLAGS="$LDFLAGS -Wl,-no_compact_linkedit"
# Clean up if it was already configured.
[ -f Makefile ] && make distclean
# Build the regular fat program
CC="$GCC" \
CFLAGS="-O2 -g $ARCHES1 -isysroot $SDK -mmacosx-version-min=$MDT" \
../configure --disable-dependency-tracking --disable-xzdec --disable-lzmadec $GTT
make
make check
make DESTDIR=`pwd`/Root install
make distclean
# Build the size-optimized program
CC="$GCC" \
CFLAGS="-Os -g $ARCHES2 -isysroot $SDK -mmacosx-version-min=$MDT" \
../configure --disable-dependency-tracking --disable-shared --disable-nls --disable-encoders --enable-small --disable-threads $GTT
make -C src/liblzma
make -C src/xzdec
make -C src/xzdec DESTDIR=`pwd`/Root install
cp -a ../extra Root/usr/local/share/doc/xz
make distclean
# Move development files to different package
test -d liblzma && rm -r liblzma
mkdir -p liblzma/usr/local
mv Root/usr/local/include liblzma/usr/local
mv Root/usr/local/lib liblzma/usr/local
mkdir -p Root/usr/local/lib
cp -p liblzma/usr/local/lib/liblzma.5.dylib Root/usr/local/lib
mkdir -p liblzma/usr/local/share/doc/xz
mv Root/usr/local/share/doc/xz/examples* liblzma/usr/local/share/doc/xz
# Strip debugging symbols and make relocatable
for bin in xz lzmainfo xzdec lzmadec; do
strip -S Root/usr/local/bin/$bin
install_name_tool -change /usr/local/lib/liblzma.5.dylib @executable_path/../lib/liblzma.5.dylib Root/usr/local/bin/$bin
done
for lib in liblzma.5.dylib; do
strip -S Root/usr/local/lib/$lib
install_name_tool -id @executable_path/../lib/liblzma.5.dylib Root/usr/local/lib/$lib
done
# Create tarball, but without the HFS+ attrib
rmdir debug lib po src/liblzma/api src/liblzma src/lzmainfo src/scripts src/xz src/xzdec src tests
( cd Root/usr/local; COPY_EXTENDED_ATTRIBUTES_DISABLE=true COPYFILE_DISABLE=true tar cvjf ../../../XZ.tbz * )
( cd liblzma; COPY_EXTENDED_ATTRIBUTES_DISABLE=true COPYFILE_DISABLE=true tar cvjf ../liblzma.tbz ./usr/local )
# Include documentation files for package
cp -p ../README Resources/ReadMe.txt
cp -p ../COPYING Resources/License.txt
# Make an Installer.app package
ID="org.tukaani.xz"
VERSION=`cd ..; sh build-aux/version.sh`
PACKAGEMAKER=/Developer/Applications/Utilities/PackageMaker.app/Contents/MacOS/PackageMaker
$PACKAGEMAKER -r Root/usr/local -l /usr/local -e Resources -i $ID -n $VERSION -t XZ -o XZ.pkg -g $PKGFORMAT --verbose
$PACKAGEMAKER -r liblzma -w -k -i $ID.liblzma -n $VERSION -o liblzma.pkg -g $PKGFORMAT --verbose
# Put the package in a disk image
if [ "$PKGFORMAT" != "10.5" ]; then
hdiutil create -fs HFS+ -format UDZO -quiet -srcfolder XZ.pkg -ov XZ.dmg
hdiutil internet-enable -yes -quiet XZ.dmg
fi
echo
echo "Build completed successfully."
echo

View File

@ -1,4 +1,10 @@
# SPDX-License-Identifier: FSFUL
# Makefile variables for PO directory in any package using GNU gettext.
#
# Copyright (C) 2003-2019 Free Software Foundation, Inc.
# This file is free software; the Free Software Foundation gives
# unlimited permission to use, copy, distribute, and modify it.
# Usually the message domain is the same as the package name.
DOMAIN = $(PACKAGE)
@ -8,7 +14,7 @@ subdir = po
top_builddir = ..
# These options get passed to xgettext.
XGETTEXT_OPTIONS = --keyword=_ --keyword=N_
XGETTEXT_OPTIONS = --keyword=_ --keyword=N_ --no-wrap --package-name='XZ Utils'
# This is the copyright holder that gets inserted into the header of the
# $(DOMAIN).pot file. Set this to the copyright holder of the surrounding
@ -18,7 +24,14 @@ XGETTEXT_OPTIONS = --keyword=_ --keyword=N_
# or entity, or to disclaim their copyright. The empty string stands for
# the public domain; in this case the translators are expected to disclaim
# their copyright.
COPYRIGHT_HOLDER =
COPYRIGHT_HOLDER = The XZ Utils authors and contributors
# This tells whether or not to prepend "GNU " prefix to the package
# name that gets inserted into the header of the $(DOMAIN).pot file.
# Possible values are "yes", "no", or empty. If it is empty, try to
# detect it automatically by scanning the files in $(top_srcdir) for
# "GNU packagename" string.
PACKAGE_GNU = no
# This is the email address or URL to which the translators shall report
# bugs in the untranslated strings:
@ -40,7 +53,35 @@ MSGID_BUGS_ADDRESS =
# message catalogs shall be used. It is usually empty.
EXTRA_LOCALE_CATEGORIES =
# Although you may need slightly wider terminal than 80 chars, it is
# much nicer to edit the output of --help when this is set.
XGETTEXT_OPTIONS += --no-wrap
MSGMERGE += --no-wrap
# This tells whether the $(DOMAIN).pot file contains messages with an 'msgctxt'
# context. Possible values are "yes" and "no". Set this to yes if the
# package uses functions taking also a message context, like pgettext(), or
# if in $(XGETTEXT_OPTIONS) you define keywords with a context argument.
USE_MSGCTXT = no
# These options get passed to msgmerge.
# Useful options are in particular:
# --previous to keep previous msgids of translated messages,
# --quiet to reduce the verbosity.
MSGMERGE_OPTIONS = --no-wrap
# These options get passed to msginit.
# If you want to disable line wrapping when writing PO files, add
# --no-wrap to MSGMERGE_OPTIONS, XGETTEXT_OPTIONS, and
# MSGINIT_OPTIONS.
#
# Although one may need slightly wider terminal than 80 chars, it is
# much nicer to edit the output of --help when --no-wrap is set.
MSGINIT_OPTIONS = --no-wrap
# This tells whether or not to regenerate a PO file when $(DOMAIN).pot
# has changed. Possible values are "yes" and "no". Set this to no if
# the POT file is checked in the repository and the version control
# program ignores timestamps.
PO_DEPENDS_ON_POT = yes
# This tells whether or not to forcibly update $(DOMAIN).pot and
# regenerate PO files on "make dist". Possible values are "yes" and
# "no". Set this to no if the POT file and PO files are maintained
# externally.
DIST_DEPENDS_ON_UPDATE_PO = yes

View File

@ -1,3 +1,5 @@
# SPDX-License-Identifier: 0BSD
# List of source files which contain translatable strings.
src/xz/args.c
src/xz/coder.c
@ -6,8 +8,10 @@ src/xz/hardware.c
src/xz/list.c
src/xz/main.c
src/xz/message.c
src/xz/mytime.c
src/xz/options.c
src/xz/signals.c
src/xz/suffix.c
src/xz/util.c
src/lzmainfo/lzmainfo.c
src/common/tuklib_exit.c

537
po/de.po

File diff suppressed because it is too large Load Diff

844
po/eo.po

File diff suppressed because it is too large Load Diff

592
po/es.po

File diff suppressed because it is too large Load Diff

632
po/fr.po

File diff suppressed because it is too large Load Diff

722
po/hr.po

File diff suppressed because it is too large Load Diff

558
po/hu.po

File diff suppressed because it is too large Load Diff

671
po/ko.po

File diff suppressed because it is too large Load Diff

562
po/pl.po

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

607
po/ro.po

File diff suppressed because it is too large Load Diff

578
po/sv.po

File diff suppressed because it is too large Load Diff

591
po/uk.po

File diff suppressed because it is too large Load Diff

842
po/vi.po

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More