Thoughts on using Zstd compression instead of lzip

  • Done
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • romulasry
Owner
unassigned
Submitted by
romulasry
Severity
normal
R
R
romulasry wrote on 10 Jun 2020 21:22
(name . bug-guix@gnu.org)(address . bug-guix@gnu.org)
mQL02MBzUI0Or1OBC-6OI05eEgYrLxIalc2JKMZWQg6ysB93UfrP-rfzmFOKYvauVJ1lLmyGHVBz3drFUKRR6tSgTaaBP58Is7f0rX98UAE=@protonmail.com

Zstandard is a real-time compression algorithm, providing high compression ratios. It offers a very wide range of compression / speed trade-off, while being backed by a very fast decoder (see [benchmarks](http://facebook.github.io/zstd/#benchmarks)below). It also offers a special mode for small data, called [dictionary compression](http://facebook.github.io/zstd/#small-data), and can create dictionaries from any sample set. Zstandard library is provided as open source software using a BSD license.

Sent with [ProtonMail](https://protonmail.com) Secure Email.
Attachment: file
T
T
Tobias Geerinckx-Rice wrote on 10 Jun 2020 22:59
(name . romulasry)(address . romulasry@protonmail.com)
87eeqmr60v.fsf@nckx
Romulas,

It's not clear where you want to see zstd used. Here's a trivial
patch to add Zstd to ‘guix pack’, if that's what you mean.

I didn't submit it in the past because I dread bikeshedding the
compression level. -3 is the default, and compresses slightly
better than gzip in a fraction of the time.

Kind regards,

T G-R
From fa2ead6ac0f6c005e6392d6073122c7191e08944 Mon Sep 17 00:00:00 2001
From: Tobias Geerinckx-Rice <me@tobias.gr>
Date: Fri, 08 Nov 2019 09:03:27 +0100
Subject: [PATCH] pack: Add zstd compressor.

* guix/scripts/pack.scm (%compressors): Add ztsd.
---
guix/scripts/pack.scm | 3 +++
1 file changed, 3 insertions(+)

Toggle diff (23 lines)
diff --git a/guix/scripts/pack.scm b/guix/scripts/pack.scm
index 518bf6e7e3..5e71ca9c21 100644
--- a/guix/scripts/pack.scm
+++ b/guix/scripts/pack.scm
@@ -4,6 +4,7 @@
;;; Copyright © 2018 Konrad Hinsen <konrad.hinsen@fastmail.net>
;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com>
;;; Copyright © 2018 Efraim Flashner <efraim@flashner.co.il>
+;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -81,6 +82,8 @@
#~(#+(file-append xz "/bin/xz") "-e"))
(compressor "bzip2" ".bz2"
#~(#+(file-append bzip2 "/bin/bzip2") "-9"))
+ (compressor "zstd" ".zst"
+ #~(#+(file-append zstd "/bin/zstd") "-3"))
(compressor "none" "" #f)))
;; This one is only for use in this module, so don't put it in %compressors.
--
2.26.2
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXuFJwAAKCRANsP+IT1Vt
eYoBAQD+Xz1vLVyGgIhRPL5EDcK/Dc87Ag8CLMfXoSuNXDXILwD/fAXb2MzWkYb/
FX3NMHpUGl9jklzKOelhgtcLQwDzOQ4=
=Ymr5
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 11 Jun 2020 23:21
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
874krh9u4c.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (3 lines)
> It's not clear where you want to see zstd used. Here's a trivial
> patch to add Zstd to ‘guix pack’, if that's what you mean.

I suppose that was for substitutes, in reference to:


The idea is worth discussing, but IMO, romulasry, discussions should
happen on guix-devel rather than guix-patches… unless the discussion
comes with a patch, of course. :-)

Closing!

Ludo’.
L
L
Ludovic Courtès wrote on 11 Jun 2020 23:21
control message for bug #41794
(address . control@debbugs.gnu.org)
8736719u41.fsf@gnu.org
tags 41794 notabug
close 41794
quit
L
L
Ludovic Courtès wrote on 13 Jun 2020 22:30
Re: bug#41794: Thoughts on using Zstd compression instead of lzip
(name . Tobias Geerinckx-Rice via Bug reports for GNU Guix)(address . bug-guix@gnu.org)
875zbuyaho.fsf@gnu.org
Hi,

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org>
skribis:

Toggle quote (7 lines)
> From fa2ead6ac0f6c005e6392d6073122c7191e08944 Mon Sep 17 00:00:00 2001
> From: Tobias Geerinckx-Rice <me@tobias.gr>
> Date: Fri, 08 Nov 2019 09:03:27 +0100
> Subject: [PATCH] pack: Add zstd compressor.
>
> * guix/scripts/pack.scm (%compressors): Add ztsd.

BTW, feel free to push it after mentioning zstd for -C under “Invoking
guix pack”.

Thanks,
Ludo’.
?
Your comment

This issue is archived.

To comment on this conversation send an email to 41794@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 41794
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch