[PATCH 0/2] self: Apply grafts to the outputs of the guix derivation.

Open

Details

3 participants

Ludovic Courtès
Christopher Baines
Simon Tournier

Owner: unassigned

Submitted by: Christopher Baines

Severity: normal

Christopher Baines wrote on 8 Feb 2023 08:46

Recipients:(address . guix-patches@gnu.org)

Message-ID:871qn08uo7.fsf@cbaines.net

These patches mean that grafts apply to the outputs of the guix

derivation, rather than having grafts apply to the derivation

itself. This moves grafting here to work like grafting for packages,

where you can think of the grafted outputs as a transformed variant of

the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the

derivations without grafts, and for these to be useful for substitutes

regardless of whether users are using grafts.

Christopher Baines (2):

packages: Add explicit-grafting record type to assist with grafts.

self: Apply grafts to the outputs of the guix derivation.

build-aux/build-self.scm | 4 ++-

guix/packages.scm | 45 +++++++++++++++++++++++++++-

guix/self.scm | 65 ++++++++++++++++++++++++++--------------

3 files changed, 89 insertions(+), 25 deletions(-)

2.38.1

Christopher Baines wrote on 8 Feb 2023 08:54

[PATCH 1/2] packages: Add explicit-grafting record type to assist with grafts.

Recipients:(address . 61363@debbugs.gnu.org)

Message-ID:20230208075403.11788-1-mail@cbaines.net

Normally the grafting takes place when lowering packages, but this record

assists with applying the same transformation to arbitrary objects/store

items.

I'm adding this to allow grafting the channel instance derivation outputs.

* guix/packages.scm (explicit-grafting, explicit-grafting?,

explicit-grafting-obj, explicit-grafting-grafts): New procedures.

---

guix/packages.scm | 45 ++++++++++++++++++++++++++++++++++++++++++++-

1 file changed, 44 insertions(+), 1 deletion(-)

Toggle diff (62 lines)diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..877bf89522 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -188,7 +188,12 @@ (define-module (guix packages)
             package-file
             package->derivation
             package->cross-derivation
-            origin->derivation))
+            origin->derivation
+
+            explicit-grafting
+            explicit-grafting?
+            explicit-grafting-obj
+            explicit-grafting-grafts))
 
 ;; The 'source-module-closure' procedure ca. 1.2.0 did not recognize
 ;; #:re-export-and-replace: <https://issues.guix.gnu.org/52694>.
@@ -2093,3 +2098,41 @@ (define package-source-derivation                 ;somewhat deprecated
          (add-to-store store (basename file) #t "sha256" file))
         (_
          (lower store source system))))))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+  (%explicit-grafting obj packages)
+  explicit-grafting?
+  (obj      explicit-grafting-obj)       ;obj
+  (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+  (match rec
+    (($ <explicit-grafting> obj packages)
+     (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+  (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+                                                  system target)
+  (match explicit-grafting
+    (($ <explicit-grafting> obj packages)
+     (mlet* %store-monad ((drv (without-grafting
+                                (lower-object obj system #:target target)))
+                          (grafts
+                           (mapm %store-monad
+                                 (lambda (pkg)
+                                   (package-grafts* pkg system #:target target))
+                                 packages)))
+       (match (delete-duplicates
+               (concatenate grafts))
+         (()
+          (return drv))
+         (grafts
+          (mlet %store-monad ((guile (package->derivation
+                                      (guile-for-grafts)
+                                      system #:graft? #f)))
+            (graft-derivation* drv grafts
+                               #:system system
+                               #:guile guile))))))))
-- 
2.38.1

Christopher Baines wrote on 8 Feb 2023 08:54

[PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.

Recipients:(address . 61363@debbugs.gnu.org)

Message-ID:20230208075403.11788-2-mail@cbaines.net

Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.

* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
 #:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
---
 build-aux/build-self.scm |  4 ++-
 guix/self.scm            | 65 ++++++++++++++++++++++++++--------------
 2 files changed, 45 insertions(+), 24 deletions(-)

Toggle diff (123 lines)diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
                                                   #:channel-metadata
                                                   '#$channel-metadata
                                                   #:pull-version
-                                                  #$pull-version)
+                                                  #$pull-version
+                                                  #:graft?
+                                                  #$(%graft?))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index 93019e1c64..c944dbe9ce 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
                         (gzip (specification->package "gzip"))
                         (bzip2 (specification->package "bzip2"))
                         (xz (specification->package "xz"))
-                        (guix (specification->package "guix")))
+                        (guix (specification->package "guix"))
+                        (graft? #t))
   "Return a file-like object that contains a compiled Guix."
   (define guile-avahi
     (specification->package "guile-avahi"))
@@ -802,6 +803,12 @@ (define dependencies
                       guile-json guile-semver guile-ssh guile-sqlite3
                       guile-lib guile-zlib guile-lzlib guile-zstd)))
 
+  (define packages
+    (cons* gzip
+           bzip2
+           xz
+           dependencies))
+
   (define *core-modules*
     (scheme-node "guix-core"
                  '((guix)
@@ -1022,28 +1029,35 @@ (define (built-modules node-subset)
                                                guile-lzma
                                                dependencies)
                                         #:guile guile-for-build
-                                        #:guile-version guile-version)))
-           (whole-package name modules dependencies
-                          #:command command
-                          #:guile guile-for-build
-
-                          ;; Include 'guix-daemon'.  XXX: Here we inject an
-                          ;; older snapshot of guix-daemon, but that's a good
-                          ;; enough approximation for now.
-                          #:daemon (module-ref (resolve-interface
-                                                '(gnu packages
-                                                      package-management))
-                                               'guix-daemon)
-
-                          #:info (info-manual source)
-                          #:miscellany (miscellaneous-files source)
-                          #:guile-version guile-version)))
+                                        #:guile-version guile-version))
+                (obj
+                 (whole-package name modules dependencies
+                                #:command command
+                                #:guile guile-for-build
+
+                                ;; Include 'guix-daemon'.  XXX: Here we inject
+                                ;; an older snapshot of guix-daemon, but
+                                ;; that's a good enough approximation for now.
+                                #:daemon (module-ref (resolve-interface
+                                                      '(gnu packages
+                                                            package-management))
+                                                     'guix-daemon)
+
+                                #:info (info-manual source)
+                                #:miscellany (miscellaneous-files source)
+                                #:guile-version guile-version)))
+           (if graft?
+               (explicit-grafting obj packages)
+               obj)))
         ((= 0 pull-version)
          ;; Legacy 'guix pull': return the .scm and .go files as one
          ;; directory.
-         (built-modules (lambda (node)
-                          (list (node-source node)
-                                (node-compiled node)))))
+         (let ((obj (built-modules (lambda (node)
+                                     (list (node-source node)
+                                           (node-compiled node))))))
+           (if graft?
+               (explicit-grafting obj packages)
+               obj)))
         (else
          ;; Unsupported 'guix pull' version.
          #f)))
@@ -1273,7 +1287,8 @@ (define (process-directory directory files output)
 (define* (guix-derivation source version
                           #:optional (guile-version (effective-version))
                           #:key (pull-version 0)
-                          channel-metadata)
+                          channel-metadata
+                          (graft? #t))
   "Return, as a monadic value, the derivation to build the Guix from SOURCE
 for GUILE-VERSION.  Use VERSION as the version string.  Use CHANNEL-METADATA
 as the channel metadata sexp to include in (guix config).
@@ -1310,7 +1325,11 @@ (define guile
                                #:pull-version pull-version
                                #:guile-version (if (>= pull-version 1)
                                                    "3.0" guile-version)
-                               #:guile-for-build guile)))
+                               #:guile-for-build guile
+                               #:graft? graft?)))
       (if guix
-          (lower-object guix)
+          (if graft?
+              (lower-object guix)
+              (without-grafting
+               (lower-object guix)))
           (return #f)))))
-- 
2.38.1

Christopher Baines wrote on 10 Feb 2023 10:16

Re: [bug#61363] [PATCH 0/2] self: Apply grafts to the outputs of the guix derivation.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 61363@debbugs.gnu.org)

Message-ID:87h6vt6foi.fsf@cbaines.net

The data service comparison is now available for this, and while there

are no differences in the packages, you can see some information.

This is the channel instances before:

https://data.qa.guix.gnu.org/revision/a582d863465990642d331bc05bf073f47fb80908/channel-instances

and this is after:

https://data.qa.guix.gnu.org/revision/9cfbb22b556d28a0af345824ae5b3e00eb3f4a15/channel-instances

Ludovic Courtès wrote on 22 Feb 2023 10:16

Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 61363@debbugs.gnu.org)

Message-ID:87sfey9i1t.fsf@gnu.org

Hi,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (4 lines)

> Rather than having grafts apply to the derivation itself. This moves grafting

> here to work like grafting for packages, where you can think of the grafted

> outputs as a transformed variant of the ungrafted outputs.

Hmm.

Toggle quote (4 lines)

> I'm looking at this as it'll allow the Guix Data Service to compute the

> derivations without grafts, and for these to be useful for substitutes

> regardless of whether users are using grafts.

How does it help exactly? By disabling grafts in that context?

Toggle quote (22 lines)> +++ b/guix/self.scm
> @@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
>                          (gzip (specification->package "gzip"))
>                          (bzip2 (specification->package "bzip2"))
>                          (xz (specification->package "xz"))
> -                        (guix (specification->package "guix")))
> +                        (guix (specification->package "guix"))
> +                        (graft? #t))
>    "Return a file-like object that contains a compiled Guix."
>    (define guile-avahi
>      (specification->package "guile-avahi"))
> @@ -802,6 +803,12 @@ (define dependencies
>                        guile-json guile-semver guile-ssh guile-sqlite3
>                        guile-lib guile-zlib guile-lzlib guile-zstd)))
>  
> +  (define packages
> +    (cons* gzip
> +           bzip2
> +           xz
> +           dependencies))
> +

[...]

Toggle quote (7 lines)

> + (let ((obj (built-modules (lambda (node)

> + (list (node-source node)

> + (node-compiled node))))))

> + (if graft?

> + (explicit-grafting obj packages)

> + obj)))

There are two things I’m not comfortable with:

1. Having <explicit-grafting> in (guix packages); it looks misplaced.

2. More importantly, manually listing packages that might require

grafting looks like a slippery slope (“oops! we’re not getting the

GnuTLS graft for that CVE, too bad”).

I designed and implemented several variants to try and delay grafting.

One of them consisted in carrying graft information in gexps:

https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts

It’s kinda similar to what you’re proposing in that graft information is

carried as far as possible. The main difference is that it’s automated.

Hmm needs more thought.

Ludo’.

Christopher Baines wrote on 22 Feb 2023 12:17

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61363@debbugs.gnu.org)

Message-ID:878rgpeo28.fsf@cbaines.net

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (6 lines)

>> I'm looking at this as it'll allow the Guix Data Service to compute the

>> derivations without grafts, and for these to be useful for substitutes

>> regardless of whether users are using grafts.

> How does it help exactly? By disabling grafts in that context?

So the Guix Data Service is somewhat built on the assumption that it's

cheap to compute derivations, at least with grafts disabled. That's

always been the case for packages, but for channel instance derivations

it's not reliably the case, since currently disabling grafts doesn't

apply to the whole process, and even if it did, the derivations you'd

get out wouldn't be that useful (since you can't transform the outputs

from those derivations to the outputs you'd get if using grafts).

With these changes, it's always relatively cheap to compute the channel

instance derivations, and it's always possible to compute the

derivations for any system without needing to be able to perform builds

for that system.

You can see this in how the data service has processed Guix before and

after these patches.

This is the channel instances before:

https://data.qa.guix.gnu.org/revision/a582d863465990642d331bc05bf073f47fb80908/channel-instances

and this is after:

https://data.qa.guix.gnu.org/revision/9cfbb22b556d28a0af345824ae5b3e00eb3f4a15/channel-instances

Given data.qa.guix.gnu.org is running on an x86_64-linux system, that

and i686-linux isn't generally a problem, but I'm guessing it only

managed to compute the powerpc64le-linux and aarch64-linux derivations

because it was able to substitute the necessary store items. For other

system computing the derivations would have failed.

I believe this change will also mean that the build farms will go from

performing the grafting for these builds, to being able to not do so, in

line with how builds for packages are handled. This isn't a big thing,

but I think it makes sense.

Toggle quote (35 lines)>> +++ b/guix/self.scm
>> @@ -752,7 +752,8 @@ (define* (compiled-guix source #:key
>>                          (gzip (specification->package "gzip"))
>>                          (bzip2 (specification->package "bzip2"))
>>                          (xz (specification->package "xz"))
>> -                        (guix (specification->package "guix")))
>> +                        (guix (specification->package "guix"))
>> +                        (graft? #t))
>>    "Return a file-like object that contains a compiled Guix."
>>    (define guile-avahi
>>      (specification->package "guile-avahi"))
>> @@ -802,6 +803,12 @@ (define dependencies
>>                        guile-json guile-semver guile-ssh guile-sqlite3
>>                        guile-lib guile-zlib guile-lzlib guile-zstd)))
>>  
>> +  (define packages
>> +    (cons* gzip
>> +           bzip2
>> +           xz
>> +           dependencies))
>> +
>
> [...]
>
>> +         (let ((obj (built-modules (lambda (node)
>> +                                     (list (node-source node)
>> +                                           (node-compiled node))))))
>> +           (if graft?
>> +               (explicit-grafting obj packages)
>> +               obj)))
>
> There are two things I’m not comfortable with:
>
>   1. Having <explicit-grafting> in (guix packages); it looks misplaced.

I didn't put it there at first, but I think it makes sense since

grafting is currently specific to packages, as is this additional code.

Toggle quote (12 lines)>   2. More importantly, manually listing packages that might require
>      grafting looks like a slippery slope (“oops! we’re not getting the
>      GnuTLS graft for that CVE, too bad”).
>
> I designed and implemented several variants to try and delay grafting.
> One of them consisted in carrying graft information in gexps:
>
>   https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts
>
> It’s kinda similar to what you’re proposing in that graft information is
> carried as far as possible.  The main difference is that it’s automated.

That's interesting, I think that making grafting not specific to
packages, and something where the replacement is handled at a lower
level (e.g. gexps) would be an alternative way to handle this.

Given that this approach works though, maybe the explicit-grafting
functionality could just sit and be used inside of (guix self). Given
that module is very explicit about what packages are used, it should be
possible to arrange the code so it's very hard to miss a package out,
which should address your concern about manually listing packages (maybe
specification->package can be tweaked so that it's possible to get all
the packages, and that can be the list considered for grafting).

I don't know of any other places where this approach would be useful, so
while it would be nice to have a more general grafting mechanism
eventually, I'd also like to be able to make these changes to channel
instance grafts sooner rather than later.

Christopher Baines wrote on 28 Feb 2023 16:47

[PATCH v2 1/3] packages: Export guile-for-grafts.

Recipients:(address . 61363@debbugs.gnu.org)

Message-ID:20230228154703.3952-1-mail@cbaines.net

So this can be used in (guix self).

* guix/packages.scm (guile-for-grafts): Export.

---

guix/packages.scm | 1 +

1 file changed, 1 insertion(+)

Toggle diff (14 lines)diff --git a/guix/packages.scm b/guix/packages.scm
index 041a872f9d..2f81ad0284 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -183,6 +183,7 @@ (define-module (guix packages)
             package-closure
 
             default-guile
+            guile-for-grafts
             default-guile-derivation
             set-guile-for-build
             package-file
-- 
2.39.1

Christopher Baines wrote on 28 Feb 2023 16:47

[PATCH v2 2/3] self: Restructure accessing packages.

Recipients:(address . 61363@debbugs.gnu.org)

Message-ID:20230228154703.3952-2-mail@cbaines.net

Both for consistency (always use specification->package as defined in this
module) and so that all the packages that are used can be accessed (which
comes in useful when applying grafts).

* guix/self.scm (%packages): New variable.
(specification->package): Use %packages.
(locale-data, translate-texi-manuals, info-manual, guix-command,
compiled-guix): Use specification->package.
---
 guix/self.scm | 97 +++++++++++++++++++++++++--------------------------
 1 file changed, 48 insertions(+), 49 deletions(-)

Toggle diff (173 lines)diff --git a/guix/self.scm b/guix/self.scm
index 93019e1c64..c5de3ab8fc 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -44,34 +44,42 @@ (define-module (guix self)
 ;;; Dependency handling.
 ;;;
 
-(define specification->package
+(define %packages
+  (let ((ref (lambda (module variable)
+               (delay
+                 (module-ref (resolve-interface
+                              `(gnu packages ,module))
+                             variable)))))
+    `(("guile"              . ,(ref 'guile 'guile-3.0-latest))
+      ("guile-avahi"        . ,(ref 'guile-xyz 'guile-avahi))
+      ("guile-json"         . ,(ref 'guile 'guile-json-4))
+      ("guile-ssh"          . ,(ref 'ssh   'guile-ssh))
+      ("guile-git"          . ,(ref 'guile 'guile-git))
+      ("guile-semver"       . ,(ref 'guile-xyz 'guile-semver))
+      ("guile-lib"          . ,(ref 'guile-xyz 'guile-lib))
+      ("guile-sqlite3"      . ,(ref 'guile 'guile-sqlite3))
+      ("guile-zlib"         . ,(ref 'guile 'guile-zlib))
+      ("guile-lzlib"        . ,(ref 'guile 'guile-lzlib))
+      ("guile-zstd"         . ,(ref 'guile 'guile-zstd))
+      ("guile-gcrypt"       . ,(ref 'gnupg 'guile-gcrypt))
+      ("guile-gnutls"       . ,(ref 'tls 'guile-gnutls))
+      ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
+      ("disarchive"         . ,(ref 'backup 'disarchive))
+      ("guile-lzma"         . ,(ref 'guile 'guile-lzma))
+      ("gzip"               . ,(ref 'compression 'gzip))
+      ("bzip2"              . ,(ref 'compression 'bzip2))
+      ("xz"                 . ,(ref 'compression 'xz))
+      ("po4a"               . ,(ref 'gettext 'po4a))
+      ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
+      ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
+      ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
+      ("graphviz"           . ,(ref 'graphviz 'graphviz))
+      ("texinfo"            . ,(ref 'texinfo 'texinfo)))))
+
+(define (specification->package name)
   ;; Use our own variant of that procedure because that of (gnu packages)
   ;; would traverse all the .scm files, which is wasteful.
-  (let ((ref (lambda (module variable)
-               (module-ref (resolve-interface module) variable))))
-    (match-lambda
-      ("guile"      (ref '(gnu packages guile) 'guile-3.0-latest))
-      ("guile-avahi" (ref '(gnu packages guile-xyz) 'guile-avahi))
-      ("guile-json" (ref '(gnu packages guile) 'guile-json-4))
-      ("guile-ssh"  (ref '(gnu packages ssh)   'guile-ssh))
-      ("guile-git"  (ref '(gnu packages guile) 'guile-git))
-      ("guile-semver"  (ref '(gnu packages guile-xyz) 'guile-semver))
-      ("guile-lib"  (ref '(gnu packages guile-xyz) 'guile-lib))
-      ("guile-sqlite3" (ref '(gnu packages guile) 'guile-sqlite3))
-      ("guile-zlib" (ref '(gnu packages guile) 'guile-zlib))
-      ("guile-lzlib" (ref '(gnu packages guile) 'guile-lzlib))
-      ("guile-zstd" (ref '(gnu packages guile) 'guile-zstd))
-      ("guile-gcrypt"  (ref '(gnu packages gnupg) 'guile-gcrypt))
-      ("guile-gnutls"  (ref '(gnu packages tls) 'guile-gnutls))
-      ("disarchive" (ref '(gnu packages backup) 'disarchive))
-      ("guile-lzma" (ref '(gnu packages guile) 'guile-lzma))
-      ("gzip"       (ref '(gnu packages compression) 'gzip))
-      ("bzip2"      (ref '(gnu packages compression) 'bzip2))
-      ("xz"         (ref '(gnu packages compression) 'xz))
-      ("po4a"       (ref '(gnu packages gettext) 'po4a))
-      ("gettext"       (ref '(gnu packages gettext) 'gettext-minimal))
-      ("gcc-toolchain" (ref '(gnu packages commencement) 'gcc-toolchain))
-      (_            #f))))                        ;no such package
+  (and=> (assoc-ref %packages name) force))
 
 
 ;;;
@@ -240,9 +248,8 @@ (define* (locale-data source domain
                       #:optional (directory domain))
   "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
 DOMAIN, a gettext domain."
-  (define gettext
-    (module-ref (resolve-interface '(gnu packages gettext))
-                'gettext-minimal))
+  (define gettext-minimal
+    (specification->package "gettext-minimal"))
 
   (define build
     (with-imported-modules '((guix build utils))
@@ -258,7 +265,7 @@ (define (compile language)
             (let ((gmo (string-append #$output "/" language "/LC_MESSAGES/"
                                       #$domain ".mo")))
               (mkdir-p (dirname gmo))
-              (invoke #+(file-append gettext "/bin/msgfmt")
+              (invoke #+(file-append gettext-minimal "/bin/msgfmt")
                       "-c" "--statistics" "--verbose"
                       "-o" gmo
                       (string-append po-directory "/" language ".po"))))
@@ -280,20 +287,19 @@ (define (translate-texi-manuals source)
   "Return the translated texinfo manuals built from SOURCE."
   (define po4a
     (specification->package "po4a"))
-  
-  (define gettext
-    (specification->package "gettext"))
+
+  (define gettext-minimal
+    (specification->package "gettext-minimal"))
 
   (define glibc-utf8-locales
-    (module-ref (resolve-interface '(gnu packages base))
-                'glibc-utf8-locales))
+    (specification->package "glibc-utf8-locales"))
 
   (define documentation
     (file-append* source "doc"))
 
   (define documentation-po
     (file-append* source "po/doc"))
-  
+
   (define build
     (with-imported-modules '((guix build utils) (guix build po))
       #~(begin
@@ -365,7 +371,7 @@ (define parallel-jobs
 
           (setenv "GUIX_LOCPATH"
                   #+(file-append glibc-utf8-locales "/lib/locale"))
-          (setenv "PATH" #+(file-append gettext "/bin"))
+          (setenv "PATH" #+(file-append gettext-minimal "/bin"))
           (setenv "LC_ALL" "en_US.UTF-8")
           (setlocale LC_ALL "en_US.UTF-8")
 
@@ -394,16 +400,13 @@ (define parallel-jobs
 (define (info-manual source)
   "Return the Info manual built from SOURCE."
   (define texinfo
-    (module-ref (resolve-interface '(gnu packages texinfo))
-                'texinfo))
+    (specification->package "texinfo"))
 
   (define graphviz
-    (module-ref (resolve-interface '(gnu packages graphviz))
-                'graphviz))
+    (specification->package "graphviz"))
 
   (define glibc-utf8-locales
-    (module-ref (resolve-interface '(gnu packages base))
-                'glibc-utf8-locales))
+    (specification->package "glibc-utf8-locales"))
 
   (define documentation
     (file-append* source "doc"))
@@ -586,8 +589,7 @@ (define* (guix-command modules
   "Return the 'guix' command such that it adds MODULES and DEPENDENCIES in its
 load path."
   (define glibc-utf8-locales
-    (module-ref (resolve-interface '(gnu packages base))
-                'glibc-utf8-locales))
+    (specification->package "glibc-utf8-locales"))
 
   (define module-directory
     ;; To minimize the number of 'stat' calls needed to locate a module,
@@ -1030,10 +1032,7 @@ (define (built-modules node-subset)
                           ;; Include 'guix-daemon'.  XXX: Here we inject an
                           ;; older snapshot of guix-daemon, but that's a good
                           ;; enough approximation for now.
-                          #:daemon (module-ref (resolve-interface
-                                                '(gnu packages
-                                                      package-management))
-                                               'guix-daemon)
+                          #:daemon (specification->package "guix-daemon")
 
                           #:info (info-manual source)
                           #:miscellany (miscellaneous-files source)
-- 
2.39.1

Christopher Baines wrote on 28 Feb 2023 16:47

[PATCH v2 3/3] self: Apply grafts to the outputs of the guix derivation.

Recipients:(address . 61363@debbugs.gnu.org)

Message-ID:20230228154703.3952-3-mail@cbaines.net

Rather than having grafts apply to the derivation itself. This moves grafting
here to work like grafting for packages, where you can think of the grafted
outputs as a transformed variant of the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the
derivations without grafts, and for these to be useful for substitutes
regardless of whether users are using grafts.

* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword
argument, to control grafting when computing the guix derivation.
* build-aux/build-self.scm (build-program): Call guix-derivation with
 #:graft? (%graft?) to make the compute-guix-derivation script use or not use
grafts as desired.
---
 build-aux/build-self.scm |   4 +-
 guix/self.scm            | 101 +++++++++++++++++++++++++++++++--------
 2 files changed, 84 insertions(+), 21 deletions(-)

Toggle diff (173 lines)diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
                                                   #:channel-metadata
                                                   '#$channel-metadata
                                                   #:pull-version
-                                                  #$pull-version)
+                                                  #$pull-version
+                                                  #:graft?
+                                                  #$(%graft?))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index c5de3ab8fc..8842275ff8 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -22,6 +22,7 @@ (define-module (guix self)
   #:use-module (guix i18n)
   #:use-module (guix modules)
   #:use-module (guix gexp)
+  #:use-module (guix grafts)
   #:use-module (guix store)
   #:use-module (guix monads)
   #:use-module (guix discovery)
@@ -32,6 +33,7 @@ (define-module (guix self)
   #:use-module ((guix build utils) #:select (find-files))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
   #:export (make-config.scm
@@ -244,6 +246,50 @@ (define* (file-append* item file #:key (recursive? #t))
      ;; which isn't great.
      (file-append item "/" file))))
 
+(define graft-derivation*
+  (store-lift graft-derivation))
+
+(define package-grafts*
+  (store-lift package-grafts))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+  (%explicit-grafting obj packages)
+  explicit-grafting?
+  (obj      explicit-grafting-obj)       ;obj
+  (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+  (match rec
+    (($ <explicit-grafting> obj packages)
+     (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+  (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+                                                  system target)
+  (match explicit-grafting
+    (($ <explicit-grafting> obj packages)
+     (mlet* %store-monad ((drv (without-grafting
+                                (lower-object obj system #:target target)))
+                          (grafts
+                           (mapm %store-monad
+                                 (lambda (pkg)
+                                   (package-grafts* pkg system #:target target))
+                                 packages)))
+       (match (delete-duplicates
+               (concatenate grafts))
+         (()
+          (return drv))
+         (grafts
+          (mlet %store-monad ((guile (package->derivation
+                                      (guile-for-grafts)
+                                      system #:graft? #f)))
+            (graft-derivation* drv grafts
+                               #:system system
+                               #:guile guile))))))))
+
 (define* (locale-data source domain
                       #:optional (directory domain))
   "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
@@ -754,7 +800,8 @@ (define* (compiled-guix source #:key
                         (gzip (specification->package "gzip"))
                         (bzip2 (specification->package "bzip2"))
                         (xz (specification->package "xz"))
-                        (guix (specification->package "guix")))
+                        (guix (specification->package "guix"))
+                        (graft? #t))
   "Return a file-like object that contains a compiled Guix."
   (define guile-avahi
     (specification->package "guile-avahi"))
@@ -1024,25 +1071,34 @@ (define (built-modules node-subset)
                                                guile-lzma
                                                dependencies)
                                         #:guile guile-for-build
-                                        #:guile-version guile-version)))
-           (whole-package name modules dependencies
-                          #:command command
-                          #:guile guile-for-build
-
-                          ;; Include 'guix-daemon'.  XXX: Here we inject an
-                          ;; older snapshot of guix-daemon, but that's a good
-                          ;; enough approximation for now.
-                          #:daemon (specification->package "guix-daemon")
-
-                          #:info (info-manual source)
-                          #:miscellany (miscellaneous-files source)
-                          #:guile-version guile-version)))
+                                        #:guile-version guile-version))
+                (obj
+                 (whole-package name modules dependencies
+                                #:command command
+                                #:guile guile-for-build
+
+                                ;; Include 'guix-daemon'.  XXX: Here we inject
+                                ;; an older snapshot of guix-daemon, but
+                                ;; that's a good enough approximation for now.
+                                #:daemon (specification->package "guix-daemon")
+
+                                #:info (info-manual source)
+                                #:miscellany (miscellaneous-files source)
+                                #:guile-version guile-version)))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         ((= 0 pull-version)
          ;; Legacy 'guix pull': return the .scm and .go files as one
          ;; directory.
-         (built-modules (lambda (node)
-                          (list (node-source node)
-                                (node-compiled node)))))
+         (let ((obj (built-modules (lambda (node)
+                                     (list (node-source node)
+                                           (node-compiled node))))))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         (else
          ;; Unsupported 'guix pull' version.
          #f)))
@@ -1272,7 +1328,8 @@ (define (process-directory directory files output)
 (define* (guix-derivation source version
                           #:optional (guile-version (effective-version))
                           #:key (pull-version 0)
-                          channel-metadata)
+                          channel-metadata
+                          (graft? #t))
   "Return, as a monadic value, the derivation to build the Guix from SOURCE
 for GUILE-VERSION.  Use VERSION as the version string.  Use CHANNEL-METADATA
 as the channel metadata sexp to include in (guix config).
@@ -1309,7 +1366,11 @@ (define guile
                                #:pull-version pull-version
                                #:guile-version (if (>= pull-version 1)
                                                    "3.0" guile-version)
-                               #:guile-for-build guile)))
+                               #:guile-for-build guile
+                               #:graft? graft?)))
       (if guix
-          (lower-object guix)
+          (if graft?
+              (lower-object guix)
+              (without-grafting
+               (lower-object guix)))
           (return #f)))))
-- 
2.39.1

Christopher Baines wrote on 28 Feb 2023 16:47

Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61363@debbugs.gnu.org)

Message-ID:871qm9aiw7.fsf@cbaines.net

Christopher Baines <mail@cbaines.net> writes:

Toggle quote (31 lines)> Ludovic Courtès <ludo@gnu.org> writes:
>
>>   2. More importantly, manually listing packages that might require
>>      grafting looks like a slippery slope (“oops! we’re not getting the
>>      GnuTLS graft for that CVE, too bad”).
>>
>> I designed and implemented several variants to try and delay grafting.
>> One of them consisted in carrying graft information in gexps:
>>
>>   https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts
>>
>> It’s kinda similar to what you’re proposing in that graft information is
>> carried as far as possible.  The main difference is that it’s automated.
>
> That's interesting, I think that making grafting not specific to
> packages, and something where the replacement is handled at a lower
> level (e.g. gexps) would be an alternative way to handle this.
>
> Given that this approach works though, maybe the explicit-grafting
> functionality could just sit and be used inside of (guix self). Given
> that module is very explicit about what packages are used, it should be
> possible to arrange the code so it's very hard to miss a package out,
> which should address your concern about manually listing packages (maybe
> specification->package can be tweaked so that it's possible to get all
> the packages, and that can be the list considered for grafting).
>
> I don't know of any other places where this approach would be useful, so
> while it would be nice to have a more general grafting mechanism
> eventually, I'd also like to be able to make these changes to channel
> instance grafts sooner rather than later.

I've sent a v2 series which changes along the above lines. The explicit
grafting stuff just sits in (guix self), and (guix self) more
rigeriously uses it's own definition of specification->package, which
should provide some protection against missing packages out. Obviously
it's not quite as rigerous as moving the grafting functionality in to
gexps, but hopefully it's rigerous enough for now.

Christopher Baines wrote on 17 Apr 2023 16:59

[PATCH v3] self: Apply grafts to the outputs of the guix derivation.

Recipients:(address . 61363@debbugs.gnu.org)(name . Christopher Baines)(address . mail@cbaines.net)

Message-ID:20230417145928.16693-1-mail@cbaines.net

Rather than having grafts apply to the derivation itself. This moves grafting

here to work like grafting for packages, where you can think of the grafted

outputs as a transformed variant of the ungrafted outputs.

I'm looking at this as it'll allow the Guix Data Service to compute the

derivations without grafts, and for these to be useful for substitutes

regardless of whether users are using grafts.

* guix/self.scm (compiled-guix, guix-derivation): Add a #:graft? keyword

argument, to control grafting when computing the guix derivation.

* build-aux/build-self.scm (build-program): Call guix-derivation with

#:graft? (%graft?) to make the compute-guix-derivation script use or not use

grafts as desired.

Signed-off-by: Christopher Baines <mail@cbaines.net>

---

build-aux/build-self.scm | 4 +-

guix/self.scm | 101 +++++++++++++++++++++++++++++++--------

2 files changed, 84 insertions(+), 21 deletions(-)

Toggle diff (173 lines)diff --git a/build-aux/build-self.scm b/build-aux/build-self.scm
index 02822a2ee8..6d0037f20c 100644
--- a/build-aux/build-self.scm
+++ b/build-aux/build-self.scm
@@ -353,7 +353,9 @@ (define fake-git
                                                   #:channel-metadata
                                                   '#$channel-metadata
                                                   #:pull-version
-                                                  #$pull-version)
+                                                  #$pull-version
+                                                  #:graft?
+                                                  #$(%graft?))
                                  #:system system))
                              derivation-file-name))))))
                   #:module-path (list source))))
diff --git a/guix/self.scm b/guix/self.scm
index 74c953bd50..bbc0beaca8 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -21,6 +21,7 @@ (define-module (guix self)
   #:use-module (guix config)
   #:use-module (guix modules)
   #:use-module (guix gexp)
+  #:use-module (guix grafts)
   #:use-module (guix store)
   #:use-module (guix monads)
   #:use-module (guix discovery)
@@ -31,6 +32,7 @@ (define-module (guix self)
   #:use-module ((guix build utils) #:select (find-files))
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-9)
+  #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
   #:export (make-config.scm
@@ -243,6 +245,50 @@ (define* (file-append* item file #:key (recursive? #t))
      ;; which isn't great.
      (file-append item "/" file))))
 
+(define graft-derivation*
+  (store-lift graft-derivation))
+
+(define package-grafts*
+  (store-lift package-grafts))
+
+;; Apply grafts explicitly
+(define-immutable-record-type <explicit-grafting>
+  (%explicit-grafting obj packages)
+  explicit-grafting?
+  (obj      explicit-grafting-obj)       ;obj
+  (packages explicit-grafting-packages)) ;list of <package>s
+
+(define (write-explicit-grafting rec port)
+  (match rec
+    (($ <explicit-grafting> obj packages)
+     (format port "#<explicit-grafting ~s ~s>" obj packages))))
+
+(define (explicit-grafting obj packages)
+  (%explicit-grafting obj packages))
+
+(define-gexp-compiler (explicit-grafting-compiler (explicit-grafting <explicit-grafting>)
+                                                  system target)
+  (match explicit-grafting
+    (($ <explicit-grafting> obj packages)
+     (mlet* %store-monad ((drv (without-grafting
+                                (lower-object obj system #:target target)))
+                          (grafts
+                           (mapm %store-monad
+                                 (lambda (pkg)
+                                   (package-grafts* pkg system #:target target))
+                                 packages)))
+       (match (delete-duplicates
+               (concatenate grafts))
+         (()
+          (return drv))
+         (grafts
+          (mlet %store-monad ((guile (package->derivation
+                                      (guile-for-grafts)
+                                      system #:graft? #f)))
+            (graft-derivation* drv grafts
+                               #:system system
+                               #:guile guile))))))))
+
 (define* (locale-data source domain
                       #:optional (directory domain))
   "Return the locale data from 'po/DIRECTORY' in SOURCE, corresponding to
@@ -753,7 +799,8 @@ (define* (compiled-guix source #:key
                         (gzip (specification->package "gzip"))
                         (bzip2 (specification->package "bzip2"))
                         (xz (specification->package "xz"))
-                        (guix (specification->package "guix")))
+                        (guix (specification->package "guix"))
+                        (graft? #t))
   "Return a file-like object that contains a compiled Guix."
   (define guile-avahi
     (specification->package "guile-avahi"))
@@ -1023,25 +1070,34 @@ (define (built-modules node-subset)
                                                guile-lzma
                                                dependencies)
                                         #:guile guile-for-build
-                                        #:guile-version guile-version)))
-           (whole-package name modules dependencies
-                          #:command command
-                          #:guile guile-for-build
-
-                          ;; Include 'guix-daemon'.  XXX: Here we inject an
-                          ;; older snapshot of guix-daemon, but that's a good
-                          ;; enough approximation for now.
-                          #:daemon (specification->package "guix-daemon")
-
-                          #:info (info-manual source)
-                          #:miscellany (miscellaneous-files source)
-                          #:guile-version guile-version)))
+                                        #:guile-version guile-version))
+                (obj
+                 (whole-package name modules dependencies
+                                #:command command
+                                #:guile guile-for-build
+
+                                ;; Include 'guix-daemon'.  XXX: Here we inject
+                                ;; an older snapshot of guix-daemon, but
+                                ;; that's a good enough approximation for now.
+                                #:daemon (specification->package "guix-daemon")
+
+                                #:info (info-manual source)
+                                #:miscellany (miscellaneous-files source)
+                                #:guile-version guile-version)))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         ((= 0 pull-version)
          ;; Legacy 'guix pull': return the .scm and .go files as one
          ;; directory.
-         (built-modules (lambda (node)
-                          (list (node-source node)
-                                (node-compiled node)))))
+         (let ((obj (built-modules (lambda (node)
+                                     (list (node-source node)
+                                           (node-compiled node))))))
+           (if graft?
+               (explicit-grafting obj
+                                  (map (compose force cdr) %packages))
+               obj)))
         (else
          ;; Unsupported 'guix pull' version.
          #f)))
@@ -1271,7 +1327,8 @@ (define (process-directory directory files output)
 (define* (guix-derivation source version
                           #:optional (guile-version (effective-version))
                           #:key (pull-version 0)
-                          channel-metadata)
+                          channel-metadata
+                          (graft? #t))
   "Return, as a monadic value, the derivation to build the Guix from SOURCE
 for GUILE-VERSION.  Use VERSION as the version string.  Use CHANNEL-METADATA
 as the channel metadata sexp to include in (guix config).
@@ -1308,7 +1365,11 @@ (define guile
                                #:pull-version pull-version
                                #:guile-version (if (>= pull-version 1)
                                                    "3.0" guile-version)
-                               #:guile-for-build guile)))
+                               #:guile-for-build guile
+                               #:graft? graft?)))
       (if guix
-          (lower-object guix)
+          (if graft?
+              (lower-object guix)
+              (without-grafting
+               (lower-object guix)))
           (return #f)))))
-- 
2.39.1

Christopher Baines wrote on 17 Apr 2023 17:06

Re: [bug#61363] [PATCH 2/2] self: Apply grafts to the outputs of the guix derivation.

Recipients:(address . 61363@debbugs.gnu.org)

Message-ID:87bkjmfsy5.fsf@cbaines.net

Christopher Baines <mail@cbaines.net> writes:

Toggle quote (42 lines)> [[PGP Signed Part:Undecided]]
>
> Christopher Baines <mail@cbaines.net> writes:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>>   2. More importantly, manually listing packages that might require
>>>      grafting looks like a slippery slope (“oops! we’re not getting the
>>>      GnuTLS graft for that CVE, too bad”).
>>>
>>> I designed and implemented several variants to try and delay grafting.
>>> One of them consisted in carrying graft information in gexps:
>>>
>>>   https://git.savannah.gnu.org/cgit/guix.git/log?h=wip-gexp-grafts
>>>
>>> It’s kinda similar to what you’re proposing in that graft information is
>>> carried as far as possible.  The main difference is that it’s automated.
>>
>> That's interesting, I think that making grafting not specific to
>> packages, and something where the replacement is handled at a lower
>> level (e.g. gexps) would be an alternative way to handle this.
>>
>> Given that this approach works though, maybe the explicit-grafting
>> functionality could just sit and be used inside of (guix self). Given
>> that module is very explicit about what packages are used, it should be
>> possible to arrange the code so it's very hard to miss a package out,
>> which should address your concern about manually listing packages (maybe
>> specification->package can be tweaked so that it's possible to get all
>> the packages, and that can be the list considered for grafting).
>>
>> I don't know of any other places where this approach would be useful, so
>> while it would be nice to have a more general grafting mechanism
>> eventually, I'd also like to be able to make these changes to channel
>> instance grafts sooner rather than later.
>
> I've sent a v2 series which changes along the above lines. The explicit
> grafting stuff just sits in (guix self), and (guix self) more
> rigeriously uses it's own definition of specification->package, which
> should provide some protection against missing packages out. Obviously
> it's not quite as rigerous as moving the grafting functionality in to
> gexps, but hopefully it's rigerous enough for now.

This has stalled a bit, but it would be good to try and get things
merged. I've gone ahead and pushed the first two patches in the series I
last sent, these just make minor changes to prepare for the functional
change here. I've also resent that patch as as v3.

Simon Tournier wrote on 16 May 2023 15:25

Re: [bug#61363] [PATCH v3] self: Apply grafts to the outputs of the guix derivation.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)

Message-ID:86353w76i6.fsf@gmail.com

Hi Chris,

I am late to the party and probably do not well understand all that

part. Just a quick comment in the same direction as Ludo.

On Mon, 17 Apr 2023 at 15:59, Christopher Baines <mail@cbaines.net> wrote:

Toggle quote (5 lines)

> diff --git a/guix/self.scm b/guix/self.scm

> index 74c953bd50..bbc0beaca8 100644

> --- a/guix/self.scm

> +++ b/guix/self.scm

[...]

Toggle quote (5 lines)

> + (if graft?

> + (explicit-grafting obj

> + (map (compose force cdr) %packages))

> + obj)))

[...]

Toggle quote (5 lines)

> + (if graft?

> + (explicit-grafting obj

> + (map (compose force cdr) %packages))

> + obj)))

It means that the grafts are only applied to %packages, right?

Other said, defined by:

Toggle snippet (33 lines)(define %packages
  (let ((ref (lambda (module variable)
               (delay
                 (module-ref (resolve-interface
                              `(gnu packages ,module))
                             variable)))))
    `(("guile"              . ,(ref 'guile 'guile-3.0-latest))
      ("guile-avahi"        . ,(ref 'guile-xyz 'guile-avahi))
      ("guile-json"         . ,(ref 'guile 'guile-json-4))
      ("guile-ssh"          . ,(ref 'ssh   'guile-ssh))
      ("guile-git"          . ,(ref 'guile 'guile-git))
      ("guile-semver"       . ,(ref 'guile-xyz 'guile-semver))
      ("guile-lib"          . ,(ref 'guile-xyz 'guile-lib))
      ("guile-sqlite3"      . ,(ref 'guile 'guile-sqlite3))
      ("guile-zlib"         . ,(ref 'guile 'guile-zlib))
      ("guile-lzlib"        . ,(ref 'guile 'guile-lzlib))
      ("guile-zstd"         . ,(ref 'guile 'guile-zstd))
      ("guile-gcrypt"       . ,(ref 'gnupg 'guile-gcrypt))
      ("guile-gnutls"       . ,(ref 'tls 'guile-gnutls))
      ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
      ("disarchive"         . ,(ref 'backup 'disarchive))
      ("guile-lzma"         . ,(ref 'guile 'guile-lzma))
      ("gzip"               . ,(ref 'compression 'gzip))
      ("bzip2"              . ,(ref 'compression 'bzip2))
      ("xz"                 . ,(ref 'compression 'xz))
      ("po4a"               . ,(ref 'gettext 'po4a))
      ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
      ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
      ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
      ("graphviz"           . ,(ref 'graphviz 'graphviz))
      ("texinfo"            . ,(ref 'texinfo 'texinfo)))))

tweaked by e5c33837cbee98d460d9ae09b463501de6f15d97. And there is a

slippery slope: the manual addition. These had been added with

e5c33837cbee98d460d9ae09b463501de6f15d97:

+ ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))

+ ("graphviz" . ,(ref 'graphviz 'graphviz))

+ ("guix-daemon" . ,(ref 'package-management 'guix-daemon))

+ ("texinfo" . ,(ref 'texinfo 'texinfo)))))

Other said, what does it happen if we forget to manually update this

list?

Cheers,

simon

Christopher Baines wrote on 3 Jun 2023 13:41

Recipients:(name . Simon Tournier)(address . zimon.toutoune@gmail.com)(address . 61363@debbugs.gnu.org)

Message-ID:87bkhw6a9k.fsf@cbaines.net

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (74 lines)> Hi Chris,
>
> I am late to the party and probably do not well understand all that
> part.  Just a quick comment in the same direction as Ludo.
>
> On Mon, 17 Apr 2023 at 15:59, Christopher Baines <mail@cbaines.net> wrote:
>
>> diff --git a/guix/self.scm b/guix/self.scm
>> index 74c953bd50..bbc0beaca8 100644
>> --- a/guix/self.scm
>> +++ b/guix/self.scm
>
> [...]
>
>> +           (if graft?
>> +               (explicit-grafting obj
>> +                                  (map (compose force cdr) %packages))
>> +               obj)))
>
> [...]
>
>> +           (if graft?
>> +               (explicit-grafting obj
>> +                                  (map (compose force cdr) %packages))
>> +               obj)))
>
> It means that the grafts are only applied to %packages, right?
>
> Other said, defined by:
>
> (define %packages
>   (let ((ref (lambda (module variable)
>                (delay
>                  (module-ref (resolve-interface
>                               `(gnu packages ,module))
>                              variable)))))
>     `(("guile"              . ,(ref 'guile 'guile-3.0-latest))
>       ("guile-avahi"        . ,(ref 'guile-xyz 'guile-avahi))
>       ("guile-json"         . ,(ref 'guile 'guile-json-4))
>       ("guile-ssh"          . ,(ref 'ssh   'guile-ssh))
>       ("guile-git"          . ,(ref 'guile 'guile-git))
>       ("guile-semver"       . ,(ref 'guile-xyz 'guile-semver))
>       ("guile-lib"          . ,(ref 'guile-xyz 'guile-lib))
>       ("guile-sqlite3"      . ,(ref 'guile 'guile-sqlite3))
>       ("guile-zlib"         . ,(ref 'guile 'guile-zlib))
>       ("guile-lzlib"        . ,(ref 'guile 'guile-lzlib))
>       ("guile-zstd"         . ,(ref 'guile 'guile-zstd))
>       ("guile-gcrypt"       . ,(ref 'gnupg 'guile-gcrypt))
>       ("guile-gnutls"       . ,(ref 'tls 'guile-gnutls))
>       ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
>       ("disarchive"         . ,(ref 'backup 'disarchive))
>       ("guile-lzma"         . ,(ref 'guile 'guile-lzma))
>       ("gzip"               . ,(ref 'compression 'gzip))
>       ("bzip2"              . ,(ref 'compression 'bzip2))
>       ("xz"                 . ,(ref 'compression 'xz))
>       ("po4a"               . ,(ref 'gettext 'po4a))
>       ("gettext-minimal"    . ,(ref 'gettext 'gettext-minimal))
>       ("gcc-toolchain"      . ,(ref 'commencement 'gcc-toolchain))
>       ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales))
>       ("graphviz"           . ,(ref 'graphviz 'graphviz))
>       ("texinfo"            . ,(ref 'texinfo 'texinfo)))))
>
> tweaked by e5c33837cbee98d460d9ae09b463501de6f15d97.  And there is a
> slippery slope: the manual addition.  These had been added with
> e5c33837cbee98d460d9ae09b463501de6f15d97:
>
>     + ("glibc-utf8-locales" . ,(ref 'base               'glibc-utf8-locales))
>     + ("graphviz"           . ,(ref 'graphviz           'graphviz))
>     + ("guix-daemon"        . ,(ref 'package-management 'guix-daemon))
>     + ("texinfo"            . ,(ref 'texinfo            'texinfo)))))
>
> Other said, what does it happen if we forget to manually update this
> list?

Well, specification->package in (guix self) won't work for the missing

packages.

It's possible to use packages outside of this list, but that doesn't

happen currently.

Christopher Baines wrote on 3 Jun 2023 13:44

tag 61363 moreinfo

Recipients:(address . control@debbugs.gnu.org)

Message-ID:875y84bwit.fsf@cbaines.net

tags 61363 + moreinfo

quit

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 61363@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 61363

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date