Grafting inconsistency with multiple outputs is causing problems

  • Open
  • quality assurance status badge
Details
2 participants
  • Dariqq
  • pelzflorian (Florian Pelz)
Owner
unassigned
Submitted by
Dariqq
Severity
important
Merged with
D
D
Dariqq wrote on 28 Dec 2024 09:52
(address . bug-guix@gnu.org)
fc701a2e-4317-459e-9685-56a14f5d2e21@posteo.net
Hello,

Yesterday I was playing around with g-golf and libguile and encountered
some weird glib issues once I linked my c program to glib/gobject.

After asking about this in #guile the problem was discovered to be 2
different glibs being in use and the workaround is use them with
'--no-grafts' instead.

I looked into why grafting is causing this problem and here is what I
have found:

The following happend on guix commit
e16cdcf37d8223b3634ec5e658356c3b7f154859


Before grafting:
/gnu/store/digl75knygzqg9lrh8adk3pz0qg9cx8h-guile-g-golf-0.8.0-a.1:
Has a reference to: /gnu/store/zgsphhmliwgmjjv1czmbyjql3gk7ynsx-glib-2.78.0
which is the out output of the hidden (@@ (gnu packages glib) glib) package


After applying grafts:
/gnu/store/2ar2h8d7nk3g7c9965y4n68gjr778prk-guile-g-golf-0.8.0-a.1:
Has reference to /gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0

But the hidden glib grafts to
/gnu/store/ls9p0k2chrzviadibzf0iz7rlpkmggms-glib-2.78.0
instead (which is the one I am linking to)


Looking at the grafting derivations the issue seems to be that in one
case all outputs of glib are grafted and in the other case only the out
output which leads to the different results.


8<---------------cut here---------------start------------->8---
/gnu/store/ikv0s6dr3yzs398i67vh3p0w8jglxm4w-glib-2.78.0.drv
Derive
([("out","/gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0","","")]
,[("/gnu/store/0s4qbdir1j1c6yfnhvj3w4kpplsaafik-pcre2-10.42.drv",["out"])
,("/gnu/store/1554x8kqj7xjw5an8rzrnfz3v40spiy7-glibc-2.39.drv",["out"])
,("/gnu/store/2c70wagxjjxk8lcj3znrqqlvn2dcj024-glibc-2.39.drv",["out"])
,("/gnu/store/523ngwgnmyhszf9mqbk537ka41jbzs00-guile-3.0.9.drv",["out"])
,("/gnu/store/7lwb609ira5a3rs9ncxk55azvsnwi6vv-glib-2.78.0.drv",["out"])
,("/gnu/store/gmp1ciyg5yna6983phnf22pwsivfanwq-zlib-1.3.drv",["out"])
,("/gnu/store/gxn87vb8lb1l3yzmnzidmxbn3nk9x3bf-gcc-11.4.0.drv",["lib"])
,("/gnu/store/h268dh05da4jfrl94xq5jkvyic7k2h57-libffi-3.4.4.drv",["out"])
,("/gnu/store/h50bkx0qkqj7x0f1sbkgkiszkbdq32qg-zlib-1.3.drv",["out"])
,("/gnu/store/hkz1acxfcxil4252qw7i25gcn0f852dw-libffi-3.4.4.drv",["out"])
,("/gnu/store/jyki6954ps4gnhpznacfws146svldkpf-gcc-11.4.0.drv",["lib"])
,("/gnu/store/vhi205g193z01plb6pbhm75b186zrhis-util-linux-2.37.4.drv",["lib"])
,("/gnu/store/w5pgbafd9apavpnczcgblbgmwwp0a8y0-util-linux-2.37.4.drv",["lib"])
,("/gnu/store/yni45i1ca3ymb8rna09qh038v102zyjj-pcre2-10.42.drv",["out"])
,("/gnu/store/zp7byldvi2imbliv154vv87iai65jwxy-module-import-compiled.drv",["out"])]
,["/gnu/store/2mml5mrmhmn1424pymllzk18m2z3l44q-glib-2.78.0-builder","/gnu/store/flaxnn4ciba6wd243ahwgrqw11082imp-module-import"]
,"x86_64-linux","/gnu/store/xv4cd7qz4yan93zkjisbmbpxfz78hah2-guile-3.0.9/bin/guile",["--no-auto-compile","-L","/gnu/store/flaxnn4ciba6wd243ahwgrqw11082imp-module-import","-C","/gnu/store/5wdfvn4mklppcg5p9i1d14ksbnsd7m8b-module-import-compiled","/gnu/store/2mml5mrmhmn1424pymllzk18m2z3l44q-glib-2.78.0-builder"]
,[("allowSubstitutes","0")
,("guix properties","((type . graft) (graft (count . 6)))")
,("out","/gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0")
,("preferLocalBuild","1")])
Toggle snippet (57 lines)

The "correctly" grafted glib derivation looks like this:
/gnu/store/xax79x40lzs4wf2v4ipbxvy2gkc9qq7p-glib-2.78.0.drv
8<---------------cut here---------------start------------->8---
gnu/store/xax79x40lzs4wf2v4ipbxvy2gkc9qq7p-glib-2.78.0.drv
Derive
([("bin","/gnu/store/q3d08x70gcjj99k5dd0jbfc8rj97j9lz-glib-2.78.0-bin","","")
,("debug","/gnu/store/pn2mdlc86i7x6vrg2j6pslp5sq551255-glib-2.78.0-debug","","")
,("out","/gnu/store/ls9p0k2chrzviadibzf0iz7rlpkmggms-glib-2.78.0","","")
,("static","/gnu/store/n5s2s4rg3jax3y7kp93bgn2w6s7vlfrz-glib-2.78.0-static","","")]
,[("/gnu/store/0s4qbdir1j1c6yfnhvj3w4kpplsaafik-pcre2-10.42.drv",["out"])
,("/gnu/store/1554x8kqj7xjw5an8rzrnfz3v40spiy7-glibc-2.39.drv",["out"])
,("/gnu/store/2c70wagxjjxk8lcj3znrqqlvn2dcj024-glibc-2.39.drv",["out"])
,("/gnu/store/523ngwgnmyhszf9mqbk537ka41jbzs00-guile-3.0.9.drv",["out"])
,("/gnu/store/7lwb609ira5a3rs9ncxk55azvsnwi6vv-glib-2.78.0.drv",["bin","debug","out","static"])
,("/gnu/store/gmp1ciyg5yna6983phnf22pwsivfanwq-zlib-1.3.drv",["out"])
,("/gnu/store/gxn87vb8lb1l3yzmnzidmxbn3nk9x3bf-gcc-11.4.0.drv",["lib"])
,("/gnu/store/h268dh05da4jfrl94xq5jkvyic7k2h57-libffi-3.4.4.drv",["out"])
,("/gnu/store/h50bkx0qkqj7x0f1sbkgkiszkbdq32qg-zlib-1.3.drv",["out"])
,("/gnu/store/hkz1acxfcxil4252qw7i25gcn0f852dw-libffi-3.4.4.drv",["out"])
,("/gnu/store/jyki6954ps4gnhpznacfws146svldkpf-gcc-11.4.0.drv",["lib"])
,("/gnu/store/piymwc0w49ylyz69ikq6vnj32xhl96h6-bash-minimal-5.1.16.drv",["out"])
,("/gnu/store/vhi205g193z01plb6pbhm75b186zrhis-util-linux-2.37.4.drv",["lib"])
,("/gnu/store/w5pgbafd9apavpnczcgblbgmwwp0a8y0-util-linux-2.37.4.drv",["lib"])
,("/gnu/store/wqa52jpxmv1xiibdxvfls7dnphq83jb5-bash-minimal-5.1.16.drv",["out"])
,("/gnu/store/yhxc842f6arm91w4vhdmgazw2nvkn7g7-python-3.10.7.drv",["out"])
,("/gnu/store/yig8rg8bjy599m49kia7nph8393x5j2g-python-3.10.7.drv",["out"])
,("/gnu/store/yni45i1ca3ymb8rna09qh038v102zyjj-pcre2-10.42.drv",["out"])
,("/gnu/store/zp7byldvi2imbliv154vv87iai65jwxy-module-import-compiled.drv",["out"])]
,["/gnu/store/65ym8q6k1adkz4d4xalrsdq409r8k8r8-glib-2.78.0-builder","/gnu/store/flaxnn4ciba6wd243ahwgrqw11082imp-module-import"]
,"x86_64-linux","/gnu/store/xv4cd7qz4yan93zkjisbmbpxfz78hah2-guile-3.0.9/bin/guile",["--no-auto-compile","-L","/gnu/store/flaxnn4ciba6wd243ahwgrqw11082imp-module-import","-C","/gnu/store/5wdfvn4mklppcg5p9i1d14ksbnsd7m8b-module-import-compiled","/gnu/store/65ym8q6k1adkz4d4xalrsdq409r8k8r8-glib-2.78.0-builder"]
,[("allowSubstitutes","0")
,("bin","/gnu/store/q3d08x70gcjj99k5dd0jbfc8rj97j9lz-glib-2.78.0-bin")
,("debug","/gnu/store/pn2mdlc86i7x6vrg2j6pslp5sq551255-glib-2.78.0-debug")
,("guix properties","((type . graft) (graft (count . 8)))")
,("out","/gnu/store/ls9p0k2chrzviadibzf0iz7rlpkmggms-glib-2.78.0")
,("preferLocalBuild","1")
,("static","/gnu/store/n5s2s4rg3jax3y7kp93bgn2w6s7vlfrz-glib-2.78.0-static")])



I think it would be good if grafts are only applied per output (which
might also help resolve #69653).
P
P
pelzflorian (Florian Pelz) wrote on 29 Dec 2024 01:24
(name . Dariqq)(address . dariqq@posteo.net)(address . 75157@debbugs.gnu.org)
87o70vxrna.fsf@pelzflorian.de
Hello Dariqq, your debugging is interesting.

My G-Golf GTK program has a critical error at Gdk-Pixbuf, not GLib at
first sight, but see below.

$ guile /gnu/store/hbf4wqssp0aa0m2cxrwaq0mkxfd5y59p-launcher.scm

(guile:1273): Gtk-WARNING **: 01:21:40.538: Unable to acquire session bus: Failed to execute child process “dbus-launch” (No such file or directory)

(guile:1273): GLib-GObject-CRITICAL **: 01:21:40.586: cannot register existing type 'GdkPixbuf'

(guile:1273): GLib-GObject-CRITICAL **: 01:21:40.586: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(guile:1273): GLib-GObject-CRITICAL **: 01:21:40.586: g_type_add_interface_static: assertion 'G_TYPE_IS_INSTANTIATABLE (instance_type)' failed

(guile:1273): GLib-CRITICAL **: 01:21:40.586: g_once_init_leave: assertion 'result != 0' failed

It cannot register existing type 'GdkPixbuf'.
Bisecting reveals the first bad commit is this:

commit 78c4d00ab02ab41a22058cdbec0329752e47580f
gpg: Signature made Mi 18 Dez 2024 08:27:09 CET
gpg: using RSA key 27D586A4F8900854329FF09F1260E46482E63562
gpg: Can't check signature: No public key
Author: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Sat Dec 14 22:52:22 2024 +0900

gnu: glibc: Graft with fix for CVE-2024-2961.
* gnu/packages/base.scm (%glibc-patches): New variable.
(glibc) [source]: Use it.
[properties]: Mark CVE-2024-2961 as hidden (resolved).
[replacement]: Add field to graft with...
(glibc/fixed): ... this new package.
Change-Id: I6dd70b0e157283925824348f180c466c2f6387c9

I look at `cat /proc/635/maps`

/gnu/store/1av61zsp5sxs0as48nl52fg78b466r8p-gdk-pixbuf-2.42.10/lib/libgdk_pixbuf-2.0.so.0.4200.10
/gnu/store/1av61zsp5sxs0as48nl52fg78b466r8p-gdk-pixbuf-2.42.10/lib/girepository-1.0/GdkPixbuf-2.0.typelib

and

/gnu/store/10i71f2m5r6rijl8gvn1mpmvzxaxs76l-gdk-pixbuf-2.42.10/lib/libgdk_pixbuf-2.0.so.0.4200.10

this one without a matching typelib, so only the latter was not loaded
through a typelib.

$ guix gc --references /gnu/store/1av61zsp5sxs0as48nl52fg78b466r8p-gdk-pixbuf-2.42.10
/gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0
...
$ guix gc --derivers /gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0
/gnu/store/ikv0s6dr3yzs398i67vh3p0w8jglxm4w-glib-2.78.0.drv
$ guix build /gnu/store/ikv0s6dr3yzs398i67vh3p0w8jglxm4w-glib-2.78.0.drv
/gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0
$ guix gc --references /gnu/store/10i71f2m5r6rijl8gvn1mpmvzxaxs76l-gdk-pixbuf-2.42.10
/gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0
$ guix gc --derivers /gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0
/gnu/store/ikv0s6dr3yzs398i67vh3p0w8jglxm4w-glib-2.78.0.drv
$ guix build /gnu/store/ikv0s6dr3yzs398i67vh3p0w8jglxm4w-glib-2.78.0.drv
/gnu/store/0466q5rlccclzg70f3yprpf9kdvlyfzf-glib-2.78.0


Both my GLibs have one output only. I still believe above guix commit
is the first bad commit for you, as well, and

guix time-machine -q --commit=78c4d00ab02ab41a22058cdbec0329752e47580f

will show the error and parent commit

guix time-machine -q --commit=08e0144d2338c8fb51e52e74274554583d0faa69

will be fine. Is it?

I believe what we describe here is yet another symptom of old bug
https://issues.guix.gnu.org/48907, but if we have a first bad commit,
and if it is the first bad commit for you, too,
it might help solve it. Or perhaps this bug can be solved separately if
really the loading through typelib vs. not through typelib is the cause.

Regards,
Florian
D
D
Dariqq wrote on 29 Dec 2024 10:30
(name . pelzflorian (Florian Pelz))(address . pelzflorian@pelzflorian.de)(address . 75157@debbugs.gnu.org)
bf6c3172-132b-4290-b023-38a68652411c@posteo.net
Hello FLorian,


I think you are experiencing a different (related) issue because the
problem I seems to vanish by linking to the glib that is used by g-golf
instead (the --no-grafts options ensures that only one glib is in my
guix shell)

On 29.12.24 01:24, pelzflorian (Florian Pelz) wrote:
Toggle quote (16 lines)
>
>
> Both my GLibs have one output only. I still believe above guix commit
> is the first bad commit for you, as well, and
>
> guix time-machine -q --commit=78c4d00ab02ab41a22058cdbec0329752e47580f
>
> will show the error and parent commit
>
> guix time-machine -q --commit=08e0144d2338c8fb51e52e74274554583d0faa69
>
> will be fine. Is it?
>



The problem i am seeing is still available in
08e0144d2338c8fb51e52e74274554583d0faa69.


Here is what I've been playing around with:
a simple c file that just loads a scm file.
a simple scm file that just tries to instantiate a gobject class (in
this example from json-glib but that is irrelevant, using something
small and simple)

Toggle snippet (52 lines)
main.c:
#include <libguile.h>

int
main (int argc, char **argv)
{
scm_init_guile();
scm_c_primitive_load("foo.scm");
}

foo.scm:
(use-modules (g-golf))

(gi-import "Json")

(describe (make <json-parser>
#:immutable #t))


Build with
gcc -o main main.c \
`pkg-config --cflags --libs guile-3.0` \
`pkg-config --cflags --libs gobject-2.0`

Running this with grafts:
./main

(process:3071): GLib-GObject-CRITICAL **: 08:50:36.946:
g_object_new_is_valid_property: object class 'JsonParser' has no
property named 'immutable'
#<<json-parser> 7f186b2f56d0> is an instance of class <json-parser>
Slots are:
g-inst = #<pointer 0x1445e900>

(process:3071): GLib-GObject-CRITICAL **: 08:50:36.947:
g_object_get_is_valid_property: object class 'JsonParser' has no
property named 'immutable'

(process:3071): GLib-GObject-CRITICAL **: 08:50:36.947:
g_object_get_is_valid_property: object class 'JsonParser' has no
property named 'immutable'
immutable = #f

When not linking to gobject , using no-grafts or running directly from
guile repl things work as I would expect:
./main
#<<json-parser> 7ff2471f1d20> is an instance of class <json-parser>
Slots are:
g-inst = #<pointer 0x2bbbe920>
immutable = #t

On 08e0144d2338c8fb51e52e74274554583d0faa69 a similar problem exists:
g-golf depends on the ungrafted out output of hidden glib
but full glib is getting grafted (because of other other outputs)



If I prepend the glib with the single output graft to pkgconfig path and
rebuild (s.t. my c prgram is linked to the same glib that is used by
g-golf) things start magically working again.


Toggle quote (7 lines)
> I believe what we describe here is yet another symptom of old bug
> <https://issues.guix.gnu.org/48907>, but if we have a first bad commit,
> and if it is the first bad commit for you, too,
> it might help solve it. Or perhaps this bug can be solved separately if
> really the loading through typelib vs. not through typelib is the cause.
>

There might be an even bigger issue here?


However regardless of it causing issues I think having 2 different
grafted glibs (or rather anything) is not really great.

Toggle quote (3 lines)
> Regards,
> Florian

Have nice day,
Dariqq
P
P
pelzflorian (Florian Pelz) wrote on 2 Jan 16:20 +0100
(name . Dariqq)(address . dariqq@posteo.net)(address . 75157@debbugs.gnu.org)
87pll58coc.fsf@pelzflorian.de
Hi Dariqq. Your glib and json bug is indeed different.

Your initial suggestion for a proper fix to the different but similar
troubles we both have:

Toggle quote (3 lines)
> I think it would be good if grafts are only applied per output (which
> might also help resolve #69653).

is the same proposal as in bug#48907.

At https://bugs.gnu.org/48907Richard Sent wrote:
Toggle quote (5 lines)
> One idea I had was to somehow graft each output as a unique derivation
> via repeated calls to graft-derivation. From what I've seen so far
> though that looks much easier said than done. It hasn't progressed
> beyond the "huh, I wonder" phase.

Therefore I will merge the two bugs.

Not entirely sure if this proper fix would currently need changes to the
C++ guix-daemon code. Anyway, I will not implement it.

Currently we both have workarounds, obviously disabling security updates
with --no-grafts is possible, but also you wrote you can set the
pkgconfig path. For GTK programs, a workaround is instead of putting

(list gdk-pixbuf
graphene gtk harfbuzz
pango)

in your Guix shell manifest or a scheme-file, I could instead rebuild
without grafts, like in Guix’ ungraft manifest.

That is, my previous home configuration for commit
08e0144d2338c8fb51e52e74274554583d0faa69 was:
(service home-sway-service-type (sway-configuration (startup-programs `(,(with-extensions (list ((options->transformation '((with-commit . "guile-g-golf=v0.8.0-rc9"))) guile-g-golf)) #~(string-append "guile " #$(scheme-file "launcher.scm" #~(begin (eval-when (expand load eval) (use-modules (oop goops)) (default-duplicate-binding-handler '(merge-generics replace warn-override-core warn last)) (setenv "GI_TYPELIB_PATH" #$(file-append (directory-union "typelibs" (list gdk-pixbuf graphene gtk harfbuzz pango)) "/lib/girepository-1.0")) (use-modules (g-golf)) (g-irepository-require "Gtk" #:version "4.0") (for-each (lambda (name) (gi-import-by-name "Gtk" name)) '("Application" "ApplicationWindow" "Builder"))) (use-modules (sxml simple)) (define ui '(interface (requires (@ (version 4.0) (lib gtk)))
Since then, I have now had to change it to:
(service home-sway-service-type (sway-configuration (startup-programs `(,(with-extensions (list ((options->transformation '((with-commit . "guile-g-golf=v0.8.0-rc9"))) guile-g-golf)) #~(string-append "guile " #$(scheme-file "launcher.scm" #~(begin (eval-when (expand load eval) (use-modules (oop goops)) (default-duplicate-binding-handler '(merge-generics replace warn-override-core warn last)) (setenv "GI_TYPELIB_PATH" #$(file-append (directory-union "typelibs" (let ((ungraft (package-mapping (lambda (p) (cond ;; Ungrafted fails to validate runpath. ((string=? (package-name p) "gst-plugins-bad") (specification->package "gst-plugins-bad-minimal")) (else (let ((old (or (package-replacement p) p))) (package (inherit old) (arguments (substitute-keyword-arguments (package-arguments old) ((#:disallowed-references _) (list)) ((#:allowed-references _) #f)))))))) #:deep? #t))) (map ungraft (list gdk-pixbuf graphene gtk harfbuzz pango)))) "/lib/girepository-1.0")) (use-modules (g-golf)) (g-irepository-require "Gtk" #:version "4.0") (for-each (lambda (name) (gi-import-by-name "Gtk" name)) '("Application" "ApplicationWindow" "Builder"))) (use-modules (sxml simple)) (define ui
Unlike --no-grafts,
this has security updates appropriate for use in production
environments, but rebuilds part of the world, most importantly Rust.
Building bootstrap rust-1.54 fails on my laptop, unless I specify the
common build option --cores=1 in guix build.

I dedicate the above snippet to the public domain as per CC0.
If someone wishes to use similar code.

Regards,
Florian
P
P
pelzflorian (Florian Pelz) wrote on 2 Jan 16:25 +0100
merge 48907 75157
(address . control@debbugs.gnu.org)
87ldvt8cg9.fsf@pelzflorian.de
severity 75157 important
merge 48907 75157
thanks
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 75157@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 75157
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch