Hi,
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
Toggle quote (29 lines)
> Hello,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hi,
>>
>> Leo Famulari <leo@famulari.name> skribis:
>>
>>> On Fri, May 29, 2020 at 05:15:40PM +0200, Ludovic Courtès wrote:
>>>> The info suggests it won’t be substituted, but it’s eventually
>>>> substituted. I wonder why, because the .drv has:
>>>>
>>>> ("allowSubstitutes","0")
>>>>
>>>> and the daemon has:
>>>>
>>>> bool substitutesAllowed(const Derivation & drv)
>>>> {
>>>> return get(drv.env, "allowSubstitutes", "1") == "1";
>>>> }
>>>>
>>>> and:
>>>>
>>>> if (settings.useSubstitutes && substitutesAllowed(drv))
>>>> foreach (PathSet::iterator, i, invalidOutputs)
>>>> addWaitee(worker.makeSubstitutionGoal(*i, buildMode == bmRepair));
>>>>
>>>> Thoughts?
It's odd, when using --dry-run, it seems the expected situation would
happen:
Toggle snippet (9 lines)
$ guix build --no-grafts -n texlive
guix build --no-grafts -n texlive
La dérivation suivante serait compilée :
/gnu/store/v10c5wzji81pkwq2fhj123gw3d8il0ic-texlivetexmf-20240312.drv
4 270,0 Mo seraient téléchargés :
/gnu/store/pb6z5d5fx6s13cjzmvlw7dykpafp9x97-texlive-20240312-texmf.tar.xz
/gnu/store/0gkx9q1kyys8cis93cw9qhlzyx584dr6-texlive-20240312
Only the private texlivetexmf is marked as non-substitutable, so the
above looks correct (though we see the intention of preserving bandwidth
wouldn't be achieved, given the source is as large as the final
package).
When removing the --dry-run, what happens though is that the output of
the texlivetexmf package, which is marked as non-substitutable, is downloaded:
Toggle snippet (11 lines)
$ guix build --no-grafts texlive
La dérivation suivante sera compilée :
/gnu/store/v10c5wzji81pkwq2fhj123gw3d8il0ic-texlivetexmf-20240312.drv
4 270,0 Mo seront téléchargés :
/gnu/store/pb6z5d5fx6s13cjzmvlw7dykpafp9x97-texlive-20240312-texmf.tar.xz
/gnu/store/0gkx9q1kyys8cis93cw9qhlzyx584dr6-texlive-20240312
substitution de /gnu/store/b5sn5ha961hab37r7vl5p2n6sf46x582-texlivetexmf-20240312...
téléchargement depuis https://bordeaux.guix.gnu.org/nar/lzip/b5sn5ha961hab37r7vl5p2n6sf46x582-texlivetexmf-20240312...
texlivetexmf-20240312 3.95GiB 2.8MiB/s 00:06 ▕ ▏ 0.4% C-c C-c^C
I've had some success as stopping in the daemon right before the downlod
starts, with this sequence:
Start the locally built guix-daemon in gdb, with a command like:
Toggle snippet (3 lines)
sudo -E ./pre-inst-env gdb --args ./guix-daemon --debug --build-users-group=guixbuild --max-silent-time 3600 --timeout 86400 --log-compression none --discover=no --substitute-urls='https://bordeaux.guix.gnu.org https://ci.guix.gnu.org'
Then, to be able to reach the point in the execution where the downloads
are about to start, we need to break in the first fork (but not the 2nd,
which is the guile substituter), like follows at the GDB prompt:
Toggle snippet (41 lines)
set follow-fork-mode child
# To break inside the first child process
break LocalStore::querySubstitutablePathInfos
run
## from a terminal, 'guix build texlive' to hit breakpoint
delete 1
# avoid entering the guile substituter fork
set follow-fork-mode parent
break LocalStore::buildPaths
continue
-> should break in
Thread 2.1 "guix-daemon" hit Breakpoint 2.1, nix::LocalStore::buildPaths
(this=0x51f4d0, drvPaths=std::set with 1 element = {...},
buildMode=nix::bmNormal) at nix/libstore/build.cc:3627
Thread 2.1 "guix-daemon" hit Breakpoint 2.1, nix::LocalStore::buildPaths (this=0x51f4d0, drvPaths=std::set with 1 element = {...}, buildMode=nix::bmNormal) at nix/libstore/build.cc:3627
3627 {
(gdb) bt
#0 nix::LocalStore::buildPaths (this=0x51f4d0, drvPaths=std::set with 1 element = {...}, buildMode=nix::bmNormal) at nix/libstore/build.cc:3627
#1 0x00000000004253b6 in performOp (from=..., to=..., op=<optimized out>, clientVersion=356, trusted=false) at nix/nix-daemon/nix-daemon.cc:481
#2 processConnection (trusted=<optimized out>, userId=<optimized out>) at nix/nix-daemon/nix-daemon.cc:841
#3 0x000000000042805f in operator() (__closure=0x4e8f50) at nix/nix-daemon/nix-daemon.cc:1003
#4 0x00000000004280ac in std::__invoke_impl<void, acceptConnection(int)::<lambda()>&> (__f=...) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/invoke.h:61
#5 std::__invoke_r<void, acceptConnection(int)::<lambda()>&> (__fn=...) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/invoke.h:154
#6 std::_Function_handler<void(), acceptConnection(int)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/std_function.h:290
#7 0x0000000000495dbb in std::function<void()>::operator() (this=0x7fffffffbb00) at /gnu/store/hzsq64kmn9bmnkhhywav83miaaapl4m1-profile/include/c++/bits/std_function.h:590
#8 nix::startProcess (fun=..., dieWithParent=dieWithParent@entry=false, errorPrefix="unexpected build daemon error: ", runExitHandlers=runExitHandlers@entry=true) at nix/libutil/util.cc:1025
#9 0x00000000004237c2 in acceptConnection (fdSocket=<optimized out>) at nix/nix-daemon/nix-daemon.cc:977
#10 daemonLoop (sockets=std::vector of length 1, capacity 1 = {...}) at nix/nix-daemon/nix-daemon.cc:1055
#11 0x00000000004241b9 in run (sockets=std::vector of length 1, capacity 1 = {...}) at nix/nix-daemon/nix-daemon.cc:1064
#12 0x0000000000420635 in main (argc=<optimized out>, argv=<optimized out>) at nix/nix-daemon/guix-daemon.cc:569
I haven't investigated where in this execution path the
substitutesAllowed procedure is called, if at all, but this is what I
will try to understand next.
--
Thanks,
Maxim
