offloading documentation and env

Hi,

I am setting up local offloading systems and I just ran

into the same issue like I did every time in the last year:

with guile in the remote 'user' PATH the guile test works *locally*.

Through ssh, the 'env' output is different than local. I'm now trying

to fix this the same way 'scp' is fixed which is by including the

application in the system profile.

This part is implicit in the documentation and not explained, I think

we should be more clear about this.

--

ng0

OpenPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588

https://krosos.org/~/ng0/https://www.infotropique.org

ng0 <ng0@infotropique.org> skribis:

This means that you need to make sure that the target machine has (guix)

in its load path.

The test is to run something like:

$ ssh localhost env |grep GUILE_

GUILE_LOAD_COMPILED_PATH=/home/ludo/.guix-profile/lib/guile/2.2/site-ccache:/home/ludo/.guix-profile/share/guile/site/2.2:/run/current-system/profile/lib/guile/2.2/site-ccache:/run/current-system/profile/share/guile/site/2.2

GUILE_LOAD_PATH=/home/ludo/.guix-profile/share/guile/site/2.2:/run/current-system/profile/share/guile/site/2.2

and you should see /run/current-system/profile/share/guile/site/2.2. If

not, you’ll have to add it somehow.

(It’s unfortunate that setting up offloading remains error-prone, but

that’s a different story!).

HTH,

Ludo’.

Ludovic Courtès transcribed 2.1K bytes:

I assume that you have read the rest of my messages and not just skipped

through them. I thought my messages were clear that (guix) is in the

loadpath, but only *locally* on the machine, not when connecting to it

via ssh. It can be in my users loadpath, but the ssh environment is

different. Comparable to how you only get scp when you put

openssh into the global (packages) and this is also mentioned nowhere.

In other words: What you get in the loadpath when you are a local user,

logged in, differs from what you get in the loadpath when you want to

get the loadpath directly from an command run through ssh connection.

So I'm guessing here: the not so obvious yet very obvious solution is to

put guix into the global (packages)?

Where global means the canonical /etc/config.scm

--

ng0

OpenPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588

infotropique: https://www.infotropique.org

personal: https://ng-0.github.iohttps://krosos.org/

ng0 <ng0@infotropique.org> skribis:

Right, this is why I suggested testing that *with a non-interactive

connection*, as in:

ssh HOST env | grep GUILE

On GuixSD, Guix is always in the global profile.

What does the above give for you?

HTH,

Ludo’.

Ludovic Courtès transcribed 2.8K bytes:

This is issued from computer A (abyayala) to computer B (shadownet).

user@abyayala ~$ ssh shadownet env |grep GUILE_

GUILE_LOAD_COMPILED_PATH=/gnu/store/m91mxi586pi2qshzys9zfsmzij8nf547-profile/lib/guile/2.2/site-ccache:/gnu/store/m91mxi586pi2qshzys9zfsmzij8nf547-profile/share/guile/site/2.2

GUILE_LOAD_PATH=/gnu/store/m91mxi586pi2qshzys9zfsmzij8nf547-profile/share/guile/site/2.2

I would have to put some ssh private keys on there or

configure sshd in a different way to issue this locally

on computer B.

--

ng0

OpenPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588

infotropique: https://www.infotropique.org

personal: https://ng-0.github.iohttps://krosos.org/

ng0 <ng0@infotropique.org> skribis:

[...]

The problem here is that

/run/current-system/profile/share/guile/site/2.2, which is where the

Guix modules are on GuixSD as I wrote above, is missing from the search

path.

The session started when you run “ssh shadownet env” does not spawn a

login shell; thus ~/.profile and similar are *not* sourced. I’m using

Bash, so on my accounts, I have this in .bashrc (‘.bashrc’ is for

non-login shells):

That way, “ssh HOST COMMAND” effectively gets the same environment as a

login shell.

If you’re using a different shell, then make sure its startup file does

something similar.

HTH!

Ludo’.

Ludovic Courtès transcribed 2.2K bytes:

I was aware of this, but I thought we had (guix) available nevertheless

and I was just pushing the wrong buttons.

I use the same on this computer, but at the end of it I source some

files, among them ~/.guix-profile/etc/profile

I would guess that sourcing ~/.guix-profile/etc/profile gets into

the way and that moving this to .bash_profile could fix the issue.

What do you think?

--

ng0

OpenPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588

infotropique: https://www.infotropique.org

personal: https://ng-0.github.iohttps://krosos.org/

ng0 <ng0@infotropique.org> skribis:

[...]

~/.guix-profile/etc/profile won't add /run/current-system/… to the

search path. You really need to source /etc/profile, which in turn will

source ~/.guix-profile/etc/profile (on GuixSD).

HTH,

Ludo’.

Ludovic Courtès transcribed 1.8K bytes:

I think the method as described never really worked.

When I do as you (and the manual) suggested, I no longer

have any results for ssh host env | grep "GUILE_".

When I extend it like this it works. I include the full

paste to show that previously I had the sourcing of /etc/profile

in it:

user@shadownet ~$ cat .bashrc

# Bash initialization for interactive non-login shells and

# for remote shells (info "(bash) Bash Startup Files").

# Export 'SHELL' to child processes. Programs such as 'screen'

# honor it and otherwise use /bin/sh.

export SHELL

if [ -n "$SSH_CLIENT" -a -z "`type -P cat`" ]

then

# We are being invoked from a non-interactive SSH session

# (as in "ssh host command") but 'cat' cannot be found

# in $PATH. Source /etc/profile so we get $PATH and other

# essential variables.

source /etc/profile

fi

# Adjust the prompt depending on whether we're in 'guix environment'.

if [ -n "$GUIX_ENVIRONMENT" ]

then

PS1='\u@\h \w [env]\$ '

else

PS1='\u@\h \w\$ '

fi

source ~/.guix-profile/etc/profile

alias ls='ls -p --color'

alias ll='ls -l'

GUILE_LOAD_COMPILED_PATH="${GUILE_LOAD_COMPILED_PATH}:/run/current-system/profile/lib/guile/2.2/site-ccache:/run/current-system/profile/share/guile/site/2.2"

GUILE_LOAD_PATH="${GUILE_LOAD_PATH}:/run/current-system/profile/share/guile/site/2.2"

I suggest that we fix the offloading documentation.

If questions are asked (I'm not the first) there are obviously problems

with how it is written.

If no one else takes on this, I will.

--

ng0

GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588

GnuPG: https://n0is.noblogs.org/my-keys

infotropique: https://www.infotropique.org

ng0 <ng0@infotropique.org> skribis:

The difference compared to /etc/skel/.bashrc is the last two lines,

right?

On my GuixSD installation they’re not needed because /etc/profile

sources /run/current-system/profile/etc/profile, which already defines

these two variables.

But maybe your global profile is slightly different from mine, which

would explain this. FWIW I have:

Anyway, if you have ideas on how to improve the doc, they’re welcome.

Thanks for your feedback!

Ludo’.

Ludovic Courtès transcribed 2.2K bytes:

Yes. And that I source ~/.guix-profile/etc/profile before those lines.

Well this file is the same on both systems involved:

user@abyayala ~$ cat /etc/profile

# Crucial variables that could be missing in the profiles' 'etc/profile'

# because they would require combining both profiles.

# FIXME: See http://bugs.gnu.org/20255.

export MANPATH=$HOME/.guix-profile/share/man:/run/current-system/profile/share/man

export INFOPATH=$HOME/.guix-profile/share/info:/run/current-system/profile/share/info

export XDG_DATA_DIRS=$HOME/.guix-profile/share:/run/current-system/profile/share

export XDG_CONFIG_DIRS=$HOME/.guix-profile/etc/xdg:/run/current-system/profile/etc/xdg

# Ignore the default value of 'PATH'.

unset PATH

# Load the system profile's settings.

GUIX_PROFILE=/run/current-system/profile \

. /run/current-system/profile/etc/profile

# Prepend setuid programs.

export PATH=/run/setuid-programs:$PATH

# Since 'lshd' does not use pam_env, /etc/environment must be explicitly

# loaded when someone logs in via SSH. See http://bugs.gnu.org/22175.

# We need 'PATH' to be defined here, for 'cat' and 'cut'. Do this before

# reading the user's 'etc/profile' to allow variables to be overridden.

if [ -f /etc/environment -a -n "$SSH_CLIENT" \

-a -z "$LINUX_MODULE_DIRECTORY" ]

then

. /etc/environment

export `cat /etc/environment | cut -d= -f1`

fi

if [ -f "$HOME/.guix-profile/etc/profile" ]

then

# Load the user profile's settings.

GUIX_PROFILE="$HOME/.guix-profile" \

. "$HOME/.guix-profile/etc/profile"

else

# At least define this one so that basic things just work

# when the user installs their first package.

export PATH="$HOME/.guix-profile/bin:$PATH"

fi

# Set the umask, notably for users logging in via 'lsh'.

# See http://bugs.gnu.org/22650.

umask 022

# Allow GStreamer-based applications to find plugins.

export GST_PLUGIN_PATH="$HOME/.guix-profile/lib/gstreamer-1.0"

if [ -n "$BASH_VERSION" -a -f /etc/bashrc ]

then

# Load Bash-specific initialization code.

. /etc/bashrc

fi

user@shadownet ~$ guix package -I gui -p /run/current-system/profile

guix 0.13.0-2.de9d8f0 out /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0

guile 2.2.2 out /gnu/store/1pzfigry5bnh3n146w0ib77vkd2g6jdc-guile-2.2.2

So it is not different.

The system in use is this: https://gitlab.com/ng0/systems/blob/master/servers/pragmatique/shadownet.scm

--

ng0

GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588

GnuPG: https://n0is.noblogs.org/my-keys

infotropique: https://www.infotropique.org

ng0 <ng0@infotropique.org> skribis:

[...]

And does /run/current-system/profile/etc/profile define GUILE_LOAD_PATH?

Ludo'.

Hi,

I'm trying to setup offloading too. Just for the report I had an issue

after setting up `%load-path` for guile with `.bashrc` snippet. It

seems that also required call `guix build` with `--substitute-urls=HOST`

because without it I get `acquired build slot` loop like in

~/.bashrc on magnolia.local

if [ -n "$SSH_CLIENT" -a "`type -P guile`" ]

then

source /etc/profile

fi

natsu@clover ~$ guix offload test

guix offload: testing 1 build machines defined in '/etc/guix/machines.scm'...

guix offload: 'magnolia.local' is running guile (GNU Guile) 2.2.2

guix offload: Guix is usable on 'magnolia.local' (test returned "/gnu/store/883yjkl46dxw9mzykykmbs0yzwyxm17z-test")

sending 1 store item to 'magnolia.local'...

exporting path `/gnu/store/j1xdys0rk60czdfv9riwykdh7iqdhszi-export-test'

guix offload: 'magnolia.local' successfully imported '/gnu/store/j1xdys0rk60czdfv9riwykdh7iqdhszi-export-test'

retrieving 1 store item from 'magnolia.local'...

guix offload: successfully imported '/gnu/store/ij4z815jpvrsmnf2awvmfw8w44vh7ndw-import-test' from 'magnolia.local'

natsu@clover ~$ guix build blender

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

process 5294 acquired build slot '/var/guix/offload/magnolia.local/0'

waiting for locks or build slots...

process 5294 acquired build slot '/var/guix/offload/magnolia.local/0'

process 5294 acquired build slot '/var/guix/offload/magnolia.local/0'

process 5294 acquired build slot '/var/guix/offload/magnolia.local/0'

process 5294 acquired build slot '/var/guix/offload/magnolia.local/0'

process 5294 acquired build slot '/var/guix/offload/magnolia.local/0'

C-c C-c

natsu@clover ~$ ssh magnolia.local cat /proc/loadavg

0.03 0.01 0.00 1/186 2660

natsu@clover ~$ ssh magnolia.local guix package -I gui -p /run/current-system/profile

guile-ssh 0.11.0 out /gnu/store/pzhi01qhc19v5xzfgjbqybnbsd7inicx-guile-ssh-0.11.0

guix 0.13.0-2.de9d8f0 out /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0

guile 2.2.2 out /gnu/store/1pzfigry5bnh3n146w0ib77vkd2g6jdc-guile-2.2.2

emacs-guix 0.3.1 out /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1

natsu@clover ~$ guix build --substitute-urls='http://magnolia.local https://mirror.hydra.gnu.orghttps://hydra.gnu.org' blender

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

The following derivations will be built:

/gnu/store/gkxd3c7pncy0pw8gz6b4w2h4yna5b0qr-blender-2.78a.drv

/gnu/store/4lcmay4nyk9j3v86wpvvncmy1qkzb6lz-fftw-3.3.5.drv

/gnu/store/as3s0ip004kjnkppkp1i56gaakhzcxq1-openimageio-1.6.15.drv

/gnu/store/v9m0icyrwdbn6wq5bncpz5cw7ng3p4dm-ffmpeg-2.8.12.drv

/gnu/store/yw10vy3i1anmwlgg38hwjn2wxica03cx-jemalloc-4.5.0.drv

51.4 MB will be downloaded:

/gnu/store/78dbbi0hqx7irsqcmjzk1xrfh1q8r5ln-jemalloc-4.5.0

/gnu/store/885wcx1sxbqg9f40900xzjpgm9c4lb4i-blender-2.78a

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'http://magnolia.local'... 100.0%

substitute: updating list of substitutes from 'https://mirror.hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

substitute: updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

@ substituter-started /gnu/store/78dbbi0hqx7irsqcmjzk1xrfh1q8r5ln-jemalloc-4.5.0 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/libexec/guix/substitute

updating list of substitutes from 'https://hydra.gnu.org'... 100.0%

Downloading https://hydra.gnu.org/guix/nar/gzip/78dbbi0hqx7irsqcmjzk1xrfh1q8r5ln-jemalloc-4.5.0...

jemalloc-4.5.0 645KiB 758KiB/s 00:01 [####################] 100.0%

@ substituter-succeeded /gnu/store/78dbbi0hqx7irsqcmjzk1xrfh1q8r5ln-jemalloc-4.5.0

@ build-started /gnu/store/v9m0icyrwdbn6wq5bncpz5cw7ng3p4dm-ffmpeg-2.8.12.drv - x86_64-linux /var/log/guix/drvs/v9//m0icyrwdbn6wq5bncpz5cw7ng3p4dm-ffmpeg-2.8.12.drv.bz2

grafting '/gnu/store/sgxg0wprb4bqlnkdfk46rvhq1a0xzfvx-ffmpeg-2.8.12' -> '/gnu/store/ghjryqmclglr8g4jiyyj21g7vdzzj40n-ffmpeg-2.8.12'...

@ build-succeeded /gnu/store/v9m0icyrwdbn6wq5bncpz5cw7ng3p4dm-ffmpeg-2.8.12.drv -

@ substituter-started /gnu/store/885wcx1sxbqg9f40900xzjpgm9c4lb4i-blender-2.78a /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/libexec/guix/substitute

Downloading https://mirror.hydra.gnu.org/guix/nar/gzip/885wcx1sxbqg9f40900xzjpgm9c4lb4i-blender-2.78a...

blender-2.78a 48.4MiB 3.8MiB/s 00:13 [####################] 100.0%

@ substituter-succeeded /gnu/store/885wcx1sxbqg9f40900xzjpgm9c4lb4i-blender-2.78a

@ build-started /gnu/store/4lcmay4nyk9j3v86wpvvncmy1qkzb6lz-fftw-3.3.5.drv - x86_64-linux /var/log/guix/drvs/4l//cmay4nyk9j3v86wpvvncmy1qkzb6lz-fftw-3.3.5.drv.bz2

grafting '/gnu/store/j3pja3bjg7jha04dzhhkb588y3765fql-fftw-3.3.5' -> '/gnu/store/rq8lav2nqxiid8i1v5bml1rq2lfj0lqb-fftw-3.3.5'...

@ build-succeeded /gnu/store/4lcmay4nyk9j3v86wpvvncmy1qkzb6lz-fftw-3.3.5.drv -

@ build-started /gnu/store/yw10vy3i1anmwlgg38hwjn2wxica03cx-jemalloc-4.5.0.drv - x86_64-linux /var/log/guix/drvs/yw//10vy3i1anmwlgg38hwjn2wxica03cx-jemalloc-4.5.0.drv.bz2

grafting '/gnu/store/78dbbi0hqx7irsqcmjzk1xrfh1q8r5ln-jemalloc-4.5.0' -> '/gnu/store/0r54ndf05vigzanx035m78jkackg8z54-jemalloc-4.5.0'...

@ build-succeeded /gnu/store/yw10vy3i1anmwlgg38hwjn2wxica03cx-jemalloc-4.5.0.drv -

@ build-started /gnu/store/as3s0ip004kjnkppkp1i56gaakhzcxq1-openimageio-1.6.15.drv - x86_64-linux /var/log/guix/drvs/as//3s0ip004kjnkppkp1i56gaakhzcxq1-openimageio-1.6.15.drv.bz2

grafting '/gnu/store/ig69hi5lq1zx68hhp6vxkk83bp507ppv-openimageio-1.6.15' -> '/gnu/store/fyi9602hv2wjgcj55qw6f0i6sam854m8-openimageio-1.6.15'...

@ build-succeeded /gnu/store/as3s0ip004kjnkppkp1i56gaakhzcxq1-openimageio-1.6.15.drv -

@ build-started /gnu/store/gkxd3c7pncy0pw8gz6b4w2h4yna5b0qr-blender-2.78a.drv - x86_64-linux /var/log/guix/drvs/gk//xd3c7pncy0pw8gz6b4w2h4yna5b0qr-blender-2.78a.drv.bz2

grafting '/gnu/store/885wcx1sxbqg9f40900xzjpgm9c4lb4i-blender-2.78a' -> '/gnu/store/1q7y82f5af720na9zgsdqm0vyzgszkfb-blender-2.78a'...

@ build-succeeded /gnu/store/gkxd3c7pncy0pw8gz6b4w2h4yna5b0qr-blender-2.78a.drv -

/gnu/store/1q7y82f5af720na9zgsdqm0vyzgszkfb-blender-2.78a

Oleg Pykhalov <go.wigust@gmail.com> writes:

Actually not always works. Still have an issue with loop. Client waits

for `/var/guix/offload/magnolia.local/0` but `/var/guix/offload` is

missing on build machine.

Oleg Pykhalov <go.wigust@gmail.com> writes:

Ah, stupid me, `/var/guix/offload/magnolia.local/0` is on client.

ls -1R /var/guix/offload

/var/guix/offload:

machine-choice.lock

magnolia.local

magnolia.local.slots.lock

/var/guix/offload/magnolia.local:

0

Hi,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

When that happens, could you check how many ‘guix offload’ processes are

running?

Also, are you sure that the build machine is eligible for offloading

(matching system type, normalized load is below 2.0, etc.)?

Thanks,

Ludo’.

ludo@gnu.org (Ludovic Courtès) writes:

Yes, machine is totally free for load.

I captured 1 process as I see:

root 23867 4.9 0.9 128648 35656 ? SLsl 20:01 0:00 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real offload x86_64-linux 0 1 0

Before

======

natsu@clover ~$ ps auxwwww | grep guix

root 341 0.0 0.1 34588 4704 ? Ss Jul03 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

guix-pu+ 433 0.0 0.6 121548 25940 ? Ssl Jul03 0:00 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real publish -u guix-publish -p 80 -C 3 --nar-path=nar --listen=0.0.0.0

root 476 0.0 0.1 34900 4212 ? Ss Jul03 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 433 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

natsu 10446 2.0 17.0 3342808 671380 ? Sl Jul04 30:29 /home/natsu/.guix-profile/bin/icecat

natsu 21472 0.0 3.0 376328 119796 pts/8 Ssl+ 18:59 0:02 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile -L /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1/share/guile/site/2.2 -C /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1/lib/guile/2.2/site-ccache -L /home/natsu/.config/guix/latest -C /home/natsu/.config/guix/latest -L /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/share/guile/site/2.2 -C /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/share/guile/site/2.2 --listen=/tmp/emacs-guix-20138f3K/repl-socket -q -L /gnu/store/620x0ky0cyqihnqx3nrjnqj5xw4zvj67-geiser-0.9/share/geiser/guile/

natsu 21479 0.2 3.9 257152 156592 pts/4 Sl 18:59 0:08 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real environment guix

root 21483 0.1 0.2 45420 8744 ? Ss 18:59 0:06 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 21479 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

root 21484 0.0 1.0 126748 42332 ? Sl 18:59 0:01 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real substitute --query

natsu 23822 0.0 0.0 8168 1464 pts/5 S+ 19:59 0:00 grep guix

natsu@clover ~$ ssh magnolia.local 'ps auxwwww | grep guix'

root 426 0.0 0.0 34584 4820 ? Ss Jul04 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

guix-pu+ 603 0.0 0.1 219164 44260 ? SLsl Jul04 0:02 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real publish -u guix-publish -p 80 -C 3 --nar-path=nar --listen=0.0.0.0

root 635 0.0 0.0 35904 5680 ? Ss Jul04 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 603 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

natsu 4056 0.0 0.0 8076 1416 pts/5 S+ Jul04 0:00 tail -f /var/log/guix-daemon.log

natsu 27391 0.0 0.0 17988 3436 ? Ss 19:59 0:00 bash -c ps auxwwww | grep guix

natsu 27396 0.0 0.0 8164 1540 ? S 19:59 0:00 grep guix

During

======

natsu@clover ~$ ps auxwwww | grep guix

root 341 0.0 0.1 34588 4704 ? Ss Jul03 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

guix-pu+ 433 0.0 0.6 121548 25940 ? Ssl Jul03 0:00 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real publish -u guix-publish -p 80 -C 3 --nar-path=nar --listen=0.0.0.0

root 476 0.0 0.1 34900 4212 ? Ss Jul03 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 433 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

natsu 10446 2.0 17.0 3342808 671200 ? Sl Jul04 30:30 /home/natsu/.guix-profile/bin/icecat

natsu 21472 0.0 3.0 376328 119796 pts/8 Ssl+ 18:59 0:02 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile -L /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1/share/guile/site/2.2 -C /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1/lib/guile/2.2/site-ccache -L /home/natsu/.config/guix/latest -C /home/natsu/.config/guix/latest -L /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/share/guile/site/2.2 -C /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/share/guile/site/2.2 --listen=/tmp/emacs-guix-20138f3K/repl-socket -q -L /gnu/store/620x0ky0cyqihnqx3nrjnqj5xw4zvj67-geiser-0.9/share/geiser/guile/

natsu 21479 0.2 3.9 257152 156592 pts/4 Sl 18:59 0:08 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real environment guix

root 21483 0.1 0.2 45420 8744 ? Ss 18:59 0:06 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 21479 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

root 21484 0.0 1.0 126748 42332 ? Sl 18:59 0:01 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real substitute --query

natsu 23854 33.7 3.5 236004 138320 pts/4 Sl+ 20:01 0:07 /home/natsu/.guix-profile/bin/guile --no-auto-compile /home/natsu/src/guix/scripts/guix build -K t-engine

root 23860 0.7 0.1 36404 6648 ? Ss 20:01 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 23854 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

root 23862 2.5 0.8 123752 35232 ? Sl 20:01 0:00 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real substitute --query

root 23867 4.9 0.9 128648 35656 ? SLsl 20:01 0:00 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real offload x86_64-linux 0 1 0

natsu 23876 0.0 0.0 8168 1556 pts/5 S+ 20:01 0:00 grep guix

natsu@clover ~$ ssh magnolia.local 'ps auxwwww | grep guix'

root 426 0.0 0.0 34584 4820 ? Ss Jul04 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

guix-pu+ 603 0.0 0.1 219164 44260 ? SLsl Jul04 0:02 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real publish -u guix-publish -p 80 -C 3 --nar-path=nar --listen=0.0.0.0

root 635 0.0 0.0 35904 5680 ? Ss Jul04 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 603 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

natsu 4056 0.0 0.0 8076 1416 pts/5 S+ Jul04 0:00 tail -f /var/log/guix-daemon.log

natsu 27432 0.0 0.0 17988 3408 ? Ss 20:01 0:00 bash -c ps auxwwww | grep guix

natsu 27437 0.0 0.0 8164 1476 ? S 20:01 0:00 grep guix

After

=====

natsu@clover ~$ ps auxwwww | grep guix

root 341 0.0 0.1 34588 4704 ? Ss Jul03 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

guix-pu+ 433 0.0 0.6 121548 25940 ? Ssl Jul03 0:00 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real publish -u guix-publish -p 80 -C 3 --nar-path=nar --listen=0.0.0.0

root 476 0.0 0.1 34900 4212 ? Ss Jul03 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 433 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

natsu 10446 2.0 16.8 3326416 664492 ? Sl Jul04 30:33 /home/natsu/.guix-profile/bin/icecat

natsu 21472 0.0 3.0 376328 119796 pts/8 Ssl+ 18:59 0:02 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile -L /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1/share/guile/site/2.2 -C /gnu/store/3ahyhf5fp6jvvln0xw0v1bjka1z6i69m-emacs-guix-0.3.1/lib/guile/2.2/site-ccache -L /home/natsu/.config/guix/latest -C /home/natsu/.config/guix/latest -L /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/share/guile/site/2.2 -C /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/share/guile/site/2.2 --listen=/tmp/emacs-guix-20138f3K/repl-socket -q -L /gnu/store/620x0ky0cyqihnqx3nrjnqj5xw4zvj67-geiser-0.9/share/geiser/guile/

natsu 21479 0.2 3.9 257152 156592 pts/4 Sl 18:59 0:08 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real environment guix

root 21483 0.1 0.2 45420 8744 ? Ss 18:59 0:06 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 21479 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

root 21484 0.0 1.0 126748 42332 ? Sl 18:59 0:01 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real substitute --query

natsu 23922 0.0 0.0 8168 1536 pts/5 S+ 20:04 0:00 grep guix

natsu@clover ~$ ssh magnolia.local 'ps auxwwww | grep guix'

root 426 0.0 0.0 34584 4820 ? Ss Jul04 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon --build-users-group guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

guix-pu+ 603 0.0 0.1 219164 44260 ? SLsl Jul04 0:02 /gnu/store/w0jb15z5sa08fzcapvgyii49cygygyx5-guile-2.2.2/bin/guile --no-auto-compile /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/.guix-real publish -u guix-publish -p 80 -C 3 --nar-path=nar --listen=0.0.0.0

root 635 0.0 0.0 35904 5680 ? Ss Jul04 0:00 /gnu/store/nrd0v38d61l8y16vqkb1gws0bw45q885-guix-0.13.0-2.de9d8f0/bin/guix-daemon 603 guixbuild --max-silent-time 0 --timeout 0 --substitute-urls https://mirror.hydra.gnu.org

natsu 4056 0.0 0.0 8076 1416 pts/5 S+ Jul04 0:00 tail -f /var/log/guix-daemon.log

natsu 27474 0.0 0.0 17988 3540 ? Ss 20:04 0:00 bash -c ps auxwwww | grep guix

natsu 27479 0.0 0.0 8164 1472 ? S 20:04 0:00 grep guix

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Hmm could it be that ‘machine-load’ in (guix scripts offload) always

returns +inf.0, for instance because it fails to run “cat /proc/loadavg”

on the remote machine? (If it succeeded we would see the message “load

on machine XYZ is N”.)

https://git.savannah.gnu.org/cgit/guix.git/tree/guix/scripts/offload.scm#n393

What does “ssh USER@magnolia.local -i KEY cat /proc/loadavg” return,

where USER and KEY are those specified in your machines.scm file?

HTH,

Ludo’.

ludo@gnu.org (Ludovic Courtès) writes:

Can I actually look what is happening somehow?

natsu@clover ~$

(list (build-machine

(name "magnolia.local")

(system "x86_64-linux")

(host-key "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMRy+enQECs2CsjrbSIfnNHExcUwzHFa7KUnhwDIeWOV")

(user "natsu")

(private-key

(string-append (getenv "HOME")

"/.ssh/identity-for-guix"))

(speed 2.)))

ssh natsu@magnolia.local -i $HOME/.ssh/identity-for-guix cat /proc/loadavg

0.05 0.08 0.02 1/247 13756

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Yes, you can make a checkout of Guix, add ‘pk’ calls in (guix scripts

offload) (see below), and run the daemon from there:

./pre-inst-env guix-daemon --build-users-group=guixbuild

‘pk’ is a function that prints its argument and returns it, so you could

do:

to print the line that is read.

Hmm, looks good.

Ludo’.

ludo@gnu.org (Ludovic Courtès) writes:

Note for somebody else having similar issue:

sudo -E ./pre-inst-env guix-daemon --build-users-group=guixbuild

(Skip bellow this stupid text to Solution)

I don't see any sshd message during `guix build` in `/var/log/messages`

on build machine. Not any message in STDOUT on build machine, too.

`/var/log/guix-daemon.log` is silent, too.

I made above diff change on build machine and notebook. On notebook I

get `accepted connection from pid 2576, user natsu` in

`/var/log/guix-daemon.log` AND `process 2587 acquired build slot

'/var/guix/offload/magnolia.local/0` in STDOUT. `/var/log/messages` no

changes.

pk is silent :-(

So, I thought, if no sshd log on build machine, why? I did

`./pre-inst-env guix offload test` and got a `guix offload: testing 0

build machines defined in '/usr/local/etc/guix/machines.scm'...`.

Adding `--prefix=/` in ./configure helped, but I got

'//etc/guix/machines.scm. Maybe need set `--prefix=''`.

`./pre-inst-env guix offload` succeeded. `/var/log/guix-daemon.log`

reports like 'accepted connection from pid 2694, user natsu'

messages. `/var/log/messages` reports about sshd successfully

connections.

But I still failed with `build` command.

I also checked how to get some logs in

https://lists.gnu.org/archive/html/guix-devel/2014-03/msg00264.htmland

made some logs.

.

Solution

========

So, I need a ssh key pair in /root too!

May be put this in doc? I prepared a patch for this.

Hi Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Excellent, glad that you found out. I reworded and pushed as

dde1e152ce42a06fe2998f3da8acb1addfd296fc.

I’m closing this bug now, because it’s been a long discussion about

various issues rather than an actual “bug report”.

Oleg & ng0, please open a new bug report if you find a problem with

offloading. Note that the subject should clearly specify a problem

(“guix foo fails with bar”, “Offloading fails when X and Y”) or an

expectation (“Offloading documentation should describe X”, “guix foo

should not override bar”). The current title does not pass this test.

:-)

TIA,

Ludo’.

tags 27386 notabug

close 27386

unarchive 27386

So I'm trying to set up the same offloading and followed the

documentation and seem to have resulted in the same issue as ng0.

Ludovic Courtès <ludo@gnu.org> writes:

So guixsd.santanas.co.za is my build system VM.

Laptop, is a guixsd system I want to not compile code, and rather offload.

root@laptop ~# ssh admin@guixsd.santanas.co.za env |grep GUILE_

Returns nothing.

However, an interactive session has the correct environment variables.

So I suppose one could set this via the .zshenv or .bash_profile .

Though I'm wondering what others do? Secondly, should the official docs

be updated?

root@guixsd ~# su - admin

admin@guixsd ~$ env|grep GUILE_

GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/2.2/site-ccache:/run/current-system/profile/share/guile/site/2.2

GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/2.2

admin@guixsd ~$

--

Divan

Divan Santana <divan@santanas.co.za> writes:

I'm wondering if for others, by default does ssh a-guixsd-system env

|grep GUILE_ return something?

If so, why on my system, and ng0 is it not set.

--

Divan

Hello Divan,

Divan Santana <divan@santanas.co.za> skribis:

Right. On GuixSD, the default .bash_profile sources .bashrc, and the

default .bashrc has this:

I suggest you do something similar.

HTH!

Ludo’.

On build nodes I added the following variable definitions to

/etc/environment:

GUILE_LOAD_PATH=/run/current-system/profile/share/guile/site/2.2

GUILE_LOAD_COMPILED_PATH=/run/current-system/profile/lib/guile/2.2/site-ccache:/run/current-system/profile/share/guile/site/2.2

You need to make sure that the guile process used by guile SSH is

restarted.

--

Ricardo

GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC

Thanks Ludo.

Though, I do have the default .bashrc and .bash_profile in place and

yet it's not working, by default.

admin@guixsd ~$ diff .bash_profile /etc/skel/.bash_profile

admin@guixsd ~$ diff .bashrc /etc/skel/.bashrc

25a26

> alias grep='grep --color'

admin@guixsd ~$

So I'm assuming this doesn't work by default for all?

Unless it's because I configured the users shell to zsh, I doubt.

Because, when switching it back to bash it still doesn't work. And the

skel files are correctly in place.

It does have your above suggestion in place:

admin@guixsd ~$ cat .bash_profile

# Honor per-interactive-shell startup file

if [ -f ~/.bashrc ]; then . ~/.bashrc; fi

admin@guixsd ~$ cat .bashrc

# Bash initialization for interactive non-login shells and

# for remote shells (info "(bash) Bash Startup Files").

# Export 'SHELL' to child processes. Programs such as 'screen'

# honor it and otherwise use /bin/sh.

export SHELL

if [ -n "$SSH_CLIENT" -a -z "`type -P cat`" ]

then

# We are being invoked from a non-interactive SSH session

# (as in "ssh host command") but 'cat' cannot be found

# in $PATH. Source /etc/profile so we get $PATH and other

# essential variables.

source /etc/profile

fi

--

Divan

Ricardo Wurmus <rekado@elephly.net> writes:

Thanks Ricardo.

So if you do this on build systems, I assume it's kinda a requirement of

sort that should go into the document?

I prefer your suggestion over my local editing of the bash/zsh files.

--

Divan