Bootstrap packages fail to build due to mes-libc lacking 'stat64' etc. syscalls

  • Open
  • quality assurance status badge
Details
6 participants
  • Andreas Enge
  • Bengt Richter
  • Carl Dong
  • Jan Nieuwenhuizen
  • Ludovic Courtès
  • Mathieu Othacehe
Owner
unassigned
Submitted by
Carl Dong
Severity
important
Merged with
C
C
Carl Dong wrote on 10 Aug 2021 21:41
bash-mesboot0: Inscrutable error in build phase
(address . bug-guix@gnu.org)
8DDCC02E-5D4A-4DF1-A893-229B58EBE31C@carldong.me
Hi all,

While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299

The error line is L1299: "make: stat:Makefile: sterror: unknown error”

This failure persists even when ran with --cores=1, across reboots. I’ve even tried building that derivation with --check on my own machine (which successfully built that derivation), and every log line matched (plus or minus some timing log lines) up until L1299.

We’ve also tried rebuilding bash-mesboot0-2.05b.drv’s dependency /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv with:

Toggle snippet (4 lines)
guix gc --delete /gnu/store/prkqai3zwh3shlqpll6xyncmmqpj49dd-gash-boot-0.2.0
guix build --cores=1 /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv

Which did not make a difference.

What is interesting is that this is not the first time this problem has cropped up for people. I see 2 past mentions of this exact error:
- Mentioned by kagevf on Dec 16th, 2020: https://logs.guix.gnu.org/guix/2020-12-16.log#043712
- Mentioned by genr8_ (with help from rekado_) on Mar 22nd, 2021: https://logs.guix.gnu.org/guix/2021-03-22.log#182406

I’m happy to spend energy investigating, but would love any pointers on what the most promising place to look is!

My friend's system is an Intel i5 system, running Guix 1.3.0 on Ubuntu. His /tmp is on the same partition as / and is ext4.

Cheers,
Carl Dong
contact@carldong.me
"I fight for the users"
B
B
Bengt Richter wrote on 11 Aug 2021 02:38
(name . Carl Dong)(address . contact@carldong.me)(address . 49985@debbugs.gnu.org)
20210811003854.GA26014@LionPure
On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
Toggle quote (7 lines)
> Hi all,
>
> While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
>
> The error line is L1299: "make: stat:Makefile: sterror: unknown error”
>--

To me, sterror looks like a typo for strerror
so grepping for sterror might find the typo, if that's what it is?

Regards,
Bengt Richter
B
B
Bengt Richter wrote on 11 Aug 2021 03:21
(name . Carl Dong)(address . contact@carldong.me)(address . 49985@debbugs.gnu.org)
20210811012118.GB26014@LionPure
On +2021-08-11 02:38:54 +0200, Bengt Richter wrote:
Toggle quote (11 lines)
> On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
> > Hi all,
> >
> > While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
> >
> > The error line is L1299: "make: stat:Makefile: sterror: unknown error”
> >--
>
> To me, sterror looks like a typo for strerror
> so grepping for sterror might find the typo, if that's what it is?
>
Looking at ./lib/mes/__mes_debug.c, it looks like you could set/export environment
MES_DEBUG=1 and run it again to see the error number ./lib/string/strerror.c
(in git repo) doesn't like.

Then run something like my error-printing kludge that will work on your system:

I called it errno-grep
Toggle snippet (15 lines)
#!/usr/bin/bash
if [ -z "$1" ];then
echo "Usage: errno-grep [ grep switches for grepping the following files in order ]"
echo " /usr/include/asm-generic/errno-base.h"
echo " /usr/include/asm-generic/errno.h"
exit 0
fi

if [ "${1:0:1}" == "-" ]; then
exec grep "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
else
exec egrep -wh "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
fi

HTH.
B
B
Bengt Richter wrote on 11 Aug 2021 14:53
(name . Carl Dong)(address . contact@carldong.me)(address . 49985@debbugs.gnu.org)
20210811125317.GA49029@LionPure
Just this disclaimer:
I am just a lurker interested in mes, not a mes developer or team member.
(I am trying to do some minimal stuff too, so I cloned the mes repo and pull once
in a while to see what's going on :)

Sorry for any implication that I really know anything about mes ;/

On +2021-08-11 03:21:18 +0200, Bengt Richter wrote:
Toggle quote (40 lines)
> On +2021-08-11 02:38:54 +0200, Bengt Richter wrote:
> > On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
> > > Hi all,
> > >
> > > While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
> > >
> > > The error line is L1299: "make: stat:Makefile: sterror: unknown error”
> > >--
> >
> > To me, sterror looks like a typo for strerror
> > so grepping for sterror might find the typo, if that's what it is?
> >
> Looking at ./lib/mes/__mes_debug.c, it looks like you could set/export environment
> MES_DEBUG=1 and run it again to see the error number ./lib/string/strerror.c
> (in git repo) doesn't like.
>
> Then run something like my error-printing kludge that will work on your system:
>
> I called it errno-grep
> --8<---------------cut here---------------start------------->8---
> #!/usr/bin/bash
> if [ -z "$1" ];then
> echo "Usage: errno-grep [ grep switches for grepping the following files in order ]"
> echo " /usr/include/asm-generic/errno-base.h"
> echo " /usr/include/asm-generic/errno.h"
> exit 0
> fi
>
> if [ "${1:0:1}" == "-" ]; then
> exec grep "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
> else
> exec egrep -wh "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
> fi
> --8<---------------cut here---------------end--------------->8---
>
> HTH.
>
>
>

--
Regards,
Bengt Richter
C
C
Carl Dong wrote on 12 Aug 2021 19:04
(name . Bengt Richter)(address . bokr@bokr.com)(address . 49985@debbugs.gnu.org)
23A9E6EC-1AE7-4412-82E0-F5930059B2D3@carldong.me
No worries! We’re all learning :-)
I ended up asking my colleague to use:

strace --env=MES_DEBUG=1

We’ll see what happens!

Toggle quote (53 lines)
> On Aug 11, 2021, at 8:53 AM, Bengt Richter <bokr@bokr.com> wrote:
>
> Just this disclaimer:
> I am just a lurker interested in mes, not a mes developer or team member.
> (I am trying to do some minimal stuff too, so I cloned the mes repo and pull once
> in a while to see what's going on :)
>
> Sorry for any implication that I really know anything about mes ;/
>
> On +2021-08-11 03:21:18 +0200, Bengt Richter wrote:
>> On +2021-08-11 02:38:54 +0200, Bengt Richter wrote:
>>> On +2021-08-10 15:41:25 -0400, Carl Dong wrote:
>>>> Hi all,
>>>>
>>>> While setting up Guix for a community member of mine, we encountered this somewhat inscrutable problem (I later learned this is not the first time Guix users have run into this problem!). When building /gnu/store/2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.drv, we encountered the following build failure: https://paste.sr.ht/~dongcarl/376b19b8349c329ed5329508c7fb43a7c3aec64b#2nvaxgs0rdxfkrwklh622ggaxg0wap6n-bash-mesboot0-2.05b.log-L1299
>>>>
>>>> The error line is L1299: "make: stat:Makefile: sterror: unknown error”
>>>> --
>>>
>>> To me, sterror looks like a typo for strerror
>>> so grepping for sterror might find the typo, if that's what it is?
>>>
>> Looking at ./lib/mes/__mes_debug.c, it looks like you could set/export environment
>> MES_DEBUG=1 and run it again to see the error number ./lib/string/strerror.c
>> (in git repo) doesn't like.
>>
>> Then run something like my error-printing kludge that will work on your system:
>>
>> I called it errno-grep
>> --8<---------------cut here---------------start------------->8---
>> #!/usr/bin/bash
>> if [ -z "$1" ];then
>> echo "Usage: errno-grep [ grep switches for grepping the following files in order ]"
>> echo " /usr/include/asm-generic/errno-base.h"
>> echo " /usr/include/asm-generic/errno.h"
>> exit 0
>> fi
>>
>> if [ "${1:0:1}" == "-" ]; then
>> exec grep "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
>> else
>> exec egrep -wh "$@" /usr/include/asm-generic/errno-base.h /usr/include/asm-generic/errno.h
>> fi
>> --8<---------------cut here---------------end--------------->8---
>>
>> HTH.
>>
>>
>>
>
> --
> Regards,
> Bengt Richter
M
M
Mathieu Othacehe wrote on 19 Aug 2021 18:20
(name . Carl Dong)(address . contact@carldong.me)(address . 49985@debbugs.gnu.org)
87lf4xflu0.fsf@gnu.org
Hello Carl,

Toggle quote (2 lines)
> The error line is L1299: "make: stat:Makefile: sterror: unknown error”

This reminds me of:

I never took the time to fix this issue. Bottom line is that building
the bootstrap toolchain fails on NVME disks because some syscalls
(stat64, lstat64 and fstat64) need to be implemented in GNU Mes.

There's a small demonstration program that you can use to demonstrate
this theory :).

Thanks,

Mathieu
C
C
Carl Dong wrote on 25 Aug 2021 20:37
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 49985@debbugs.gnu.org)
004F2E7A-22C7-4C54-9524-DAE0E71EF9C1@carldong.me
Mathieu,

I think this was exactly the problem, because mounting a tmpfs at /tmp solved it. Thanks for your help!

Cheers,
Carl Dong

Toggle quote (20 lines)
> On Aug 19, 2021, at 12:20 PM, Mathieu Othacehe <othacehe@gnu.org> wrote:
>
>
> Hello Carl,
>
>> The error line is L1299: "make: stat:Makefile: sterror: unknown error”
>
> This reminds me of:
> https://lists.gnu.org/archive/html/bug-guix/2020-05/msg00335.html.
>
> I never took the time to fix this issue. Bottom line is that building
> the bootstrap toolchain fails on NVME disks because some syscalls
> (stat64, lstat64 and fstat64) need to be implemented in GNU Mes.
>
> There's a small demonstration program that you can use to demonstrate
> this theory :).
>
> Thanks,
>
> Mathieu
L
L
Ludovic Courtès wrote on 1 Sep 2021 15:56
control message for bug #41264
(address . control@debbugs.gnu.org)
87wno0761w.fsf@gnu.org
merge 41264 49985
quit
L
L
Ludovic Courtès wrote on 1 Sep 2021 15:59
(address . control@debbugs.gnu.org)
87v93k75xu.fsf@gnu.org
retitle 41264 Bootstrap packages fail to build due to mes-libc lacking 'stat64' etc. syscalls
quit
L
L
Ludovic Courtès wrote on 1 Sep 2021 15:59
(address . control@debbugs.gnu.org)
87tuj475xp.fsf@gnu.org
severity 41264 important
quit
J
J
Jan Nieuwenhuizen wrote on 13 Feb 2023 12:28
Re: bug#49985: Bootstrap packages fail to build due to mes-libc lacking 'stat64' etc. syscalls
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87bklxztvt.fsf_-_@gnu.org
Jan Nieuwenhuizen writes:

Hello!

Toggle quote (1 lines)
> Mathieu Othacehe writes:
[..]
Toggle quote (10 lines)
>> lib/linux/stat.c should be modified this way:
>>
>> #if __i386__
>> #define STAT_SYSCALL SYS_stat64
>> #else
>> #define STAT_SYSCALL SYS_stat
>> #endif
>
> Ah...the stat64 syscall is meant for i386; now it at starts making at
> least some sense to me.
[..]
Toggle quote (5 lines)
>> This way, we would replicate the glibc behavior.
>
> Beautiful, thanks for getting to the bottom of this. Now that you
> already have gone this far, would you like to whip-up a full patch for
> GNU Mes?
[..]

To use stat64 and friends on 32bit, I created the attached patch for GNU
Mes and hope to create a 0.24.2 release from


Also, I have update my core-updates branch with preliminary 0.24.2 mes
and mes-boot packages here


Greetings,
Janneke
Attachment: 0001-lib-stat-Use-SYS_stat64-for-32bit-platforms.patch
J
J
Janneke Nieuwenhuizen wrote on 15 Feb 2023 09:45
(name . Mathieu Othacehe)(address . othacehe@gnu.org)
87a61fgvuu.fsf@gnu.org
Jan Nieuwenhuizen writes:

Hello,

Toggle quote (10 lines)
> To use stat64 and friends on 32bit, I created the attached patch for GNU
> Mes and hope to create a 0.24.2 release from
>
> https://gitlab.com/janneke/mes/-/tree/wip-stat64
>
> Also, I have update my core-updates branch with preliminary 0.24.2 mes
> and mes-boot packages here
>
> https://gitlab.com/janneke/guix/-/tree/core-updates

I've got a confirmation this works*, have released 0.24.2 and updated
mes-boot on core-updates as
b928e38bd333e6186727fe5c5e94b85d157b79d6

Hoping to finally close these bugs!

Greetings,
Janneke


--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
J
J
Janneke Nieuwenhuizen wrote on 16 Feb 2023 16:03
Re: Merging core-updates?
(name . Andreas Enge)(address . andreas@enge.fr)
87h6vlfya4.fsf@gnu.org
Andreas Enge writes:

Toggle quote (7 lines)
> Am Wed, Feb 15, 2023 at 09:39:39AM +0100 schrieb Janneke Nieuwenhuizen:
>> I have released 0.24.2 and updated mes-boot on core-updates as
>> Let's hope this fixes these bugs.
>
> With your latest patch, I have successfully bootstrapped core-updates
> on x86_64 up to hello and mpc. Thanks a lot!

Great, thanks so much for checking! Are you using any of tmpfs or btrfs
on /tmp?

Greetings,
janneke

--
Janneke Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond https://LilyPond.org
Freelance IT https://www.JoyOfSource.com| Avatar® https://AvatarAcademy.com
A
A
Andreas Enge wrote on 16 Feb 2023 16:24
(name . Janneke Nieuwenhuizen)(address . janneke@gnu.org)
Y+5KklptkTuzW+dd@jurong
Am Thu, Feb 16, 2023 at 04:03:15PM +0100 schrieb Janneke Nieuwenhuizen:
Toggle quote (3 lines)
> Great, thanks so much for checking! Are you using any of tmpfs or btrfs
> on /tmp?

No, it is all on SSD, so we probably cannot conclude for the bugs,
unfortunately.

Andreas
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 49985@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 49985
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch