grub build issue

[SergiiDmytruk]

Following instructions in the README (keeping caching on), I get this error:

ERROR: grub-native-2.06-r0 do_configure: ExecutionError('/build/tmp/work/x86_64-linux/grub-native/2.06/temp/run.do_configure.383', 128, None, None)
ERROR: Logfile of failure stored in: /build/tmp/work/x86_64-linux/grub-native/2.06/temp/log.do_configure.383
Log data follows:
| DEBUG: Executing python function extend_recipe_sysroot
| NOTE: Direct dependencies are ['/work/poky/meta/recipes-core/gettext/gettext-minimal-native_0.22.5.bb:do_populate_sysroot', '/work/poky/meta/recipes-devtools/libtool/libtool-native_2.4.7.bb:do_populate_sysroot', '/work/poky/meta/recipes-devtools/quilt/quilt-native_0.67.bb:do_populate_sysroot', '/work/poky/meta/recipes-extended/texinfo-dummy-native/texinfo-dummy-native.bb:do_populate_sysroot', 'virtual:native:/work/poky/meta/recipes-core/gettext/gettext_0.22.5.bb:do_populate_sysroot', 'virtual:native:/work/poky/meta/recipes-devtools/autoconf/autoconf_2.72e.bb:do_populate_sysroot', 'virtual:native:/work/poky/meta/recipes-devtools/automake/automake_1.16.5.bb:do_populate_sysroot', 'virtual:native:/work/poky/meta/recipes-devtools/bison/bison_3.8.2.bb:do_populate_sysroot', 'virtual:native:/work/poky/meta/recipes-devtools/flex/flex_2.6.4.bb:do_populate_sysroot', 'virtual:native:/work/poky/meta/recipes-devtools/pkgconfig/pkgconfig_git.bb:do_populate_sysroot']
| NOTE: Installed into sysroot: []
| NOTE: Skipping as already exists in sysroot: ['gettext-minimal-native', 'libtool-native', 'm4-native', 'quilt-native', 'texinfo-dummy-native', 'gettext-native', 'autoconf-native', 'automake-native', 'bison-native', 'flex-native', 'gnu-config-native', 'pkgconfig-native', 'xz-native']
| DEBUG: Python function extend_recipe_sysroot finished
| DEBUG: Executing shell function autotools_preconfigure
| DEBUG: Shell function autotools_preconfigure finished
| DEBUG: Executing python function autotools_aclocals
| DEBUG: SITE files ['endian-little', 'common-linux', 'common-glibc', 'bit-64', 'x86_64-linux', 'common']
| DEBUG: Python function autotools_aclocals finished
| DEBUG: Executing shell function do_configure
| /build/tmp/work/x86_64-linux/grub-native/2.06/git/bootstrap: Bootstrapping from checked-out grub sources...
| /build/tmp/work/x86_64-linux/grub-native/2.06/git/bootstrap: consider installing git-merge-changelog from gnulib
| /build/tmp/work/x86_64-linux/grub-native/2.06/git/bootstrap: getting gnulib files...
| Cloning into 'gnulib'...
| fatal: unable to look up git.sv.gnu.org (port 9418) (Temporary failure in name resolution)
| WARNING: exit code 128 from a shell command.

It seems like network is just unavailable at that point of build process, but it's required for build. Any idea why it happens and how can be addressed?

[m-iwanicki]

@SergiiDmytruk which command have you used? Normal build: kas-container build meta-trenchboot/kas-generic-tb.yml?
From which branch/commit?

[SergiiDmytruk]

kas-container build meta-trenchboot/kas-generic-tb.yml on master (827552e).

[m-iwanicki]

Do you use kas-container version 3.0.2?
Can you nmap -p 9418 git.sv.gnu.org from outside and inside kas-container?

kas-container shell meta-trenchboot/kas-generic-tb.yml
sudo apt update
sudo apt install nmap
nmap -p 9418 git.sv.gnu.org

[SergiiDmytruk]

[~]$ kas-container --version
kas-container 3.0.2

Outside container:

[~]$ nmap -p 9418 git.sv.gnu.org
Starting Nmap 7.95 ( https://nmap.org ) at 2024-07-03 17:42 EEST
Nmap scan report for git.sv.gnu.org (209.51.188.168)
Host is up (0.14s latency).
Other addresses for git.sv.gnu.org (not scanned): 2001:470:142::168
rDNS record for 209.51.188.168: vcs2.savannah.gnu.org

PORT     STATE SERVICE
9418/tcp open  git

Nmap done: 1 IP address (1 host up) scanned in 2.78 seconds

Inside container:

builder@093ec895f562:/build$ nmap -p 9418 git.sv.gnu.org
Starting Nmap 7.80 ( https://nmap.org ) at 2024-07-03 14:43 UTC
Nmap scan report for git.sv.gnu.org (209.51.188.168)
Host is up (0.14s latency).
Other addresses for git.sv.gnu.org (not scanned): 2001:470:142::168
rDNS record for 209.51.188.168: vcs2.savannah.gnu.org

PORT     STATE SERVICE
9418/tcp open  git

Nmap done: 1 IP address (1 host up) scanned in 2.80 second

[m-iwanicki]

That's interesting, just now I cleaned grub-native recipe and tried to rebuild and got different error.
From what I can find it can be fixed if you have access to remote repository you are pulling from but as it's official GRUB repo that's impossible.

| DEBUG: Executing shell function do_configure
| /build/tmp/work/x86_64-linux/grub-native/2.06/git/bootstrap: Bootstrapping from checked-out grub sources...
| /build/tmp/work/x86_64-linux/grub-native/2.06/git/bootstrap: consider installing git-merge-changelog from gnulib
| /build/tmp/work/x86_64-linux/grub-native/2.06/git/bootstrap: getting gnulib files...
| Cloning into 'gnulib'...
| remote: warning: suboptimal pack - out of memory
| remote: fatal: Out of memory, malloc failed (tried to allocate 5977375 bytes)
| remote: aborting due to possible repository corruption on the remote side.
| fatal: early EOF
| fatal: index-pack failed
| WARNING: exit code 128 from a shell command.
ERROR: Task (virtual:native:/work/poky/meta/recipes-bsp/grub/grub_2.12.bb:do_configure) failed with exit code '1'
NOTE: Tasks Summary: Attempted 4444 tasks of which 4439 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  virtual:native:/work/poky/meta/recipes-bsp/grub/grub_2.12.bb:do_configure
Summary: There was 1 WARNING message.
Summary: There was 1 ERROR message, returning a non-zero exit code.

You can temporarily fix this by cloning gnulib repo:

cd build/tmp/work/x86_64-linux/grub-native/2.06/git
git clone git://git.sv.gnu.org/gnulib

It worked for me when I did this outside of kas-container. Tomorrow I'll check more.
And can you try to build again to see If you get same/different error?

[SergiiDmytruk]

It was still fatal: unable to look up git.sv.gnu.org (port 9418) (Temporary failure in name resolution) on a retry.

Cloning gnulib to build/tmp/work/core2-64-tb-linux/grub/2.06/git/gnulib and build/tmp/work/core2-64-tb-linux/grub-efi/2.06/git/gnulib (not grub-native) resulted in a different error:

[...]
| gnulib/gnulib-tool: *** file /build/tmp/work/core2-64-tb-linux/grub/2.06/git/gnulib/m4/lib-ld.m4 not found
| gnulib/gnulib-tool: *** Stop.
| /build/tmp/work/core2-64-tb-linux/grub/2.06/git/bootstrap: gnulib-tool failed

That file and some others in/m4 actually get removed during build, but I guess they need to be there initially.
After checking out d271f868a8df9bbec29049d01e056481b7a1a263 (as in bootstrap.conf), retrying several times and doing reset --hard in kas-container shell I was able to finish the build. Thanks for the pointers with path and shell.

---

It could be one of weird issues of docker, seems quite common: https://stackoverflow.com/questions/44761246/temporary-failure-in-name-resolution-errno-3-with-docker. That said I did try to hard-code address in /etc/hosts and use host network and then got an issue connecting to that address, so it wasn't just DNS resolution. Next time I'll try to restart docker first (there was no reboot since yesterday, so no restart between retries).

If you won't find anything obvious quickly, just close this as it's probably unrelated to the contents of the repository. Could even be some kernel container issue.

[m-iwanicki]

@SergiiDmytruk This new error you got after cloning to grub and grub-efi is most likely related to #31 issue.

Does the problem still persist? I just build TrenchBoot from zero and it completed successfully.

[SergiiDmytruk]

This new error you got after cloning to grub and grub-efi is most likely related to #31 issue.

That sounds like it and there is a better workaround there.

Does the problem still persist? I just build TrenchBoot from zero and it completed successfully.

I still get the error and restarting docker doesn't help, but it's probably some local thing.

[macpijan]

Are you still expieriencing it?

| fatal: unable to look up git.sv.gnu.org (port 9418) (Temporary failure in name resolution)

It means we clone some stuffing during the do_configure phase?
I am pretty sure it would not be welcome in the upstream Yocto layers, or it comes from the upstream in fact @m-iwanicki ?

The general design of building Yocto is that you should be able to complete the build off-network as long as you have the downloads directory in place already. See e.g. https://docs.yoctoproject.org/dev-manual/building.html?highlight=bb_no_network#replicating-a-build-offline

So I was going to say, we should be able to solve this error via cache, but if it happens in do_configure it sounds like the recipe is not perfect.

[macpijan]

I am trying to replicate this locally, the bootstrap seem to be stuck forever:

./bootstrap 
./bootstrap: Bootstrapping from checked-out grub sources...
./bootstrap: consider installing git-merge-changelog from gnulib
./bootstrap: getting gnulib files...
Cloning into 'gnulib'...

[macpijan]

Tried again, it did clone but it took a very long time to actually start cloning;

./bootstrap 
./bootstrap: Bootstrapping from checked-out grub sources...
./bootstrap: consider installing git-merge-changelog from gnulib
./bootstrap: getting gnulib files...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
Initialized empty Git repository in /home/macpijan/projects/trenchboot/grub/gnulib/.git/
warning: no common commits
remote: Counting objects: 254507, done.
remote: Compressing objects: 100% (30480/30480), done.
remote: Total 254507 (delta 224518), reused 252827 (delta 223956)
Receiving objects: 100% (254507/254507), 57.67 MiB | 11.31 MiB/s, done.
Resolving deltas: 100% (224518/224518), done.
From https://git.savannah.gnu.org/git/gnulib
 * branch                      9f48fb992a3d7e96610c4ce8be969cff2d61a01b -> FETCH_HEAD
HEAD is now at 9f48fb992a3d filevercmp: fix several unexpected results
Note: switching to '9f48fb992a3d7e96610c4ce8be969cff2d61a01b'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 9f48fb992a3d filevercmp: fix several unexpected results

[macpijan]

@SergiiDmytruk perhaps this will help a bit: #36

[SergiiDmytruk]

It means we clone some stuffing during the do_configure phase?

That explains the failure, although it's still a bit of a mystery why it didn't work for me specifically. Maybe use of build cache changes something but build with cache must have been tested as well, so who knows.

#36 does fix the build for me, thank you!