Skip to content

Latest commit

 

History

History
1891 lines (1444 loc) · 63.4 KB

HACKING.org

File metadata and controls

1891 lines (1444 loc) · 63.4 KB

Pjotr’s hacking guide to GNU Guix

Table of Contents

Introduction

‘You are in a maze of twisty packages all alike…’

Hacking GNU Guix is an adventure. Not least because it is using Scheme LISP in the GNU Ubiquitous Intelligent Language for Extensions implementation, also known as Guile. You are encouraged to dive in to LISP, a language that is so good it simply refuses to go away.

GNU Guix stands out as the ‘hackable’ package manager. Mostly because it uses a powerful high-level programming language.

This document should get you started with Guile and GNU Guix. Just pick any package description in the ./gnu directory and work from there. The examples here are pulled from the ruby.scm package.

Once you have a running GNU Guix (see INSTALL), the next step is to compile a package from a recent git checkout of the sources. Check out the source tree following the instructions in the manual. Here we start from a checked out GNU Guix git repository.

First a note of caution. Try to work one step at a time. GNU Guix is not a simple system, so you are bound to get frustrated. But it is hackable, that means you can solve any problem! And the reward is sane software deployment. Guix will pay back your investment.

Before hacking GNU Guix it may be worth contemplating the speed of the network connection: in short, the faster the better. Caching packages somewhere may be worth considering too. Finally a fast server may be a good thing too because GNU Guix is designed to build packages in parallel.

Guile: the language

Guile is a Scheme LISP. Here we list some Scheme specific material that is used by GNU Guix. There is much information on the web on Scheme. Check out Scheme at a Glance, for example.

Functions vs procedures

In the Scheme world, we prefer to use the word `procedure’ instead of `function’, to reflect the fact that it is not merely a mapping from inputs to outputs but can perform side effects as well.

In this document I carelessly mix the two terms, you are warned.

Boolean #f, #t notation

#f signifies false, #t signifies true.

Hash colon (#:) notation

The #: signifies literal keyword syntax and is used to create unique identifiers, see also

http://practical-scheme.net/wiliki/schemexref.cgi?keyword%3F

in Ruby, for comparison, #:key would be in colon notation :key (which is known as a symbol in Ruby, but differs from a symbol in LISP).

Percentage (%) notation

The percentage is a syntactical name helper used to generate and create values available in scope. For example, the output file name is reified via the %output variable automatically added to builder’s scope. Input file names are similarly reified through the %build-inputs variable. Both variables are non-hygienically introduced in the build expression by build-expression->derivation.

Note that it is merely a convention, like ‘_’ in C. Scheme LISP treats ‘%’ exactly the same as any other letter.

Key-values

GNU Guix uses key-value pairs extensively. With

(build-system
  (name 'ruby)
  (description "The standard Ruby build system")
  (lower lower))

the Guix build-system record constructor is called with the field names name, description and lower, where the last is a function with the same name. These definitions are actually resolved as Guile records can be found in ./guix/packages.scm. Look up ‘define-record-type*’ defined in ./guix/build-system.scm to see how that works (the asterisk * implies that fields are bound as per letrec*, allowing them to refer to one another):

(define-record-type* <build-system> build-system make-build-system
  build-system?
  (name        build-system-name)         ; symbol
  (description build-system-description)  ; short description
  (lower       build-system-lower))       ; args ... -> bags

;; "Bags" are low-level representations of "packages".  The system and target
;; of a bag is fixed when it's created.  This is because build systems may
;; choose inputs as a function of the system and target.
(define-record-type* <bag> bag %make-bag
  bag?
  (name          bag-name)               ;string

  (system        bag-system)             ;string
  (target        bag-target              ;string | #f
                 (default #f))

  ;; Here we use build/host/target in the sense of the GNU tool chain (info
  ;; "(autoconf) Specifying Target Triplets").
  (build-inputs  bag-build-inputs        ;list of packages
                 (default '()))
  (host-inputs   bag-host-inputs         ;list of packages
                 (default '()))

  ;; "Target inputs" are packages that are built natively, but that are used
  ;; by target programs in a cross-compilation environment.  Thus, they act
  ;; like 'inputs' as far as search paths are concerned.  The only example of
  ;; that is the cross-libc: it is an input of 'cross-gcc', thus built
  ;; natively; yet, we want it to be considered as a target input for the
  ;; purposes of $CPATH, $LIBRARY_PATH, etc.
  (target-inputs bag-target-inputs
                 (default '()))
  (outputs       bag-outputs             ;list of strings
                 (default '("out")))
  (arguments     bag-arguments           ;list
                 (default '()))
  (build         bag-build))             ;bag -> derivation

In GNU Guix the record data is available as build-system-name, build-system-description etc. Same for the package record which delivers package-name, package-version, etc.

Also literal keyword syntax is used, e.g.,

(build-expression->derivation store name builder
                              #:inputs inputs
                              #:system system
                              #:modules imported-modules
                              #:outputs outputs
                              #:guile-for-build guile-for-build))

calls build-expression->derivation (note that Guile can use more than alphanum characters to create a function name) with parameters store, name, builder and a list of variable key-value pairs named #:inputs, inputs etc. The idea is that the number of parameters is variable to the build-expression->derivation function.

Defining a function

define and define* are used to define functions - well actually to bind identifiers to any value. Note that functions are defined in a module or function local scope. define-module at the top of a package can export functions, e.g.

(define-module (guix build-system ruby)
  #:use-module (guix store)
  #:export (ruby-build
            ruby-build-system))

The difference between define and define* is that the latter can handle variable length parameter lists.

A thing to note is that every LISP function returns a value, i.e., the last expression evaluated.

Defining a variable

let and let* allow defining multiple variables in scope. The difference between let and let* is that let* guarantees sequential initialization, so you can cross-reference values in the list. The more important difference between let and let* is that let* allows the initializers of later variables to refer to the earlier variables, whereas the initializers of let only see variables outside of the let. For example:

 (let ((a 1) (b 2))
   (let ((b a) (a b))
     (list a b)))

returns (2 1), but if the inner let is replaced with let*, then it
will return (1 1).

Inside functions

One thing to note is the extensive use of backquote in GNU Guix. Backquote (quasiquote in Scheme LISP jargon) is like quote, but selected subexpressions are evaluated. These are assigned with a comma (an unquote), e.g.

(ruby-build #:name ,name
            #:source ,(match (assoc-ref inputs "source")
                        (((? derivation? source))
                         (derivation->output-path source))
                        ((source)
                         source)
                        (source
                         source))
            #:system ,system
            #:test-target ,test-target
            #:tests? ,tests?
            #:phases ,phases)

Note match operator which is used for expression matching. Here ‘source’ is matched to pull out the source path and generate a #:source key-value pair.

When ,@ is used (shorthand for unquote-splicing), e.g. in

(host-inputs `(,@(if source
                     `(("source" ,source))
                     '())
               ,@inputs

               ;; Keep the standard inputs of 'gnu-build-system'.
               ,@(standard-packages)))

it indicates an expression to be evaluated and the elements of the returned list inserted (the resulting list is ‘spliced in’).

More about Guile/scheme

Use your editor to jump to function definitions inside the GNU Guix source tree. With emacs you can use ‘ctags -R -e’ in the base directory and load the TAGS file. Jump to a tag with M-x find-tag. If that does not find the tag, look the function up in the Guile manual.

Guile/scheme is a minimalistic implementation of LISP (though Guile is moderately large for a Scheme). This means it is pretty easy to learn the language. To read up on available functionality, read the Guile manual online or in PDF. The procedure index contains all available function calls for the language.

Running Guile stand-alone is easy using a command line REPL or inside emacs. That allows you to play with language features, as well as call GNU Guix functionality directly.

Emacs helpers

Emacs has a range of packages to deal with Lisp syntax. My favorites are:

  • Rainbow delimiters (color brackets)
  • Guile mode
  • Paredit (auto bracket completion)
  • Geiser (REPL)

More tips here.

Guix: the language

GNU Guix is not a language per se. But as they say, LISP is used to create a new language for every purpose (using macros). So here we list some of the commonly used macros.

Bags

Recently GNU Guix introduced bags as an intermediate form between packages and derivations. A bag includes all the implicit inputs which is useful for processing.

Expressions

A good explanation of expressions (a derivation in Nix-speak) and how they are implemented can be found on Wikisource. Actually at the low level an expression returns a derivation variable or structure. s-expressions (sexprs or sexps) are a notation for nested list data similar to JSON but fully Lisp which uses it both for data and source code. G-expressions are Guix expressions which can expand paths in the store and act similar to backquote and comma for list expansion - but use ‘#~’ and ‘#$’ instead. G-expr can be used to generate derivations.

Renaming and moving files

Replace the install phase with a function that adds /bin to outputs and makes sure to make the directory and copy a file named mpc123 into bin:

;...
    (build-system gnu-build-system)
    (arguments
     '(#:phases
       (modify-phases %standard-phases
         (delete 'check) ;; Don't run the 'make check' step of the gnu-build-system
         (replace 'install ;; Replace the install step with the function defined below
           (lambda* (#:key outputs #:allow-other-keys)
             (let* ((out (assoc-ref outputs "out"))
                    (bin (string-append out "/bin")))
               (mkdir-p bin)
               (copy-file "mpc123" (string-append bin "/mpc123"))))))))
;...

Starting the daemon

Do not forget to start the daemon

guix-daemon --build-users-group=guixbuild

The daemon runs ar root, the actual build processes run as unprivileged users.

Install Guix from the git repository

See the section Building GNU Guix from source in INSTALL.

Creating a package

Updating the version

The version is located in the package definition. E.g.

(define-public ruby-2.1
  (package (inherit ruby)
    (version "2.1.6")
    (source
     (origin
       (method url-fetch)
       (uri (string-append "http://cache.ruby-lang.org/pub/ruby/"
                           (version-major+minor version)
                           "/ruby-" version ".tar.bz2"))
       (sha256
        (base32
         "1r4bs8lfwsypbcf8j2lpv3by40729vp5mh697njizj97fjp644qy"))))))

Updating the HASH value

guix download http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.3.tar.gz

if you have downloaded a package or checked out a git repo you can also do

~/.config/guix/current/bin/guix hash /gnu/store/ddg95a3q30qiiqz4gdkmmldj46s9bfmp-gemma-gn2-0.98-6b1e007-checkout -r

Use import to convert other packages from GNU, pypi, rubygems and Nix

Guix can read package definitions from other sources and write a Guix expression to stdout. Make sure gnutls is installed (to avoid a JSON error) and

guix package -i gnutls
guix import pypi readline

prints out

(package
  (name "python-readline")
  (version "6.2.4.1")
  (source
    (origin
      (method url-fetch)
      (uri (string-append
             "https://pypi.python.org/packages/source/r/readline/readline-"
             version
             ".tar.gz"))
      (sha256
        (base32
          "01yi9cls19nglj0h172hhlf64chb0xj5rv1ca38yflpy7ph8c3z0"))))
  (build-system python-build-system)
  (inputs
    `(("python-setuptools" ,python-setuptools)))
  (home-page
    "http://github.com/ludwigschwardt/python-readline")
  (synopsis
    "The standard Python readline extension statically linked against the GNU readline library.")
  (description
    "The standard Python readline extension statically linked against the GNU readline library.")
  (license #f))

Dependencies

All software (except for the Linux kernel) depends on other software to build or to run. Guix keeps track of them and by adding a dependency all underlying dependencies get pulled in too. The build systems will pull in the usual dependencies, but often you need to specify a few more. Guix understands the following inputs

  1. native-inputs: required for building but not runtime - installing a package through a substitute won’t install these inputs
  2. inputs: installed in the store but not in the profile, as well as being present at build time
  3. propagated-inputs: installed in the store and in the profile, as well as being present at build time

Building the package

From a prebuilt guix in the source tree one can start with

./pre-inst-env guix package -A ruby
  ruby    1.8.7-p374      out     gnu/packages/ruby.scm:119:2
  ruby    2.1.6   out     gnu/packages/ruby.scm:91:2
  ruby    2.2.2   out     gnu/packages/ruby.scm:39:2

to see if the package compiles. Note that Guix contains three versions of Ruby! Next try the explicit package compile which should return the destination

./pre-inst-env guix build -K -e '(@ (gnu packages ruby) ruby-2.1)'
/gnu/store/c13v73jxmj2nir2xjqaz5259zywsa9zi-ruby-2.1.6

Debugging the package

Log files

You can find the log files generated during the build process with

guix build --log-file something

Ricardo wrote: does this also work for failed builds - without rebuilding it again? It does seem to work. To test this I added (error “foo”) to a build phase in the “diamond” package and ran

  guix package -i diamond

This ends with

  Build failed:  /gnu/store/wk9qbhmdzs62mp40casrndcgm3p50m3b-diamond-0.9.22.drv
  guix package: error: build failed: build of `/gnu/store/wk9qbhmdzs62mp40casrndcgm3p50m3b-diamond-0.9.22.drv' failed

So I ran

  guix build --log-file /gnu/store/wk9qbhmdzs62mp40casrndcgm3p50m3b-diamond-0.9.22.drv

which gave me

  /var/log/guix/drvs/wk/9qbhmdzs62mp40casrndcgm3p50m3b-diamond-0.9.22.drv.bz2

which contains the build log for this failed build, including the “foo” error message.

I would like this error log file location to be shown unprompted, but I think we would need to change build.cc, so that BuildError prints it in addition to the error message.

Environment

Before debugging it is important to have a clean environment.

You can view the environment variable definitions Guix recommends with

guix package --search-paths

Mine looks like:

set|grep guix
  ACLOCAL_PATH=/home/pjotr/.guix-profile/share/aclocal
  BASH=/home/pjotr/.guix-profile/bin/bash
  CPATH=/home/pjotr/.guix-profile/include
  GUILE_LOAD_COMPILED_PATH=/home/pjotr/.guix-profile/share/guile/site/2.0
  GUILE_LOAD_PATH=/home/pjotr/.guix-profile/share/guile/site/2.0
  LIBRARY_PATH=/home/pjotr/.guix-profile/lib
  LOCPATH=/home/pjotr/.guix-profile/lib/locale
  PATH=/home/pjotr/.guix-profile/bin:/home/pjotr/.guix-profile/sbin
  PKG_CONFIG_PATH=/home/pjotr/.guix-profile/lib/pkgconfig

Inside Guile (REPL)

With most packaging systems the only way to debug them is by sprinkling print statements, using a debugger or hoping for the best (TM). The equivalent in a guix expression would be, for example

(pk 'ECHO (which "echo"))

GNU Guix is written in scheme lisp with the GNU Guile interpreter/compiler. This means code can be run and data can be inspected in the REPL.

From the command line with guile use the REPL like this:

$ ./pre-inst-env guile
  GNU Guile 2.0.11
  Copyright (C) 1995-2014 Free Software Foundation, Inc.

Enter `,help' for help.
scheme@(guile-user)>
;;; read-line support
(use-modules (ice-9 readline))
(activate-readline)
;;; help may come in useful
,help
;;; some LISP
(define a 3)
a
;;; $1 = 3
,pretty-print a
;;; $2 = 3

Load guix (the leading comma interprets the command)

,use (gnu packages ruby)
,use (guix)
,use (guix build-system)

Note that the order of gnu/packages/ruby is simply the directory structure of the git repository. Now start talking with the daemon

(define s (open-connection))
ruby
;;; $1 = #<package ruby-2.2.2 gnu/packages/ruby.scm:39 2ed9f00>
ruby-2.1
;;; $1 = #<package ruby-2.1.6 gnu/packages/ruby.scm:91 36f10c0>
(package-derivation s ruby)
;;; $2 = #<derivation /gnu/store/cvsq4yijavhv7vj7pk3ns0qmvvxdp935-ruby-2.2.2.drv => /gnu/store/66nc9miql9frizn0v02iq1siywsq65w5-ruby-2.2.2 3a9d7d0>
,pretty-print s
;;; $3 = #<build-daemon 256.14 32b7800>

Inspect package (and bag)

Let’s inspect the package using the methods defined in guix/packages.scm

(define p ruby)
(package-name p)
;;; "ruby"
(package-inputs p)
;;; (("readline" #<package readline-6.3 gnu/packages/readline.scm:39 2aa2840>)
;;; ("openssl" #<package openssl-1.0.2b gnu/packages/openssl.scm:30 2f15d80>)
;;; ("libffi" #<package libffi-3.1 gnu/packages/libffi.scm:34 2b8b900>)
;;; etc.
(package->bag p)


$22 = #<<bag> name: "ruby-2.2.2" system: "x86_64-linux" target: #f
build-inputs: (
("source" #<origin "http://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.2.tar.xz" 6az3luekwvyihzemdwa3zvzztftvpdbxbnte3kiockrsrekcirra () 36f28c0>)
("tar" #<package tar-1.28 gnu/packages/bootstrap.scm:145 3953540>)
("gzip" #<package gzip-1.6 gnu/packages/bootstrap.scm:145 39533c0>)
("bzip2" #<package bzip2-1.0.6 gnu/packages/bootstrap.scm:145 3953240>)
("xz" #<package xz-5.0.4 gnu/packages/bootstrap.scm:145 39530c0>)
("file" #<package file-5.22 gnu/packages/bootstrap.scm:145 395cf00>)
("diffutils" #<package diffutils-3.3 gnu/packages/bootstrap.scm:145 395cd80>)
("patch" #<package patch-2.7.5 gnu/packages/bootstrap.scm:145 395cc00>)
("sed" #<package sed-4.2.2 gnu/packages/bootstrap.scm:145 395ca80>)
("findutils" #<package findutils-4.4.2 gnu/packages/bootstrap.scm:145 395c900>)
("gawk" #<package gawk-4.1.1 gnu/packages/bootstrap.scm:145 395c780>)
("grep" #<package grep-2.21 gnu/packages/bootstrap.scm:145 39536c0>)
("coreutils" #<package coreutils-8.23 gnu/packages/bootstrap.scm:145 3953840>)
("make" #<package make-4.1 gnu/packages/bootstrap.scm:145 3953a80>)
("bash" #<package bash-4.3.33 gnu/packages/bootstrap.scm:145 3953e40>)
("ld-wrapper" #<package ld-wrapper-0 gnu/packages/commencement.scm:644 39539c0>)
("binutils" #<package binutils-2.25 gnu/packages/bootstrap.scm:145 394d3c0>)
("gcc" #<package gcc-4.8.4 gnu/packages/commencement.scm:530 394d180>)
("libc" #<package glibc-2.21 gnu/packages/commencement.scm:454 394d600>)
("locales" #<package glibc-utf8-locales-2.21 gnu/packages/commencement.scm:621 3953c00>)
)
host-inputs: (
("readline" #<package readline-6.3 gnu/packages/readline.scm:39 2aa2840>)
("openssl" #<package openssl-1.0.2b gnu/packages/openssl.scm:30 2f15d80>)
("libffi" #<package libffi-3.1 gnu/packages/libffi.scm:34 2b8b900>)
("gdbm" #<package gdbm-1.11 gnu/packages/gdbm.scm:26 2b8b6c0>)
("zlib" #<package zlib-1.2.7 gnu/packages/compression.scm:33 36f1c00>)
)
target-inputs: ()
outputs: ("out")
arguments: (#:system "x86_64-linux" #:test-target "test" #:parallel-tests? #f #:phases
(alist-cons-before (quote configure) (quote replace-bin-sh)
  (lambda _ (substitute* (quote ("Makefile.in" "ext/pty/pty.c" "io.c"
"lib/mkmf.rb" "process.c" "test/rubygems/test_gem_ext_configure_builder.rb"
"test/rdoc/test_rdoc_parser.rb" "test/ruby/test_rubyoptions.rb"
"test/ruby/test_process.rb" "test/ruby/test_system.rb"
"tool/rbinstall.rb"))
(("/bin/sh") (which "sh")))) %standard-phases)
)
build: #<procedure gnu-build (store name input-drvs #:key guile
outputs search-paths configure-flags make-flags out-of-source? tests?
test-target parallel-build? parallel-tests? patch-shebangs?
strip-binaries? strip-flags strip-directories validate-runpath? phases
locale system imported-modules modules substitutable?
allowed-references)>>

where bag is the actual data that gets passed to the build system.

Store monad and G-expressions

Guix uses monad to handle the store state. Read up on these and G-expressions if you intend to hack Guix. To run a procedure within a Store do something like

,use (guix git-download)
(git-reference (url "https://github.com/pjotrp/genenetwork2.git") (commit "860bdcebde5cbb1898c26da80ac67207480c0803"))
$3 = #<<git-reference> url: "https://github.com/pjotrp/genenetwork2.git" commit: "860bdcebde5cbb1898c26da80ac67207480c0803" recursive?: #f>
,enter-store-monad
   (git-fetch $3 'sha256
                  (base32
                   "0yvkv7pnigvcifas3vcr8sk87xrrb8y9nh9v1yx2p43k0xz1q8vz"))

$4 = #<derivation /gnu/store/fmpk2sck6ny5dgyx12s539qcadzky24n-mypackage.drv => /gnu/store/k6q69arfmsm116a8hfkqqah
m0ddzacjc-mypackage 50b9e10>

Here $3 is the git-reference record and $4 is a derivation object, and calling ‘built-derivations’ starts the build process

(built-derivations (list $4))
building path(s) `/gnu/store/fid19bds4rak2zn8pzfhrjdcpmqwhjn4-module-import'
building path(s) `/gnu/store/vf1pmac8yz2g0d4ln5ibwg0xaffdrnpq-module-import-compiled'
building path(s) `/gnu/store/k6q69arfmsm116a8hfkqqahm0ddzacjc-mypackage'
(...)
(run-with-store s
  (git-fetch ref ...))

The principle of a monad is simply to handle `state’ (here the store) outside the called procedures (here the package builder). This prevents passing around state parameters all the time leading to simpler code. For a description of how monads can be implemented in Guile, read Chris Okasaki brilliant writeup `Monadic Programming in Scheme’. If you are a Ruby guy (like me) and want to understand monads, read Tom Stuart’s more gentle `Refactoring Ruby with Monads’.

Using the debugger

It is also possible to step through code and view progress and the contents of variables at every stage. The debugger comes with Guile by default. You can set breakpoints and step through code with step, next and finish.

The user init file

You can set up an init file that gets loaded every time Guile gets started in interactive mode. Mine contains:

;; Init file in ~/.guile

;;; read-line support
(use-modules (ice-9 readline))
 (activate-readline)

;;; GNU Guix
(use-modules (guix hash) (guix) (guix build-system))

Running a Guile script

Instead of using the Guile REPL is is also possible to run the code as a script. Create a script:

(define-module (gnu packages mytest)
  #:use-module (gnu packages ruby)
  #:use-module (guix)
  )

(define s (open-connection))
(define p ruby-2.1)

(write (package->bag p))
(newline)(newline)
(write (string-append (package-name p) "-" (package-version p)))

Run it as

./pre-inst-env guile -s test.scm
(lots of info)

:

"ruby-2.1.6"

IN PROGRESS Using guile in emacs (geiser)

But the best thing, if you use Emacs, is to use Geiser, as noted in ‘HACKING’. In addition to a REPL, it brings stuff like autodoc, jump-to-definition, expression evaluation from the buffer, etc.

Install Geiser and add the guile path to ~/.emacs with

(setq-default geiser-guile-load-path '("~/src/guix"))

Start geiser and you should be able to replicate above commands.

Fixing problems

Compiling the package there may be build problems. cd into the build directory

cd /gnu/tmp/guix-build-ldc-0.17.2.drv-0

and

. environment-variables
make

will recreate the build environment. Now you can see where the build stopped by running commands.

Here I show how you can drill down on tests, disable/fix them fast and create the patch by using ‘git diff’. While this is about the D compiler build system with CMake, the strategy is generic. According to Guix build

The following tests FAILED:
        239 - std.datetime (Failed)
        299 - std.regex.internal.tests (Failed)
        569 - std.datetime-debug (Failed)
        629 - std.regex.internal.tests-debug (Failed)
        670 - dmd-testsuite-debug (Failed)
        673 - llvm-ir-testsuite (Failed)

Using guix build with -K option; I changed into the printed dir after build failure and checked the logs

grep datetime -r *|grep 239
  Testing/Temporary/LastTestsFailed.log:239:std.datetime
  Testing/Temporary/LastTest.log:239/673 Testing: std.datetime
  Testing/Temporary/LastTest.log:239/673 Test: std.datetime

Looking in the log

239/673 Testing: std.datetime
239/673 Test: std.datetime
Command: "/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos2-test-runner" "std.datetime"
Directory: /gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime
"std.datetime" start time: Dec 11 16:16 Europe
Output:
----------------------------------------------------------
FAIL release64 std.datetime
core.time.TimeException@/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos/std/datetime.d(560):
  Fractional seconds must be less than one second.
----------------
<end of output>
Test time =   0.19 sec
----------------------------------------------------------
Test Failed.
"std.datetime" end time: Dec 11 16:16 Europe
"std.datetime" time elapsed: 00:00:00
----------------------------------------------------------

It complains

core.time.TimeException@/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos/std/datetime.d(560):
  Fractional seconds must be less than one second.

On line 560 we find

        enforce(fracSecs < seconds(1), new DateTimeException("Fractional second
s must be less than one second."));

First fix of choice: let’s disable this test by commenting it out. But first fix the build dir permissions and start using git

git init
git add runtime/phobos/std/datetime.d
git commit -a -m 'datetime.d'

comment out the test and ‘git diff’ should show

-        enforce(fracSecs < seconds(1), new DateTimeException("Fractional seconds must be less than one second."));
+        // enforce(fracSecs < seconds(1), new DateTimeException("Fractional seconds must be less than one second."));

Next, rerun the test. If you check the Testlog again you can see it can be invoked as

monza:/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime$ ./phobos2-test-runner-debug std.datetime

First run make again and rerun the test

make
runtime/phobos2-test-runner-debug std.datetime
  ****** FAIL release64 std.datetime
  core.time.TimeException@/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos/std/datetime.d(560): Fractional seconds must be less than one second.

Still complaining! This is we because we also need to build phobos with unittests - unfortunately D creates one huge BLOB of a binary. After some digging in the ctest manual and trial and error I found you can do that by first building the build ‘test’ (as listed by ctest -N):

ctest -R build-phobos2-test-runner-debug

updates runtime/phobos2-test-runner-debug, so now we can

make
runtime/phobos2-test-runner-debug std.datetime

You may use the additional –build-noclean switch, provided it is the same build you are using (e.g., with or without debug). So, next round

make
ctest -R build-phobos2-test-runner-debug --build-noclean
runtime/phobos2-test-runner-debug std.datetime

should be faster. But now we got a different error:

****** FAIL release64 std.datetime
core.exception.AssertError@/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos/std/datetime.d(594): assertThrown failed: No TimeException was thrown.
----------------

which tests the test we disabled. So we disable that too. And we have success:

monza:/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2$   make
[  0%] Built target idgen
[  1%] Built target impcnvgen
[ 16%] Built target LDCShared
[ 16%] Built target ldc2
[ 16%] Built target FileCheck
[ 16%] Built target gen_gccbuiltins
[ 16%] Built target not
[ 18%] Built target ldmd2
[ 18%] Generating std/datetime.o
[ 18%] Linking C static library ../lib/libphobos2-ldc.a
[ 35%] Built target phobos2-ldc
[ 59%] Built target druntime-ldc-debug
[ 59%] Generating std/datetime-debug.o
[ 59%] Linking C static library ../lib/libphobos2-ldc-debug.a
[ 75%] Built target phobos2-ldc-debug
[100%] Built target druntime-ldc

monza:/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2$   ctest -R build-phobos2-test-runner-debug --build-noclean
Test project /gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2
    Start 8: build-phobos2-test-runner-debug
1/1 Test #8: build-phobos2-test-runner-debug ...   Passed   17.73 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) =  17.84 sec

monza:/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2$   runtime/phobos2-test-runner-debug std.datetime
****** FAIL release64 std.datetime
core.exception.AssertError@/gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos/std/datetime.d(594): assertThrown failed: No TimeException was thrown.
----------------

See below section on gdb if you get an exception.

It may be some build stuff gets messed up. You can regenerate all relevant binaries with

make clean
make
ctest -R build-phobos2-test-runner-debug\|build-phobos2-ldc-unittest-debug\|build-druntime-test-runner-debug\|build-druntime-ldc-unittest-debug\|std.datetime-debug

When all tests are ‘fixed’ we can create the patch with

git diff > ldc_disable_failing_tests.patch

When we have done these we can look at fixing some tests - and perhaps communicating with upstream to see if they want to fix/patch some of these in turn, so we don’t need to redo this work next time round. But at least we can run most of the ldc tests now in Guix.

Note also, because we are using git, we can roll back to an earlier edition of the build dir, e.g., to roll back on changes you have not commited

git reset --hard

Using gdb (the debugger) to find issues

In above section I had a segfault at some point and needed to find out where it went wrong. Similar to the earlier command run with gdb in the build directory

~/.guix-profile/bin/gdb --args runtime/phobos2-test-runner-debug std.datetime

And inside GDB:

GNU gdb (GDB) 7.12
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
(gdb) r
Starting program: /gnu/tmp/guix-build-ldc-0.17.2.drv-0/ldc-0.17.2/runtime/phobos2-test-runner-debug std.datetime
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x000000000082cf9e in std.datetime.SysTime.this(const(std.datetime.DateTime), const(core.time.Duration), immutable(std.datetime.TimeZone)) (
    dateTime=<incomplete type>, fracSecs=<incomplete type>,
    tz=0x614a2d313030302d, this=...) at datetime.d:567
567             immutable standardTime = nonNullTZ.tzToUTC(adjustedTime.total!"hnsecs");
(gdb)

You can spot the problem is at line number 567.

Creating patches for GNU Guix

Sometimes you need to modify a source package to compile it on Guix. Here I show my way of creating a patch. This patch with the error log you may want to send upstream to the authors/maintainers, otherwise it will need fixing with every update/release.

Error log

First step is to build the package as is with Guix and capture the output so it can be shared. Building from the source tree

./pre-inst-env guix package -i elixir --no-grafts -K &> error.out

You may remove the boiler plate in that file.

Start from a pristine source tree

To make sure no patches were applied modify the package so the patch does not get applied. In this case comment out

(patches (search-patches "elixir-disable-failing-tests.patch"))))

And rerun the Error log to get a full list of errors.

Force a build to stop

You may also want to force the build to stop right after unpacking by injecting

(arguments
 `(
   #:phases
    (modify-phases %standard-phases
     (add-before 'build 'break (lambda () (#f)))
     ...

Now the build will fail with

ERROR: In procedure #f:
note: keeping build directory `/tmp/guix-build-elixir-1.5.1.drv-5'

Another option is to inject (#f)

(replace 'build
  (lambda _
    (invoke "bash" "-c" "set")
    (#f)
    (invoke "./build.sh")))

will halt after showing the environment

Use git to create a patch

Initialize git using a first terminal

cd /tmp/guix-build-elixir-1.5.1.drv-5
cd elixir*
git init

Add the files you are modifying

git add ./lib/elixir/test/elixir/kernel/dialyzer_test.exs
git add ./lib/elixir/test/elixir/kernel/cli_test.exs
git add ./lib/elixir/test/elixir/system_test.exs
git commit -a -m start

Optionally apply the previous patches by hand - we do this now so the become visible in the new patch.

patch -p1 < /tmp/elixir-disable-failing-tests.patch

Hopefully it mostly takes it. Now fix the problems that occur in the source tree and create a new patch using git

git diff > /tmp/elixir-disable-failing-tests-5.patch

Now plug this patch into the source tree again, enable patch application, and retry above steps.

Note: patching can be done incrementally and patches can be merged into one file (by hand). When you get better at this you can probably save on a few build cycles.

Note: always send the errors and patch(es) upstream. Even if they do nothing about it, at least you have recorded the problems for posterity. Ideally, tag the upstream issue to your GNU Guix patch.

Errors in GNU Guix alone

Things get a bit complicated when a build passes in the Keep directory, but fails in GNU Guix. This usually has to do with files being copied into disallowed directories or network access. Just be smart about reading the code and patching it. Worst case you’ll need to build inside a container/VM to find and fix the problems.

./pre-inst-env guix environment erlang -C --ad-hoc erlang vim make git glibc-locales --no-grafts --pure --share=/tmp/guix-build-elixir-1.5.1.drv-12 --network
export LC_ALL=en_US.UTF-8
cd /tmp/guix-build-elixir-1.5.1.drv-12/elixir-1.5.1/

Installing the package

Once the build works you can use standard guix to install the package

./pre-inst-env guix package -i ruby

This will also build from the source tree and blindly merges that directory into your profile, but lacks information for updates etc:

./pre-inst-env guix package -e '(@ (gnu packages ruby) ruby)'
guix package -i $(guix build ruby)

Where (guix build ruby) is a LISP call which translates into a raw path. With the last example, passing a raw directory name to “guix package -i” does not really know what package it is, so it just blindly merges that directory into your profile. Later upgrades, propagated inputs, and search-path advisories aren’t handled correctly.

Using the search path

One can run:

 GUIX_PROFILE=$HOME/.guix-profile . ~/.guix-profile/etc/profile

or

 eval `guix package --search-paths`

See http://www.gnu.org/software/guix/manual/html_node/Invoking-guix-package.html.

And nowadays one can also use –search-paths=suffix or –search-paths=prefix, for more flexibility.

Making a patch and submit to the Guix project

Using the debbugs interface

debbugs help can be found here.

Bugs can be submitted by E-mail to [email protected] after checking https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix. Use the parameters as described in bug reporting.

Patches are handled through the GNU debbugs server. A current list can be seen at https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix-patches or https://bugs.gnu.org/guix-patches. A mailing list is attached you can subscribe to.

In emacs (using guix package -i emacs-debbugs) the same list can be browsed with

  M-x debbugs-gnu-search <RET> guix-patches

or

  C-u M-x debbugs-gnu <RET> <RET> guix-patches <RET> n y

Possibly you need to add this to your .emacs configuration

(add-to-list 'debbugs-gnu-all-packages "guix-patches")

In debbugs mode hit ‘?’ for key-bindings. Use the ‘C’ key from the emacs interface to tag bugs.

To close a bug mail reply and modify the destination address to issuenumber-done@debbugs.

The patch system

A first time patch is submitted by E-mail to [email protected]. Use the parameters as described in bug reporting.

Each message sent to guix-patches creates a Debbugs entry, as is the case with bug-guix. One can then follow up to [email protected], where NNN is the bug number.

For patch series, please read Glenn’s suggestions. For general questions about Debbugs, see this.

Creating a patch

Check the Guix guidelines first. Note that submitting patches is handled via the debbugs interface now, see above section.

To avoid conflictes, before you start, ascertain the Guix tree is at HEAD

git pull guix master
git log

Make sure your terminal and editors are running in UTF-8. With vim you can force encoding with

:set bomb
:set fileencoding=utf-8
:wq

Use ‘git rebase –interactive’ to merge and squash patches into one. E.g.,

git rebase -i HEAD~4

This can be done with emacs magit. Next use the GNU ChangeLog format which is a header with a filewise change description, for example

gnu: Add Ruby.

* gnu/packages/ruby.scm (Ruby): New file.
* guix/licenses.scm: Add Ruby license information.

To change the last commit message do

git commit --amend

Use git format-patch to send a patch to the mailing list.

git format-patch -1

to generate a patch file, which you can then send to the Guix debbugs ([email protected]). Note: to generate the last 2 patches use -2.

Create the patch

git format-patch -1 --to [email protected]

Before sending the patch(es) out, make sure tabs are turned into spaces. The emacs commands are here. Lines should be broken (use M-q in emacs). And use the Emacs TAB in guix-prettify-mode to find the right LISP indentation.

Which creates a file 0001-gnu-patchname.patch and mail it with something like

git send-email --from "Pjotr Prins <pjotr.guix@mymail>" --to [email protected] 0001-gnu-patchname.patch

You may also need to install ‘guix package -i git:send-email’ to get E-mail support.

Multiple patches can be passed in with something like

git format-patch -10  # create patches for the past 10 commits
git send-email [email protected] *.patch

Probably a good idea to try and send the mail to yourself first. Don’t send the same E-mail twice ;). One example I used

git send-email --from "Pjotr Prins <[email protected]>" --to [email protected] ~/tmp/0001-gnu-ldc-Update-to-1.10.0.patch --suppress-cc=all --subject="[PATCH] Updating ldc to 1.10.0"

To change credentials for the patch use git config. Note that the maintainers will run something like

git am *.patch

to apply the patches.

Once a debbugs entry exists

You can simply reply to the patch with the bug number in the header. So, to resubmit a revised patch with bug number 25704

git send-email --from "Pjotr Prins <pjotr.guix@mymail>" --to [email protected] 0001-gnu-patchname.patch

Environment

You can set up an environment to hack on Guix by entering the clone directory and running

guix environment guix

Then you can just run make to see if everything builds fine. If it does, make a commit with an appropriate commit message, e.g. by using git rebase (see the guix manual) or by creating a diff between branches (useful when there are conflicts etc.)

git diff master > intermediate.patch
git checkout master
git checkout -b submit_branch
patch -p1 < intermediate.patch
git commit -a

Note that the GNU Guix developers want one patch per variable. So submit packages one at a time.

For more information see the official HACKING document in the Guix git repo.

Using git branches

It may be a good idea to keep the master branch in sync with that of Guix. When adding something new checkout a branch first

git checkout -b dev

Now to creat a patch to send to the mailing list do

git commit -a -m 'My last commit'
git checkout master
git checkout -b submit
git rebase --interactive dev

Squash the commits into one

Dealing with the review process

When you write many patches that potentially depend on each other and the review system get choked (the reviewers can’t allways keep up) I resort to a system where I develop patches in a separate branch or even source repository.

If you are using GUIX_PACKAGE_PATH for the separate tree is makes sense to use a different name space (not the gnu directory) and give the packages different names too - so that when you overlap with the GNU Guix package tree there is no name conflict. With the GeneNetwork tree we use the gn/packages path (so modules are in the gn namespace).

Workflow for packaging

The general workflow for adding and maintaining packages is a bit complex. Everything goes via de guix-dev mailing list and includes a review process which can be discouraging and is more geared towards reviewers than towards newbies. This should not discourage you because GNU Guix is great. Note that the reviewers do this work voluntarily and most ‘rules’ have been agreed by the community. In the end your labours will get rewarded. So, how to start?

  1. Work on a recent git checkout of guix
  2. Use ‘guix import’ if you can (e.g. for python and R modules)
  3. Build the package yourself
  4. If tests are there, make sure they pass
  5. Test your work with ‘guix lint’
  6. Create a patch as described above
  7. Send it to debbugs as described above
  8. Submit one patch at a time and submit the next one when it goes in
  9. Be patient, review can take a while - monitor debbugs

With small problems the reviewers will often modify the patch for you. Larger problems you need to fix yourself. See it as a learning process.

Note: sometimes I use an older GNU Guix tree since it is a work in progress and the master may fail for whatever reason. Simply use git cherry-pick to update a single module and it should still work to submit a patch.

Hacking tips

The guile write function

Guile has a ‘write’ function which writes to stdout by default. This can be very useful to generate output on package install.

To inspect variables I may inject something like

(write "****************")
(write out)
(write debug)
(newline)
(#f)

The last command will compile and break at runtime. Together with the -K option it helps trouble shooting.

If that does not work you can also run a command that fails, such as

(write "HELLO WORLD")
(chdir "END HERE")

Show environment settings

Another useful hack is to show the environment

(replace 'build
  (lambda _
    (setenv "LD_LIBRARY_PATH" (getenv "LIBRARY_PATH"))
      (invoke "bash" "-c" "set")
      (...)
      (invoke "./build.sh")))

Using -K (keep build) to create a patch

Using -K you can keep the build dir after failure (induced in above paragraph). You may need to patch the source code to make it to work. What I do is use git. Go into the kept directory and run ‘git init’ and add files you change. That way you can generate a patch file that can be added to the guix source tree.

Guix at the REPL

(thanks to Swedebugia’s Xmas 2018 message and Ludovic’s 2016 REPL talk at FOSDEM)

Run the REPL from guix

guix repl

and set up an environment

(use-modules
  (guix packages)
  (guix import utils)
  (gnu)
  (gnu packages ruby))

E.g. (package<TAB><TAB> shows this list of nice procedures availiable:

package
package->cross-derivation
package->definition
package->derivation
package-build-system
package-cross-build-system-error?
package-cross-derivation
package-derivation
package-description
package-direct-inputs
package-direct-sources
package-error-invalid-input
package-error-package
package-error?
package-field-location
package-file
package-full-name
package-grafts
package-home-page
package-input-error?
package-input-rewriting
package-inputs
package-license
package-location
package-maintainers
package-mapping
package-name
package-native-inputs
package-native-search-paths
package-output
package-outputs
package-patched-vulnerabilities
package-propagated-inputs
package-properties
package-search-paths
package-source
package-source-derivation
package-superseded
package-supported-systems
package-synopsis
package-transitive-inputs
package-transitive-native-inputs
package-transitive-native-search-paths
package-transitive-propagated-inputs
package-transitive-sources
package-transitive-supported-systems
package-transitive-target-inputs
package-upstream-name
package-version
package/inherit
package?

In addition to this there are the following origin-record-procedures:

E.g. (origin<TAB><TAB> shows this list

origin
origin->derivation
origin-actual-file-name
origin-file-name
origin-method
origin-modules
origin-patch-flags
origin-patch-guile
origin-patch-inputs
origin-patches
origin-sha256
origin-snippet
origin-uri
origin?

How about getting the url of a specific package?

scheme@(guix-user)> (origin-uri (package-source ruby))
$8 = "http://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.3.tar.xz"

Fetching it?

scheme@(guix-user)> (url-fetch (origin-uri (package-source ruby)) "temp")
scheme@(guix-user) [43]> (url-fetch (origin-uri (package-source ruby)) "temp")

Starting download of temp
From http://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.3.tar.xz...
 ....3.tar.xz  10.9MiB                133KiB/s 01:24 [##################] 100.0%
$15 = "temp"

With fold-packages you can walk through the whole stack of package records if you would like and count say the number of packages with the prefix “python-“:

scheme@(guile-user)> (define snakes
                        (fold-packages
                             (lambda (package lst)
                               	(if (string-prefix? "python"
                                       (package-name package))
				       (cons package lst)
			     lst))
			'()))

Now we can work on this list. As of writing this we have this many items in the list:

scheme@(guix-user)> (length snakes)
$5 = 1732

or get the URLs

scheme@(guile-user)> (define snakes
                        (fold-packages
                             (lambda (package lst)
                               	(if (string-prefix? "python"
                                       (package-name package))
				       (cons (origin-url package) lst)
			     lst))
			'()))

This is all well for fetching information from Guix packages. How about telling the daemon to build something? At the REPL, you can use the ‘enter-store-monad’ command:

scheme@(guix-user) [50]> ,use (guix monad-repl)
scheme@(guix-user) [50]> ,enter-store-monad

note the change of prompt here!

store-monad@(guix-user) [51]> (package->derivation ruby)
$22 = #<derivation /gnu/store/cb0aag3qha7znkrv9z12j2smvk3kn8h2-ruby-2.5.3.drv => /gnu/store/wd9fcab8pzc8l8bbw958yxa1hmfh0irk-ruby-2.5.3 7f74e2229b90>

(define s (open-connection)) (built-derivations s (package->derivation ruby)) scheme@(guix-user) [76]>(built-derivations s (package->derivation ruby)) $25 = #<procedure 7f74e2c76580 at guix/store.scm:1726:28 (store)>

–8<---------------cut here---------------start------------->8— scheme@(gnu packages bioinformatics-test)> ,enter-store-monad store-monad@(gnu packages bioinformatics-test) [1]> (git-fetch ref ‘sha256 (base32 “0yvkv7pnigvcifas3vcr8sk87xrrb8y9nh9v1yx2p43k0xz1q8vz”) “mypackage”) $4 = #<derivation /gnu/store/fmpk2sck6ny5dgyx12s539qcadzky24n-mypackage.drv => /gnu/store/k6q69arfmsm116a8hfkqqahm0ddzacjc-mypackage 50b9e10> –8<---------------cut here---------------end--------------->8—

Here $4 is a derivation object, and calling ‘built-derivations’ on it actually starts the build process (which fails here because of the bogus URL):

–8<---------------cut here---------------start------------->8— store-monad@(gnu packages bioinformatics-test) [1]> (built-derivations (list $4)) building path(s) `/gnu/store/fid19bds4rak2zn8pzfhrjdcpmqwhjn4-module-import’ building path(s) `/gnu/store/vf1pmac8yz2g0d4ln5ibwg0xaffdrnpq-module-import-compiled’ building path(s) `/gnu/store/k6q69arfmsm116a8hfkqqahm0ddzacjc-mypackage’ fatal: repository ‘some_source.git’ does not exist environment variable `PATH’ unset builder for `/gnu/store/fmpk2sck6ny5dgyx12s539qcadzky24n-mypackage.drv’ failed to produce output path `/gnu/store/k6q69arfmsm116a8hfkqqahm0ddzacjc-mypackage’ guix/store.scm:627:0: In procedure build-things: guix/store.scm:627:0: Throw to key `srfi-34’ with args `(#<condition &nix-protocol-error [message: “build of `/gnu/store/fmpk2sck6ny5dgyx12s539qcadzky24n-mypackage.drv’ failed” status: 1] 4cc61b0>)’.

Entering a new prompt. Type `,bt’ for a backtrace or `,q’ to continue. –8<---------------cut here---------------end--------------->8—

Alternately, you can use ‘run-with-store’, as in:

(define s (open-connection))

(run-with-store s
  (git-fetch ref ...))

where s is the connection to the Guix daemon (in source code also referred to as %store, daemon and store).

scheme@(guix-user) [58]> ,use (guix gexp) scheme@(guix-user) [58]> #~(symlink #$ruby #$output) $24 = #<gexp (symlink #<gexp-input #<package [email protected] gnu/packages/ruby.scm:78 7f74e5311f20>:out> #<gexp-output out>) 7f74e1e80930>

scheme@(guix-user) [76]> s (bag-name bag) $29 = #<store-connection 256.99 22d1360> $30 = “ruby-2.5.3”

scheme@(guix-user) [81]> s (car (bag-build-inputs bag)) $45 = #<store-connection 256.99 22d1360> $46 = (“source” #<origin “http://cache.ruby-lang.org/pub/ruby/2.5/ruby-2.5.3.tar.xz” dte5anm2r2rv7rqrd3edbujomali6o43gbndyjlygv6tmd6pgbxq () 221fba0>)

Guix makes extensive use of records. You can update a record with

In guix/derivations.scm

derivation? derivation-outputs derivation-inputs derivation-sources derivation-system derivation-builder derivation-builder-arguments derivation-builder-environment-vars derivation-file-name derivation-prerequisites derivation-build-plan

Update field in derivation

scheme@(guix-user) [104]> ,use (srfi srfi-9 gnu) scheme@(guix-user) [104]> (set-field drv (derivation-file-name) “TST”) $72 = #<derivation TST => /gnu/store/wd9fcab8pzc8l8bbw958yxa1hmfh0irk-ruby-2.5.3 31f1140>

Hints

Read the HACKING documentation in the Guix source tree.

There are also videos on hacking in gnu.org/s/guix.

Using emacs

Emacs has powerful support for editing LISP (unsurprisingly, perhaps).

Key binding

  • C-M-f and C-M-b move to forward/backward to matching braces

Filing a bug

Send a mail to the bug list, it should look like this:

From: Pjotr Prins <pjotr.public12@email>
To: [email protected]
Bcc:
Subject: guix lint fails with -dc switch missing
Reply-To:

When I run lint on a recent ceckout

  ./pre-inst-env guix lint

or

  ./pre-inst-env guix lint python

I get

  filtered-port: failed to execute ' -dc ': No such file or directory

Backtrace:
In unknown file:
   ?: 19 [apply-smob/1 #<catch-closure 16dfcc0>]
In ice-9/boot-9.scm:
  63: 18 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 17 [eval # #]

Git CA certificates

When you get the dreaded `server certificate verification failed. CAfile: none CRLfile: none’ you may want to fix the path to certificates. Example:

fatal: unable to access 'https://git.savannah.gnu.org/git/guix.git/': server certificate verification failed. CAfile: none CRLfile: none

If you already have CA certificates, you can point git to them using the GIT_SSL_CAINFO variable. In .bashrc:

export GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt

It is also possible to checkout a repository using

env GIT_SSL_NO_VERIFY=true git clone URI

(note there are security implications) and next update inside the repo with

git config http.sslVerify false

to override certificate checking.

Recovering from a ‘guix pull’

Occasionally you do a guix pull and regret it. It is pretty easy to recover. Basically a guix pull fetches the latest guix source tree, puts it in the store and symlinks the directory to ~/.config/guix/latest.

So, to change things, change the symlink and point it to a checked out guix git repository (for example).

ls ~/.config/guix/latest
gnu  gnu.go  gnu.scm  guix  guix.go  guix.scm

Dealing with slow internet connection

When working over mobile networks Guix can be painful. The options to check are –no-substites - so you only download source tarballs which can sometimes be less bulky than binaries. Also the –no-grafts option may prevent large downloads of rebuilt (grafted) packages.

The Ruby package

Ruby Gems

The first Ruby gem support by GNU Guix is ruby-i18n (internationalization). The definition looked like

(define-public ruby-i18n
(package
  (name "ruby-i18n")
  (version "0.6.11")
  (source (origin
            (method url-fetch)
            (uri (string-append "https://github.com/svenfuchs/i18n/archive/v"
                                version ".tar.gz"))
            (sha256
             (base32
              "1fdhnhh1p5g8vibv44d770z8nq208zrms3m2nswdvr54072y1m6k"))))
  (build-system ruby-build-system)
  (arguments
   '(#:tests? #f)) ; requires bundler
  (synopsis "Internationalization library for Ruby")

so it downloads the tar ball. The build system looks like

(define ruby-build-system
  (build-system
    (name 'ruby)
    (description "The standard Ruby build system")
    (lower lower)))

which creates an expression using the standard build-system and the local lower function.

When you install it says

The following environment variable definitions may be needed:
 export GEM_PATH="/home/pjotr/.guix-profile/lib/ruby/gems/2.1.3"

which contains

ls /home/pjotr/.guix-profile/lib/ruby/gems/2.1.3/gems/i18n-0.6.11/
  gemfiles  lib  MIT-LICENSE  README.md  test

Dealing with special packages

Some packages won’t make it into GNU Guix.

If you have need a special section, simply create a directory with packages and add them to the GUIX_PACKAGE_PATH:

export GUIX_PACKAGE_PATH="~/code/guix-special"

this is also useful for packages that are in Guix but that you would like to customize, for instance with a different set of dependencies or different build flags. Make sure it is a full module, a simple module would be:

(define-module (pylmm)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (gnu packages)
  #:use-module (gnu packages python)
  #:use-module (guix download)
  #:use-module (guix packages)
  #:use-module (guix git-download)
  #:use-module (guix utils)
  #:use-module (guix build-system gnu)
  #:use-module (guix build-system python)
  #:use-module (guix build-system trivial)
  #:use-module (srfi srfi-1))

(define-public python-pylmm
  (package
    (name "python-pylmm")
    (version "1.0.0")
    (source
     (origin
       (method url-fetch)
       (uri (string-append
             "https://pypi.python.org/packages/source/p/pylmm/pylmm-"
             version ".tar.gz"))
       (sha256
        (base32 "0bzl9f9g34dlhwf09i3fdv7dqqzf2iq0w7d6c2bafx1nla98qfbh"))))
    (build-system python-build-system)
    (arguments '(#:tests? #f))
    (native-inputs
     `(("python-setuptools" ,python-setuptools)))
    (home-page "https://github.com/genenetwork/pylmm_gn2")
    (synopsis "Python LMM resolver")
    (description
      "Python LMM resolver")
    (license license:gpl-3)))

(define-public python2-pylmm
  (package-with-python2 python-pylmm))

Save it as a file named pylmm.scm (the name of the module!) and add the path

env GUIX_PACKAGE_PATH=~/python/pylmm_gn2/guix guix package -A python-pylmm
  python-pylmm    1.0.0   out     ~/python/pylmm_gn2/guix/pylmm.scm:15:2

Note that, even though GUIX_PACKAGE_PATH can be a feasible way of adding and maintaining packages, it has two largish downsides: (1) it is removed from the main package tree and therefore not easily shared and integrated and (2) to remain compatible you need to juggle two git trees which may go out of synch.

Also take a look at guix ‘channels’ which acts has more advanced options for sharing packages out of tree.

Create a caching server

The Guix daemon contains a build server. It also can distribute built binaries.

See REPRODUCIBLE.org