@@ -12,94 +12,90 @@ An attempt to provide a reproducible, automatic, complete end-to-end
1212bootstrap from a minimal number of binary seeds to a supported fully
1313functioning operating system.
1414
15- Get me started!
16- ---------------
15+ How do I use this?
16+ ------------------
17+
18+ Quick start:
19+
20+ See ``./rootfs.py --help `` and follow the instructions given there.
21+ This uses a variety of userland tools to prepare the bootstrap.
22+
23+ (*Currently, there is no way to perform the bootstrap without external
24+ preparations! This is a currently unsolved problem. *)
25+
26+ Without using Python:
1727
18281. ``git clone https://github.com/fosslinux/live-bootstrap ``
19292. ``git submodule update --init --recursive ``
20- 3. Provide a kernel (vmlinuz file) as the name ``kernel `` in the root of the
21- repository. **This must be a 32-bit kernel. **
22- 4. ``./rootfs.py --qemu `` - ensure your account has kvm privileges and qemu
23- installed.
24-
25- a. Alternatively, run ``./rootfs.py --chroot `` to run it in a chroot.
26- b. Alternatively, run ``./rootfs.py --bwrap `` to run it in a bubblewrap
27- sandbox. When user namespaces are supported, this mode is rootless.
28- c. Alternatively, run ``./rootfs.py `` but don’t run the actual
29- virtualization and instead copy sysa/tmp/initramfs to a USB or
30- some other device and boot from bare metal. NOTE: we now require
31- a hard drive. This is currently hardcoded as sda. You also need
32- to put ``sysc/tmp/disk.img `` onto your sda on the bootstrapping
33- machine.
34- d. Alternatively, do not use python at all, see "Python-less build"
35- below.
36-
37- 5. Wait.
38- 6. If you can, observe the many binaries in ``/usr/bin ``! When the
39- bootstrap is completed ``bash `` is launched providing a shell to
40- explore the system.
41-
30+ 3. Consider whether you are going to run this in a chroot, in QEMU, or on bare
31+ metal. (All of this *can * be automated, but not in a trustable way. See
32+ further below.)
33+ a. **chroot: ** Create a directory where the chroot will reside, run
34+ ``./download-distfiles.sh ``, and copy:
35+ * The entire contents of ``seed/stage0-posix `` into that directory.
36+ * All other files in ``seed `` into that directory.
37+ * ``steps/ `` and ``distfiles/ `` into that directory.
38+ * At least all files listed in ``steps/pre-network-sources `` must be
39+ copied in. All other files will be obtained from the network.
40+ * Run ``/bootstrap-seeds/POSIX/x86/kaem-optional-seed `` in the chroot.
41+ (Eg, ``chroot rootfs /bootstrap-seeds/POSIX/x86/kaem-optional-seed ``).
42+ b. **QEMU: ** Create two blank disk images.
43+ * On the first image, write
44+ ``seed/stage0-posix/bootstrap-seeds/NATIVE/x86/builder-hex0-x86-stage1.img ``
45+ to it, followed by ``kernel-bootstrap/builder-hex0-x86-stage2.hex0 ``,
46+ followed by zeros padding the disk to the next sector.
47+ * distfiles can be obtained using ``./download-distfiles.sh ``.
48+ * See the list in part a. For every file within that list, write a line to
49+ the disk ``src <size-of-file> <path-to-file> ``, followed by the contents
50+ of the file.
51+ * *Only * copy distfiles listed in ``steps/pre-network-sources `` into
52+ this disk.
53+ * Optionally (if you don't do this, distfiles will be network downloaded):
54+ * On the second image, create an MSDOS partition table and one ext3
55+ partition.
56+ * Copy ``distfiles/ `` into this disk.
57+ * Run QEMU, with 4+G RAM, optionally SMP (multicore), both drives (in the
58+ order introduced above), a NIC with model E1000 (``-nic
59+ user,model=e1000 ``), and ``-machine kernel-irqchip=split ``.
60+ c. **Bare metal: ** Follow the same steps as QEMU, but the disks need to be
61+ two different *physical * disks, and boot from the first disk.
4262
4363Background
4464----------
4565
46- This project is a part of the bootstrappable project, a project that
47- aims to be able to build complete computing platforms through the use of
48- source code. When you build a compiler like GCC, you need another C
49- compiler to compile the compiler - turtles all the way down. Even the
50- first GCC compiler was written in C. There has to be a way to break the
51- chain…
52-
53- There has been significant work on this over the last 5 years, from
54- Jeremiah Orians’ stage0, hex2 and M2-Planet to janneke’s Mes. We have a
55- currently, fully-functioning chain of bootstrapping from the 357-byte
56- hex0 seed to a complete GCC compiler and hence a full Linux operating
57- system. From there, it is trivial to move to other UNIXes. However,
58- there is only currently one vector through which this can be
59- automatically done, GNU Guix.
60-
61- While the primary author of this project does not believe Guix is a bad
62- project, the great reliance on Guile, the complexity of many of the
63- scripts and the rather steep learning curve to install and run Guix make
64- it a very non plug-and-play solution. Furthermore, there is currently
65- (Jan 2021) no possible way to run the bootstrap from outside of a
66- pre-existing Linux environment. Additionally, Guix uses many scripts and
67- distributed files that cannot be considered source code.
68-
69- (NOTE: Guix is working on a Full Source Bootstrap, but I’m not
70- completely sure what that entails).
71-
72- Furthermore, having an alternative bootstrap automation tool allows
73- people to have greater trust in the bootstrap procedure.
74-
75- Comparison between GNU Guix and live-bootstrap
76- ----------------------------------------------
77-
78- +----------------------+----------------------+----------------------+
79- | Item | Guix | live-bootstrap |
80- +======================+======================+======================+
81- | Total size of seeds | ~30MB (Reduced | ~1KB |
82- | [1] | Source Bootstrap) | |
83- | | [2] | |
84- +----------------------+----------------------+----------------------+
85- | Use of kernel | Linux-Libre Kernel | Any Linux Kernel |
86- | | | (2.6+) [3] |
87- +----------------------+----------------------+----------------------+
88- | Implementation | Yes | No (in development) |
89- | complete | | |
90- +----------------------+----------------------+----------------------+
91- | Automation | Almost fully | Optional user |
92- | | automatic | customization |
93- +----------------------+----------------------+----------------------+
94-
95- [1]: Both projects only use software licensed under a FSF-approved
96- free software license. Kernel is excluded from seed.
97- [2]: Reiterating that Guix is working on a full source bootstrap,
98- although that still uses guile (~12 MB). [3]: Work is ongoing to use
99- other, smaller POSIX kernels.
100-
101- Why would I want bootstrapping?
102- -------------------------------
66+ Problem statement
67+ =================
68+
69+ live-bootstrap's overarching problem statement is;
70+
71+ > How can a usable Linux system be created with only human-auditable, and
72+ wherever possible, human-written, source code?
73+
74+ Clarifications:
75+
76+ * "usable" means a modern toolchain, with appropriate utilities, that can be
77+ used to expand the amount of software on the system, interactively, or
78+ non-interactively.
79+ * "human-auditable" is discretionary, but is usually fairly strict. See
80+ "Specific things to be bootstrapped" below.
81+
82+ Why is this difficult?
83+ ======================
84+
85+ The core of a modern Linux system is primarily written in C and C++. C and C++
86+ are **self-hosting **, ie, nearly every single C compiler is written in C.
87+
88+ Every single version of GCC was written in C. To avoid using an existing
89+ toolchain, we need some way to be able to compile a GCC version without C. We
90+ can use a less well-featured compiler, TCC, to do this. And so forth, until we
91+ get to a fairly primitive C compiler written in assembly, ``cc_x86 ``.
92+
93+ Going up through this process requires a bunch of other utilities as well; the
94+ autotools suite, guile and autogen, etc. These also have to be matched
95+ appropriately to the toolchain available.
96+
97+ Why should I care?
98+ ------------------
10399
104100That is outside of the scope of this README. Here’s a few things you can
105101look at:
@@ -117,7 +113,7 @@ bootstrapping. However, there are a number of non-auditable files used
117113in many of their packages. Here is a list of file types that we deem
118114unsuitable for bootstrapping.
119115
120- 1. Binaries (apart from seed hex0, kaem, kernel ).
116+ 1. Binaries (apart from seed hex0, kaem, builder-hex0 ).
1211172. Any pre-generated configure scripts, or Makefile.in’s from autotools.
1221183. Pre-generated bison/flex parsers (identifiable through a ``.y ``
123119 file).
@@ -131,56 +127,18 @@ How does this work?
131127
132128**For a more in-depth discussion, see parts.rst. **
133129
134- sysa
135- ~~~~
136-
137- sysa is the first ‘system’ used in live-bootstrap. We move to a new
138- system after a reboot, which often occurs after the movement to a new
139- kernel. It is run by the seed Linux kernel provided by the user. It
140- compiles everything we need to be able to compile our own Linux kernel.
141- It runs fully in an initramfs and does not rely on disk support in the
142- seed Linux kernel.
143-
144- sysb
145- ~~~~
146-
147- sysb is the second 'system' of live-bootstrap. This uses the Linux 4.9.10
148- kernel compiled within sysa. As we do not rely on disk support in sysa, we
149- need this intermediate system to be able to add the missing binaries to sysc
150- before moving into it. This is executed through kexec from sysa. At this point,
151- a SATA disk IS required.
152-
153- sysc
154- ~~~~
155-
156- sysc is the (current) last 'system' of live-bootstrap. This is a continuation
157- from sysb, executed through util-linux's ``switch_root `` command which moves
158- the entire rootfs without a reboot. Every package from here on out is compiled
159- under this system, taking binaries from sysa. Chroot and bubblewrap modes skip
160- sysb, as it is obviously irrelevant to them.
161-
162- Python-less build
163- -----------------
164-
165- Python is no longer a requirement to set up the build system. The
166- repository is almost completely in a form where it can be used as the
167- source of a build.
168-
169- 1. Download required tarballs into ``sysa/distfiles `` and ``sysc/distfiles ``.
170- You can use the ``download-distfiles.sh `` script.
171- 2. Copy sysa/stage0-posix/src/* to the root of the repository.
172- 3. Copy sysa/stage0-posix/src/bootstrap-seeds/POSIX/x86/kaem-optional-seed
173- to init in the root of the repository.
174- 4. Copy sysa/after.kaem to after.kaem
175- 5. Create a CPIO archive (eg, ``cpio --format newc --create --directory . > ../initramfs ``).
176- 6. Boot your initramfs and kernel.
177-
178- chroot builds
179- ~~~~~~~~~~~~~
130+ Firstly, ``builder-hex0 `` is launched. ``builder-hex0 `` is a minimal kernel that is
131+ written in ``hex0 ``, existing in 3 self-bootstrapping stages.
180132
181- For chroot based bootstraps you can skip creation of initramfs and instead start bootstrap with
133+ This is capable of executing the entirety of ``stage0-posix ``, (see
134+ ``seed/stage0-posix ``), which produces a variety of useful utilities and a basic
135+ C language, ``M2-Planet ``.
182136
183- ``sudo chroot . bootstrap-seeds/POSIX/x86/kaem-optional-seed ``
137+ ``stage0-posix `` runs a file called ``after.kaem ``. This is a shell script that
138+ builds and runs a small program called ``script-generator ``. This program reads
139+ ``steps/manifest `` and converts it into a series of shell scripts that can be
140+ executed in sequence to complete the bootstrap.
184141
185- It is also recommended to copy everything to a new directory as bootstrapping messes up with files
186- in git repository and cannot be re-run again.
142+ From this point forward, ``steps/manifest `` is effectively self documenting.
143+ Each package built exists in ``steps/<pkg> ``, and the build scripts can be seen
144+ there.
0 commit comments