Support writing a new file system

[rick-masters]

This request originates from @oriansj in connection with transitioning to a new kernel in relation to the https://github.com/oriansj/stage0-posix project and related projects such as https://github.com/fosslinux/live-bootstrap.

The file system design is contained in the https://github.com/oriansj/GFK project.

The specific requirement requested was to support writing a 20G file system to the hard drive, which I accepted. I naively thought this could be supported with immediate writes in a similar way to the disk read support. However, upon further reflection, this will require further discussion to confirm that the specific requirements can be met, and subsequently whether the approach I had in mind is appropriate.

The main issue is that 20G exceeds the range of a 32-bit integer. If seeking is required, this may require a new syscall: llseek. Implementing llseek would be much more work than I anticipated.

Even if lseek is not required, there may be issues with the current CHS disk geometry used when interfacing with the BIOS. Current disk sizes are presented as 16 heads per track. This is an old ATA limitation, resulting in a 504MB disk size limit. This can probably be expanded to 255 heads without a change in strategy, but it may introduce incompatibilities with older hardware and still limits a disk to 8GB in size. There are well known techniques to exceed this limit, but it may require substantial development and may introduce further incompatibilities with older hardware.

(With respect to qemu and SeaBIOS, I believe it uses LBA internally and can support very large disks with a CHS interface, so this should be OK in that scenario. BTW, qemu appears to "guess" the CHS disk geometry [1], which should be fine because builder-hex0 does ask the BIOS for the geometry instead of hard coding it, but this feature has not been tested much.)

It is also worth noting that currently there are only around 60MB or memory reserved for memory files. So that begs the question: "Where is the data going to come from that necessitates 20GB of file system?". If the idea is that we may need it later, then perhaps the file system should be extended later by a kernel/utility that can support that.

Summarizing the options (which may combine or overlap)::

• Support 20G
• Support 8G
• Support 2G
• Support 504MB
• Support immediate writes
  • Support seeking
    • Support llseek
    • Support lseek
    • Support SEEK_SET, SEEK_CUR, and SEEK_END
    • Support subset of SEEK_SET, SEEK_CUR, and SEEK_END
  • Do not support seeking
• Stick with memory files with lseek support
  • Expand memory file size as much as possible

Finally, if substantial work is required then it opens the discussion of whether to proceed to an intermediate kernel which is easier to develop in. I am willing to do substantial more work, but I would suggest doing it hex2, which I believe can be trivially ported to from hex0.

1. https://github.com/qemu/qemu/blob/master/hw/block/hd-geometry.c

[oriansj]

To the question of disk size, a 504MB will be sufficient for most of the steps.

To the idea of more work in hex(0-2) I am against the idea as it is something we are trying to minimize in usage. (It might not fit inside the current guidelines for the definition of source code)

However if you were to do the work in M0 or M1 assembly that would be great and I would be willing to do the porting work for you if you would like.

[rick-masters]

This Issue is mostly outdated by the kernel bootstrap work done for live-bootstrap. The kernel bootstrap implementation builds and transitions to a new kernel without writing to the hard drive by using the ram drive instead and doing a live kexec to Fiwix also using a ram drive.

That said, there was discussion in the live-bootstrap 1.0 roadmap suggesting that Fiwix should boot from the hard drive. This would require builder-hex0 to write an ext2 image to the drive before kexec'ing Fiwix.

In order to write an image and then execute a kexec command, a new internalshell command should be supported to trigger writing the /dev/hda memory file to the disk. This "flush" command must be issued before the kexec command.

The only caveat is that writing an image to the hard drive could overwrite the kexec command if it has not yet been read from the hard drive. Therefore, the implementation should accept the flush command, set an internal flag to indicate that the disk image needs to be flushed, it should read the next command from the disk, and then flush the disk right before executing the (last) command.

Regarding the size limitations, the QEMU Seabios and many other BIOS implementations support an "Extended CHS" scheme to support up to 8.4GB drives by presenting large disks with up to 255 heads. This is combined with 1024 cylinder BIOS limit, a typical 63 sectors per track, and 512 bytes/sector leads to the calculation: 1024 * 255 * 63 * 512 is 8.4GB.

The actual size limit however is constrained by how much memory builder-hex0 can use to create a disk image in memory which is around 1.8GB but will be much lower in practice because source and build files will take away from that.

[rick-masters]

This feature was implemented in commit 3f20b99