Skip to content

Add File System #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions docs/RustPOSIX/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,72 @@ The following are a breif description of the public methods
- `insert_next_pipe` - given a pipe, insert it into the pipe table
- `add_to_fd_table`, `rm_from_fd_table`, and `changedir` are **UNUSED!**

# FILE SYSTEM
# Inode Structure

An important part of file representation, the Inode structure in SafePOSIX encompasses various formats of files supported by the system. Each file and directory in the filesystem has a different inode number.

## Types of Inodes

- **File**: Represents a standard file that contains data.
- **CharDev**: Denotes a character device such as `/dev/null` or `/dev/zero`, including random number generators.
- **Socket**: Represents a Unix domain socket, used for inter-process communication.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for all sockets not just unix

- **Dir**: Represents a directory, organizing files and other directories.

## Metadata within Inodes

Each variant of Inode stores specific metadata:

- **Common Metadata (all Inode types)**:
- `size`: Size of the file in bytes.
- `uid`, `gid`: User and group identity numbers.
Copy link
Member

@JustinCappos JustinCappos May 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be more useful to explain how we handle things like uid / gid. Do they have a meaning in our current implementation? Do they get set with setuid, etc.? Are the mode bits checked?

So, instead of just documenting the fields, it may be better to explain how what we have done differs from POSIX so a developer knows what to expect. (This is a general comment, not meant to only apply to uid, gid, etc.)

- `mode`: Permissions of the file (read, write, execute).
- `linkcount`: Number of hard links that point to the inode.
- `refcount`: Counts open file descriptors of the file, allowing the system to track the file's lifetime internally.
- `atime`, `ctime`, `mtime`: Access, creation, modification timestamps.

- **Directory-specific metadata (DirectoryInode)**:
- `filename_to_inode_dict`: A fast HashMap of filenames.

- **Character Device-Specific Metadata (DeviceInode)**:
- `dev`: A DevNo struct holding the major and minor device numbers, uniquely identifying the device.

## Ref Counting and File Lifetimes

Refcount is used to manage file lifetimes. When a file is opened, its refcount increases; when closed, it decreases. The file is deleted only if both the link and ref count are 0, ensuring no current process can access the file.

# Metadata

## FilesystemMetadata Data Structure

The FilesystemMetadata struct holds central filesystem-wide data:

- `nextinode`: The next available inode number, ensuring uniqueness for new files and directories.
- `dev_id`: The device ID where the filesystem resides.
- `Inodetable`: HashMap mapping inode numbering to InodeEnum for quick lookups of file and directory metadata.

# Logging for Correctness and Recovery

## Write Ahead Logging

- **Log File**: Modifications to filesystem metadata are initially recorded in the "lind.md.log" file before being committed to actual metadata structures.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a little bit more here would also be useful. Espeically explaining when we log

- **Crash Recovery**: Upon reboot, the log file is scanned. If any inconsistencies are found, it will replay the logged changes, restoring the filesystem to a consistent state.

## Logging Functionality

- `log_metadata`: Adds a log entry (inode number and potentially some inode data) to a log file, enabling crash recovery or the ability to undo certain changes.
- `persist_metadata`: Serializes the entire FilesystemMetadata object and writes its data onto disk, allowing the system to remember the state of the filesystem.

# Path Traversal
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a bit more explanation of how these functions work under the hood would be extremely helpful.


Translation from paths to inode numbers, and vice versa, is aided by the following functions:

- `metawalkandparent`: Given a path, Returns the inode number of the last element in it, and the inode number of its parent directory.
- `metawalk`: Similar to metawalkandparent, but returns only inode number of the last element in it.
- `pathnamefrominodenum`: Given an inode number, performs the opposite of the above operation.
- `filenamefrominode`: Given a directory inode and a target inode number that belongs to any file within the directory, it returns the filename of the given inode from the directory’s files.


## FS Metadata

Our metadata table (in the code, this is `FilesystemMetadata` struct) is similar to how the metadatatable from SafePOSIX-Repy operated.
Expand Down