git-interface.txt

The aurweb Git and SSH interface
================================

Git storage
-----------

Since release 4.0.0, aurweb uses Git repositories to store packages. Git
namespaces (see gitnamespaces(7)) are used to share the object database, such
that delta compression can be applied across package base boundaries.

Internally, all packages are stored in a single Git repository. Special refs,
so-called namespaced branches, are used to refer to the commits corresponding
to the actual package bases. For convenience, we also create a branch for each
package repository that carries the name of the corresponding package base,
such that one can easily access the history of a given package base by running
`git log <pkgbase>`. To the end-user, the individual namespaced branches are
presented as separate Git repositories.

Authentication: git-auth
------------------------

Pushing to package repositories is possible via SSH. In order to access the SSH
interface, users first need to add an SSH public key to their account using the
web interface. Authentication is performed by the git-auth
AuthorizedKeysCommand script (see sshd_config(5) for details) which looks up
the public key in the AUR user table. Using this concept of "virtual users",
there is no need to create separate UNIX accounts for each registered AUR user.

If the public key is found, the corresponding authorized_keys line is printed
to stdout. If the public key does not exist, the login is denied. The
authorized_keys line also contains a forced command such that authenticated
users cannot access anything on the server except for the aurweb SSH interface.
The forced command can be configured in the aurweb configuration file and it
usually points to the git-serve program.

If SSH has been configured to pass on the AUR_OVERWRITE environment variable
(via SendEnv, see ssh_config(5) for details) and the user's account is a
registered Package Maintainer or Developer, this will be passed on to the git-update
program in order to enable a non-fast-forward push.

The INSTALL file in the top-level directory contains detailed instructions on
how to configure sshd(8) to use git-auth for authentication.

The Shell: git-serve
--------------------

The git-serve command, the "aurweb shell", provides different subcommands:

* The adopt/disown commands can be used to change ownership of a package base.
* The flag/unflag commands can be used to flag/unflag a package.
* The help command shows a list of available commands.
* The list-repos command lists all repositories of the authenticated user.
* The restore command can be used to restore a deleted package base.
* The set-comaintainers command modifies the co-maintainers of a package base.
* The set-keywords command modifies the keywords assigned to a package base.
* The vote/unvote command can be used to vote/unvote for a package base.
* The git-{receive,upload}-pack commands are redirected to git-shell(1).

The command is extracted from the SSH_ORIGINAL_COMMAND environment variable
which is usually set by the SSH daemon. If no command is specified, git-serve
displays a message stating that aurweb does not provide an interactive shell.

When invoking git-shell(1), the git-serve command also redirects all paths to
the shared Git repository and sets up the GIT_NAMESPACE environment variable
such that Git updates the correct namespaced branch.

The Update Hook: git-update
---------------------------

The Git update hook, called git-update, performs several subtasks:

* Prevent from creating branches or tags other than master.
* Deny non-fast-forwards, except for Package Maintainers and Developers.
* Deny blacklisted packages, except for Package Maintainers and Developers.
* Verify each new commit (validate meta data, impose file size limits, ...)
* Update package base information and package information in the database.
* Update the named branch and the namespaced HEAD ref of the package.

It needs to be added to the shared Git repository, see INSTALL in the top-level
directory for further information.

Accessing Git repositories via HTTP
-----------------------------------

Git repositories can also be accessed via HTTP by configuring the web server to
forward specific requests to git-http-backend(1). Note that, since Git
namespaces are used internally, the web server needs to rewrite URIs and setup
the GIT_NAMESPACE environment variable accordingly before forwarding a request.

An example configuration for nginx and fcgiwrap can be found in the INSTALL
instructions in the top-level directory.

Further Configuration and Administration
----------------------------------------

When using Git namespaces, Git advertises refs outside the current namespace as
so-called "have" lines. This is normally used to reduce traffic but it has the
opposite effect in the case of aurweb: Most of the refs transferred to the
client during `git push` operations belong to branches of other package bases
and are essentially useless.

In order to omit these advertisements, one can add the strings "^refs/",
"!refs/" and "!HEAD" to the transfer.hideRefs configuration setting. Note that
the order of these patterns is important ("^refs/" must come first) and that
Git 2.7 or newer is required for them to work.

Since garbage collection always affects all objects (from all namespaces), it
is also recommended to disable automatic garbage collection by setting
receive.autogc to false. Remember to periodically run `git gc` manually or
setup a maintenance script which initiates the garbage collection if you follow
this advice. For gc.pruneExpire, we recommend "3.months.ago", such that commits
that became unreachable by Package Maintainer intervention are kept for a while.

Script Wrappers (poetry)
------------------------

When aurweb is installed within a poetry virtualenv, commands need
to be prefixed with `poetry run`. A few wrapper scripts are provided
which automate this, as long as the executing user's `$HOME` is
aurweb's project root.

- `examples/aurweb-git-auth.sh`
- `examples/aurweb-git-serve.sh`
- `examples/aurweb-git-update.sh`

These scripts should be installed somewhere on the aurweb host. The
following options need to be updated to the installed wrappers:

- `[auth] git-serve-cmd`
- `[serve] git-update-cmd`