Table of Contents

• Description
  • This is a SIMP module
• Setup
  • What simp_gitlab affects
  • Setup Requirements
    • Supported GitLab versions
    • Isolated network requirements
    • Upgrade caveats
      • Upgrade to 0.7.0
      • Upgrade to 0.6.0
      • Upgrade to 0.3.0
  • Beginning with simp_gitlab
• Usage
  • A basic GitLab setup using PKI
  • Passing custom parameters to puppet-gitlab
  • Configuring Nginx
• Reference
  • Further Reference for munging GitLab Omnibus
  • GitLab LDAP integration reference
• Limitations
  • SIMP PKI management does not support Let's Encrypt
  • Gitlab's LDAP TLS is configured to re-use Omnibus' trusted-certs/ instead of ca_file
  • GitLab
    • Puppet runs can fail if GitLab Omnibus's internal services don't start in time
    • Puppet runs can fail if gitlab-rails console load times out
    • Nessus scans may incorrectly report CRIME vulnerability in GitLab
    • Redis log warnings
• Development
  • Acceptance tests
    • Environment variable TEST_GITLAB_CE_VERSION
    • Environment variable TRUSTED_NETS
    • Manually inspecting the SUT GitLab server with a web browser
    • Interactive debugging using pry

Description

This module provides profiles for integrating GitLab Omnibus with SIMP.

This is a SIMP module

This module is a component of the System Integrity Management Platform, a compliance-management framework built on Puppet.

It is designed to be used within a larger SIMP ecosystem, but it can be used independently:

• When included within the SIMP ecosystem, security compliance settings will be managed from the Puppet server.
• If used as an independent module, all SIMP-managed security subsystems are disabled by default and must be explicitly opted into by administrators. Please review the parameters in simp/simp_options for details.

If you run into problems, please let us know by filing an issue at https://simp-project.atlassian.net/.

Setup

What simp_gitlab affects

This module is a profile that integrates Gitlab with SIMP. It makes extensive use of the component module puppet/gitlab. It is important to note that puppet-gitlab configures the GitLab Omnibus's /etc/gitlab/gitlab.rb and runs gitlab-ctl reconfigure.

As a profile module, simp_gitlab has a few functions:

• Integrate SIMP and SIMP's global catalysts with GitLab Omnibus
  • Supported SIMP Options (global catalysts):
    • simp_options::trusted_nets
    • simp_options::firewall
    • simp_options::pki
    • simp_options::ldap::*
  • Intentionally unimplemented:
    • simp_options::selinux: The GitLab Omnibus installer appears to handle SELinux correctly
    • simp_options::tcpwrappers: Nothing in Omnibus is linked to TCP Wrapper
    • simp_options::auditing: Nothing in Omnibus needs special auditd logic
  • Deferred:
    • simp_options::syslog (not clear that we want to support this outside of ee)
  • SIMP integrations:
    • Open access for a local git SSH user
      • set up a SIMP pam::access::rule to permit GitLab's local git user
      • configures sshd such that the GitLab's local git user's SSH authorized keys file path is managed only by the GitLab Omnibus installer.
    • The postfix service that comes with GitLab Omnibus is disabled in favor of the SIMP postfix module.
• Ensure that GitLab Omnibus can be installed without internet access
  • This requires a local mirror of the Gitlab repositories
• Simplify GitLab configuration for common scenarios
  • GitLab
  • GitLab + Omnibus version of NGINX
  • Let's Encrypt
  • Mattermost
  • GitLab CI runner
  • Intentionally unimplemented:
    • Prometheus
      • (Omnibus's integrated Prometheus app monitoring requires Gitlab Omnibus to be installed within a docker container)
    • GitLab CI Runner (docker)
• Permit customization of GitLab Omnibus
• Satisfy as many compliance-relevant criteria as possible

Setup Requirements

Supported GitLab versions

This module was last tested with GitLab Community Edition 15.0.1. It may work for other GitLab versions. You can verify it works for a specific version by executing the acceptance tests with that version.

Isolated network requirements

If using this module from an isolated network, ensure that package and repo management are disabled from the module, and that the gitlab-ce or gitlab-ee package is installed. Be sure that the $simp_gitlab::edition parameter is set to the correct edition.

Upgrade caveats

Upgrade to 0.7.0

simp_gitlab version 0.7.0 now defaults simp_gitlab::allow_fips to true, as the version of GitLab this module was tested with, 14.0.0, supports FIPS mode. This will allow you to install and configure GitLab on FIPS_enabled servers, but not affect existing GitLab servers configured by earlier versions of this module.

Upgrade to 0.6.0

simp_gitlab version 0.6.0 introduced a new mechanism for setting the GitLab root user password upon initial installation of GitLab. As a side effect, by default, the password will be automatically set to the value of simp_gitlab::gitlab_root_password, unless the (empty) marker file /etc/gitlab/.root_password_set exists or the parameter simp_gitlab::set_gitlab_root_password is set to false. If during an upgrade of this module you forget to disable this automation or just want to reset the GitLab root password, simply run /usr/local/sbin/change_gitlab_root_password <new_password> manually. You do not need to know the previous password to set the new password.

Upgrade to 0.3.0

Upgrading from simp_gitlab 0.2.0 to 0.3.0 requires you to copy the authorized key file from /etc/ssh/local_keys/git to /var/opt/gitlab/.ssh/authorized_keys. Alternately you can re-add your deployment keys in the gitlab interface. See the CHANGELOG entries for version 0.3.0 for more details.

Beginning with simp_gitlab

The most basic GitLab usage within a SIMP-managed infrastructure where all simp_options:: are present:

include 'simp_gitlab'

Usage

A basic GitLab setup using PKI

class { 'simp_gitlab':
  trusted_nets => [
                    '10.0.0.0/8',
                    '192.168.21.21',
                    '192.168.21.22',
                    '127.0.0.1/32',
                  ],
  pki          => 'simp',
  firewall     => true,
}

Passing custom parameters to puppet-gitlab

Parameters for puppet/gitlab can be passed in directly using the (Hash) $simp_gitlab::gitlab_options parameter.

Warning: $simp_gitlab::gitlab_options parameters that conflict with the rest of simp_gitlab may lead to undefined behavior / broken systems.

Hint: Many of the data structures used by puppet/gitlab's parameters are documented at https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template

# Name the local git user account `gitlab` (instead of the default `git`)
class { 'simp_gitlab':
  trusted_nets   => $simp_options::trusted_nets
  pki            => 'simp',
  firewall       => true,
  gitlab_options => {
    'user' => {
      'username' => 'gitlab',
      'group'    => 'gitlab',
    }
  },
}

Configuring Nginx

simp_gitlab configures the GitLab's internal Nginx server to look for additional .conf files under /etc/gitlab/nginx/conf.d/.

If you need to configure the main Nginx server, use a file resource from your own profiles to establish a new .conf file.

Reference

See REFERENCE.md for API documentation.

Further Reference for munging GitLab Omnibus

• GitLab Omnibus
  • documentation: https://docs.gitlab.com/omnibus/README.html
    • Common installation problems
    • Maintenance commands
    • Troubleshooting
  • architecture: https://docs.gitlab.com/omnibus/architecture/README.html
    • Global GitLab configuration template
    • Templates for configurations of components
    • gitlab-reconfigure
  • source: https://gitlab.com/gitlab-org/omnibus-gitlab
  • optional services:
    • Mattermost (chat): https://docs.gitlab.com/omnibus/gitlab-mattermost/README.html
    • Prometheus (monitoring): https://docs.gitlab.com/ce/administration/monitoring/prometheus/index.html
    • GitLab Docker images: https://docs.gitlab.com/omnibus/docker/README.html
• puppet/gitlab component module:
  • https://github.com/voxpupuli/puppet-gitlab

• Security & compliance
  • https://www.stigviewer.com/stig/web_server/

GitLab LDAP integration reference

GitLab LDAP Setup has detailed information about configuring GitLab to authenticate users using LDAP. Within this section, there are two sub-sections especially of interest when debugging LDAP connection problems:

• LDAP TLS limitations
• LDAP troubleshooting

Limitations

SIMP PKI management does not support Let's Encrypt

simp_gitlab PKI management does not yet support the use of Let's Encrypt within GitLab. At this time, if you want to use Let's Encrypt, you will need to disable SIMP management of PKI by setting simp_gitlab::pki to false and then manage the certificates manually.

Gitlab's LDAP TLS is configured to re-use Omnibus' trusted-certs/ instead of ca_file

simp_gitlab configures the GitLab Rails server LDAP TLS settings to use the Omnibus trusted_certs (built from /etc/gitlab/trusted-certs) instead of its own ca_file option.

The LDAP ca_file setting is known to cause problems elsewhere in GitLab SSL. There is a (currently undocumented) workaround to these issues: GitLab's LDAP TLS will re-use the GitLab Omnibus trusted-certificates/ directory―but only when the LDAP ca_file option has not been set

The drawback to this solution is that GitLab's LDAP client must share the same TLS settings as GitLab's web server―which is something that we try to keep distinct, as there could be situations in which their configurations legitimately vary.

GitLab

Puppet runs can fail if GitLab Omnibus's internal services don't start in time

Note: These are limitations of upstream components

• The Exec that runs gitlab-ctl reconfigure may take a long time start its internal services, particularly during the first run that installs GitLab. These services are internal to the GitLab Omnibus installation and are not registered with the host OS or the Puppet providers for service. It is possible for the services to take so long to start in the background that dependencies within puppet-gitlab will fail.

  In these cases, it should be sufficient to simply run Puppet again after the services finish starting (status with gitlab-ctl status).

• If the GitLab Omnibus package is already installed but the gitlab-runsvdir service is stopped, the service will not start and catalog compilation will fail.

Puppet runs can fail if gitlab-rails console load times out

Exec[set_gitlab_root_password] will fail if the gitlab-rails console does not come up within the time configured by simp_gitlab::rails_console_load_timeout. If this happens, set the GitLab root password by running /usr/local/sbin/change_gitlab_root_password <new_password> manually, and then run puppet again.

Nessus scans may incorrectly report CRIME vulnerability in GitLab

This is almost certainly a false positive―GitLab configures compression to 0 when HTTPS is enabled.

• See https://docs.gitlab.com/ce/security/crime_vulnerability.html for details.

Redis log warnings

Right now, redis logs these warnings (running in beaker/vagrant VMs):

# WARNING overcommit_memory is set to 0! Background save may fail under low
memory condition. To fix this issue add 'vm.overcommit_memory = 1' to
/etc/sysctl.conf and then reboot or run the command 'sysctl
vm.overcommit_memory=1' for this to take effect.

# WARNING you have Transparent Huge Pages (THP) support enabled in your kernel.
This will create latency and memory usage issues with Redis. To fix this issue
run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as
root, and add it to your /etc/rc.local in order to retain the setting after
a reboot. Redis must be restarted after THP is disabled.

Development

Please read our Contribution Guide.

Acceptance tests

This module includes Beaker acceptance tests using the SIMP Beaker Helpers. By default the tests use Vagrant with VirtualBox as a back-end; Vagrant and VirtualBox must both be installed to run these tests without modification. To execute the tests run the following:

bundle install
bundle exec rake beaker:suites

Please refer to the SIMP Beaker Helpers documentation for more information.

Environment variable TEST_GITLAB_CE_VERSION

TEST_GITLAB_CE_VERSION is an environment variable that can be used to specify the version of gitlab-ce to use in the acceptance tests. When set, it must either a version string for a specific gitlab-ce package version or 'latest' to indicate the latest available version. When unset, the latest version is tested.

TEST_GITLAB_CE_VERSION=15.0.1 bundle exec rake beaker:suites

Environment variable TRUSTED_NETS

TRUSTED_NETS is an environment variable that may contain a comma-delimited list of trusted networks to add to the gitlab SUT's firewall.

TRUSTED_NETS=192.168.11.0/24,10.0.2.2 bundle exec rake beaker:suites

Note: if the TRUSTED_NETS configuration is too broad, it may cause some acceptance tests (for denied clients) to fail.

Manually inspecting the SUT GitLab server with a web browser

Each nodeset in spec/acceptance/nodesets/ contains a commented-out forwarded_ports: section. If you want to use a web browser to manually inspect the SUT GitLab server during any of the tests, uncomment this section and add the NAT network router's IP address to the TRUSTED_NETS variable.

For example:

TRUSTED_NETS=10.0.2.2 BEAKER_destroy=no bundle exec rake beaker:suites

Then, in a browser on the host machine, navigate to https://127.0.0.1:<port>, where <port> is the forwarded port for the test being executed, i.e., 8080, 8443, or 8777.

Note: If you need to login as the administrator to the GitLab instance, use the login of root and the password found on the GitLab server host at /opt/puppetlabs/puppet/cache/simp/environments/production/simp_autofiles/gen_passwd/simp_gitlab_server.<your domain>.

Interactive debugging using pry

Setting the environment variable PRY=yes will cause the acceptance tests to drop into a pry console under certain circumstances (usually just before failures in examples with complex or hard-to-debug state). This will effectively pause the test, to allow you to debug.

TRUSTED_NETS=10.0.2.2 BEAKER_destroy=no PRY=yes bundle exec rake beaker:suites