As per this advisory blog post, we've decided to change the default publishing rules for the Matrix room directory.
Our general goal is to favor privacy and security when running personal (family & friends) and corporate homeservers. Both of these likely benefit from having a more secure default of not showing the room directory without authentication and not publishing the room directory over federation.
As with anything else, these new defaults can be overriden by changing the matrix_synapse_allow_public_rooms_without_auth and matrix_synapse_allow_public_rooms_over_federation variables, respectively.
Postgres upgrading and importing have been improved to add support for multiple databases and roles.
Previously, the playbook would only take care of the homeserver database and synapse user.
We now back up and restore all databases and users on the Postgres server.
For now, the playbook only uses that one database (homeserver) and that one single user (synapse), so it's all the same.
However, in the future, additional components besides Synapse may also make use the Postgres database server.
One such example is the matrix-appservice-slack bridge, which strongly encourages use of Postgres in its v1.0 release. We are yet to upgrade to it.
Additionally, Postgres upgrading now uses gzipped dump files by default, to minimize disk space usage.
The playbook now installs Postgres 12 by default.
If you have have an existing setup, it's likely running on an older Postgres version (9.x, 10.x or 11.x). You can easily upgrade by following the upgrading PostgreSQL guide.
Synapse 1.4.0 is out with lots of changes related to privacy.
Its new defaults (which we adopt as well) mean that certain old data will automatically get purged after a certain number of days. 1.4.0 automatically garbage collects redacted messages (defaults to 7 days) and removes unused IP and user agent information stored in the user_ips table (defaults to 30 days). If you'd like to preserve this data, we encourage you to look at the redaction_retention_period and user_ips_max_age options (controllable by the matrix_synapse_redaction_retention_period and matrix_synapse_user_ips_max_age playbook variables, respectively) before doing the upgrade. If you'd like to keep data indefinitely, set these variables to null (e.g. matrix_synapse_redaction_retention_period: ~).
From now on the trusted_key_servers setting for Synapse is configurable. It still defaults to matrix.org just like it always has, but in a more explicit way now. If you'd like to use another trusted key server, adjust the matrix_synapse_trusted_key_servers playbook variable.
Synapse 1.4.0 also changes lots of things related to identity server integration. Because Synapse will now by default be responsible for validating email addresses for user accounts, running without an identity server looks more feasible. We still have concerns over disabling the identity server by default, so for now it remains enabled.
There have been lots of invite-spam attacks lately and Travis has created a Synapse module (synapse-simple-antispam) to let people protect themselves.
From now on, you can easily install and configure this spam checker module through the playbook.
Learn more in Setting up Synapse Simple Antispam.
Similarly to Extensible Synapse configuration (below), Riot-web configuration is also extensible now.
From now on, you can extend/override Riot-web's configuration by making use of the matrix_riot_web_configuration_extension_json variable.
This should be enough for most customization needs.
If you need even more power, you can now also take full control and override matrix_riot_web_configuration_default (or matrix_riot_web_configuration) directly.
Learn more in Configuring Riot-web.
Previously, we had to create custom Ansible variables for each and every Synapse setting. This lead to too much effort (and configuration ugliness) to all of Synapse's settings, so naturally, not all features of Synapse could be controlled through the playbook.
From now on, you can extend/override the Synapse server's configuration by making use of the matrix_synapse_configuration_extension_yaml variable.
This should be enough for most customization needs.
If you need even more power, you can now also take full control and override matrix_synapse_configuration (or matrix_synapse_configuration_yaml) directly.
Learn more here in Configuring Synapse.
Thanks to the great work of kingoftheconnors and Stuart Mumford (Cadair), the playbook now supports bridging to Slack via the appservice-slack bridge.
Additional details are available in Setting up Appservice Slack bridging.
Thanks to the great work of Eduardo Beltrame (Munfred) and Robbie D (microchipster), the playbook now supports bridging to Google Hangouts via the mautrix-hangouts bridge.
Additional details are available in Setting up Mautrix Hangouts bridging.
Support for Email2Matrix has been added.
It's an optional feature that you can enable via the playbook.
To learn more, see the playbook's documentation on Email2Matrix.
After some discussion in our support room, we've decided to change the default logging level for Synapse from INFO to WARNING.
This greatly reduces the number of log messages that are being logged, leading to:
- much less disk space dedicated to Synapse and thus, logs kept for longer
- easier to find some important
WARNING,ERRORandCRITICALmessages, as they're not longer buried in thousands of non-importantINFOmessages
If you'd like to track down an issue, you can always increase the logging level as described here.
The playbook can now help you with Synapse's maintenance.
There's a new documentation page about Synapse maintenance and another section on Postgres vacuuming.
Among other things, if your Postgres database has grown significantly over time, you may wish to ask the playbook to purge unused data with synapse-janitor for you.
Some internal playbook control variables have been renamed.
This change only affects people who run this playbook's roles from another playbook. If you're using this playbook as-is, you're not affected and don't need to do anything.
The following variables have been renamed:
- from
run_import_postgrestorun_postgres_import - from
run_import_sqlite_dbtorun_postgres_import_sqlite_db - from
run_upgrade_postgrestorun_postgres_upgrade - from
run_import_media_storetorun_synapse_import_media_store - from
run_register_usertorun_synapse_register_user - from
run_update_user_passwordtorun_synapse_update_user_password
Following what the official Synapse Docker image is doing (#5565) and what we've been doing for mostly everything installed by this playbook, Synapse no longer logs to text files (/matrix/synapse/run/homeserver.log*).
From now on, Synapse would only log to console, which goes to systemd's journald.
To see Synapse's logs, execute: journalctl -fu matrix-synapse
Because of this, the following variables have become obsolete and were removed:
matrix_synapse_max_log_file_size_mbmatrix_synapse_max_log_files_count
To prevent confusion, it'd be better if you delete all old files manually after you've upgraded (rm -f /matrix/synapse/run/homeserver.log*).
Because Synapse is incredibly chatty when it comes to logging (here's one such issue describing the problem), if you're running an ancient distribution (like CentOS 7.0), be advised that systemd's journald default logging restrictions may not be high enough to capture all log messages generated by Synapse. This is especially true if you've got a busy (Synapse) server. We advise that you manually add RateLimitInterval=0 and RateLimitBurst=0 under [Storage] in the /etc/systemd/journald.conf file, followed by restarting the logging service (systemctl restart systemd-journald).
Until now, the config.yaml file for the Discord bridge was managed by the playbook, but the registration.yaml file was not.
From now on, the playbook will keep both configuration files sync for you.
This means that if you were making manual changes to the /matrix/appservice-discord/discord-registration.yaml configuration file, those would be lost the next time you run the playbook.
The bridge now stores configuration in a subdirectory (/matrix/appservice-discord/config).
Likewise, data is now also stored in a subdirectory (/matrix/appservice-discord/data). When you run the playbook with an existing database file (/matrix/appservice-discord/discord.db), the playbook will stop the bridge and relocate the database file to the ./data directory. There's no data-loss involved. You'll need to restart the bridge manually though (--tags=start).
The main directory (/matrix/appservice-discord) may contain some leftover files (user-store.db, room-store.db, config.yaml, discord-registration.yaml, invite_link). These are no longer necessary and can be deleted manually.
We're now following the default sample configuration for the Discord bridge.
If you need to override some values, define them in matrix_appservice_discord_configuration_extension_yaml.
Until now, configuration files for the WhatsApp bridge were created by the playbook initially, but never modified later on.
From now on, the playbook will keep the configuration in sync for you.
This means that if you were making manual changes to the /matrix/mautrix-whatsapp/config.yaml or /matrix/mautrix-whatsapp/registration.yaml configuration files, those would be lost the next time you run the playbook.
The bridge now stores configuration in a subdirectory (/matrix/mautrix-whatsapp/config), so your old configuration remains in the base directory (/matrix/mautrix-whatsapp).
You need to migrate any manual changes over to the new matrix_mautrix_whatsapp_configuration_extension_yaml variable, so that the playbook would apply them for you.
Likewise, data is now also stored in a subdirectory (/matrix/mautrix-whatsapp/data). When you run the playbook with an existing database file (/matrix/mautrix-whatsapp/mautrix-whatsapp.db), the playbook will stop the bridge and relocate the database file to the ./data directory. There's no data-loss involved. You'll need to restart the bridge manually though (--tags=start).
We're now following the default configuration for the WhatsApp bridge.
Until now, configuration files for the IRC bridge were created by the playbook initially, but never modified later on.
From now on, the playbook will keep the configuration in sync for you.
This means that if you were making manual changes to the /matrix/appservice-irc/config.yaml or /matrix/appservice-irc/registration.yaml configuration files, those would be lost the next time you run the playbook.
The bridge now stores configuration in a subdirectory (/matrix/appservice-irc/config), so your old configuration remains in the base directory (/matrix/appservice-irc).
Previously, we asked people to configure bridged IRC servers by extending the bridge configuration (matrix_appservice_irc_configuration_extension_yaml). While this is still possible and will continue working forever, we now recommend defining IRC servers in the easier to use matrix_appservice_irc_ircService_servers variable. See our IRC bridge documentation page for an example.
If you decide to continue using matrix_appservice_irc_configuration_extension_yaml, you might be interested to know that ircService.databaseUri and a few other keys now have default values in the base configuration (matrix_appservice_irc_configuration_yaml). You may wish to stop redefining those keys, unless you really intend to override them. You most likely only need to override ircService.servers.
Bridge data (passkey.pem and database files) is now also stored in a subdirectory (/matrix/appservice-irc/data).
When you run the playbook with an existing /matrix/appservice-irc/passkey.pem file, the playbook will stop the bridge and relocate the passkey and database files (rooms.db and users.db) to the ./data directory. There's no data-loss involved. You'll need to restart the bridge manually though (--tags=start).
Until now, configuration files for the Telegram bridge were created by the playbook initially, but never modified later on.
From now on, the playbook will keep the configuration in sync for you.
This means that if you were making manual changes to the /matrix/mautrix-telegram/config.yaml or /matrix/mautrix-telegram/registration.yaml configuration files, those would be lost the next time you run the playbook.
The bridge now stores configuration in a subdirectory (/matrix/mautrix-telegram/config), so your old configuration remains in the base directory (/matrix/mautrix-telegram).
You need to migrate any manual changes over to the new matrix_mautrix_telegram_configuration_extension_yaml variable, so that the playbook would apply them for you.
Likewise, data is now also stored in a subdirectory (/matrix/mautrix-telegram/data). When you run the playbook with an existing database file (/matrix/mautrix-telegram/mautrix-telegram.db), the playbook will stop the bridge and relocate the database file to the ./data directory. There's no data-loss involved. You'll need to restart the bridge manually though (--tags=start).
Also, we're now following the default configuration for the Telegram bridge, so some default configuration values are different:
edits_as_replies(used to befalse, nowtrue) - previously replies were not sent over to Matrix at all; ow they are sent over as a reply to the original messageinline_images(used to betrue, nowfalse) - this has to do with captioned images. Inline-image (included caption) are said to exhibit troubles on Riot iOS. Whenfalse, the caption arrives on the Matrix side as a separate message.authless_portals(used to befalse, nowtrue) - creating portals from the Telegram side is now possiblewhitelist_group_admins(used to befalse, nowtrue) - allows Telegram group admins to use the bot commands
If the new values are not to your liking, use matrix_mautrix_telegram_configuration_extension_yaml to specify an override (refer to matrix_mautrix_telegram_configuration_yaml to figure out which variable goes where).
With Synapse v1.0 now available and most people being on at least Synapse v0.99, it's time to remove the _matrix._tcp DNS SRV record that we've been keeping for compatibility with old Synapse versions (<= 0.34).
According to the Server Discovery specification, it's no harm to keep the DNS SRV record. But since it's not necessary for federating with the larger Matrix network anymore, you should be safe to get rid of it.
Note: don't confuse the _matrix._tcp and _matrix-identity._tcp DNS SRV records. The latter, must not be removed.
For completeness, we must say that using a _matrix._tcp SRV record for Server Delegation is still valid and useful for certain deployments. It's just that our guide recommends the /.well-known/matrix/server Server Delegation method, due to its easier implementation when using this playbook.
Besides this optional/non-urgent DNS change, assuming you're already on Synapse v0.99, upgrading to Synapse v1.0 should be as simple as re-running the playbook.
Until now, configuration files for the Facebook bridge were created by the playbook initially, but never modified later on.
From now on, the playbook will keep the configuration in sync for you.
This means that if you were making manual changes to the /matrix/mautrix-facebook/config.yaml or /matrix/mautrix-facebook/registration.yaml configuration files, those would be lost the next time you run the playbook.
The bridge now stores configuration in a subdirectory (/matrix/mautrix-facebook/config), so your old configuration remains in the base directory (/matrix/mautrix-facebook).
You need to migrate any manual changes over to the new matrix_mautrix_facebook_configuration_extension_yaml variable, so that the playbook would apply them for you.
Likewise, data is now also stored in a subdirectory (/matrix/mautrix-facebook/data). When you run the playbook with an existing database file (/matrix/mautrix-facebook/mautrix-facebook.db), the playbook will stop the bridge and relocate the database file to the ./data directory. There's no data-loss involved. You'll need to restart the bridge manually though (--tags=start).
Until now, various roles supported a matrix_*_expose_port variable, which would expose their container's port to the host. This was mostly useful for reverse-proxying manually (in case matrix-nginx-proxy was disabled). It could also be used for installing some playbook services (e.g. bridges, etc.) and wiring them to a separate (manual) Matrix setup.
matrix_*_expose_port variables were not granular enough - sometimes they would expose one port, other times multiple. They also didn't provide control over where to expose (to which port number and to which network interface), because they would usually hardcode something like 127.0.0.1:8080.
All such variables have been superseded by a better (more flexible) way to do it.
Most people (including those not using matrix-nginx-proxy), don't need to bother with this.
Porting examples follow for people having more customized setups:
-
from
matrix_synapse_container_expose_client_api_port: truetomatrix_synapse_container_client_api_host_bind_port: '127.0.0.1:8008' -
from
matrix_synapse_container_expose_federation_api_port: truetomatrix_synapse_container_federation_api_plain_host_bind_port: '127.0.0.1:8048'and possiblymatrix_synapse_container_federation_api_tls_host_bind_port: '8448' -
from
matrix_synapse_container_expose_metrics_port: truetomatrix_synapse_container_metrics_api_host_bind_port: '127.0.0.1:9100' -
from
matrix_riot_web_container_expose_port: truetomatrix_riot_web_container_http_host_bind_port: '127.0.0.1:8765' -
from
matrix_mxisd_container_expose_port: truetomatrix_mxisd_container_http_host_bind_port: '127.0.0.1:8090' -
from
matrix_dimension_container_expose_port: truetomatrix_dimension_container_http_host_bind_port: '127.0.0.1:8184' -
from
matrix_corporal_container_expose_ports: truetomatrix_corporal_container_http_gateway_host_bind_port: '127.0.0.1:41080'and possiblymatrix_corporal_container_http_api_host_bind_port: '127.0.0.1:41081' -
from
matrix_appservice_irc_container_expose_client_server_api_port: truetomatrix_appservice_irc_container_http_host_bind_port: '127.0.0.1:9999' -
from
matrix_appservice_discord_container_expose_client_server_api_port: truetomatrix_appservice_discord_container_http_host_bind_port: '127.0.0.1:9005'
As always, if you forget to remove usage of some outdated variable, the playbook will warn you.
Thanks to @danbob, the playbook now supports the new Ansible 2.8.
A manual change is required to the inventory/hosts file, changing the group name from matrix-servers to matrix_servers (dash to underscore).
To avoid doing it manually, run this:
- Linux:
sed -i 's/matrix-servers/matrix_servers/g' inventory/hosts - Mac:
sed -i '' 's/matrix-servers/matrix_servers/g' inventory/hosts
The playbook no longer insists on installing Synapse via the matrix-synapse role.
If you would prefer to install Synapse another way and just use the playbook to install other services, it should be possible (matrix_synapse_enabled: false).
Note that it won't necessarily be the best experience, since the playbook wires things to Synapse by default. If you're using your own Synapse instance (especially one not running in a container), you may have to override many variables to point them to the correct place.
Having Synapse not be a required component potentially opens the door for installing alternative Matrix homeservers.
Bridges are no longer part of the matrix-synapse role.
Each bridge now lives in its own separate role (roles/matrix-bridge-*).
These bridge roles are independent of the matrix-synapse role, so it should be possible to use them with a Synapse instance installed another way (not through the playbook).
For better consistency, the following variables have been renamed:
matrix_enable_room_list_searchwas renamed tomatrix_synapse_enable_room_list_searchmatrix_alias_creation_ruleswas renamed tomatrix_synapse_alias_creation_rulesmatrix_nginx_proxy_matrix_room_list_publication_rulesdata_pathwas renamed tomatrix_synapse_room_list_publication_rules
Besides a myriad of bug fixes and minor improvements, here are the more notable (bigger) features we can announce today.
The playbook now supports bridging with Facebook by installing the mautrix-facebook bridge. This playbook functionality is available thanks to @izissise.
Additional details are available in Setting up Mautrix Facebook bridging.
The playbook can now help you integrate with mxisd's Registration feature.
Learn more in mxisd-controlled Registration.
If you prefer using the Caddy webserver instead of our own integrated nginx, we now have examples for it in the examples/caddy directory
Until now, you could optionally host Synapse's media repository on Amazon S3, but we now also support using other S3-compatible object stores,
Due to recent playbook improvements and the fact that the world keeps turning, we're bumping the version requirement for Ansible (2.4 -> 2.5).
We've also started building our own Docker image of Ansible (devture/ansible), which is useful for people who can't upgrade their local Ansible installation (see Using Ansible via Docker).
We've added TLS support to the Coturn TURN server installed by the playbook by default. The certificates from the Matrix domain will be used for the Coturn server.
This feature is enabled by default for new installations. To make use of TLS support for your existing Matrix server's Coturn, make sure to rebuild both Coturn and Synapse:
ansible-playbook -i inventory/hosts setup.yml --tags=setup-coturn,setup-synapse,startPeople who have an extra firewall (besides the iptables firewall, which Docker manages automatically), will need to open these additional firewall ports: 5349/tcp (TURN over TCP) and 5349/udp (TURN over UDP).
People who build their own custom playbook from our roles should be aware that:
-
the
matrix-coturnrole and actually starting Coturn (e.g.--tags=start), requires that certificates are already put in place. For this reason, it's usually a good idea to have thematrix-coturnrole execute aftermatrix-nginx-proxy(which retrieves the certificates). -
there are a few variables that can help you enable TLS support for Coturn. See the
matrix-coturnsection in group_vars/matrix-servers.
If you don't have a dedicated server for your base domain and want to set up Server Delegation via a well-known file, the playbook has got you covered now.
It's now possible for the playbook to obtain an SSL certificate and serve the necessary files for Matrix Server Delegation on your base domain. Take a look at the new Serving the base domain documentation page.
matrix_nginx_proxy_data_path was renamed to matrix_nginx_proxy_base_path.
There's a new matrix_nginx_proxy_data_path variable, which has a different use-purpose now (it's a subdirectory of matrix_nginx_proxy_base_path and is meant for storing various data files).
Thanks to NullIsNot0, the playbook can now (optionally) install the Dimension Integration Manager. To learn more, see the Setting up Dimension documentation page.
Thanks to Sylvia van Os, mxisd's email templates can now be customized easily. To learn more, see the Customizing email templates documentation page.
@Lionstiger has done some great work adding Discord bridging support via matrix-appservice-discord. To learn more, see the Setting up Appservice Discord bridging documentation page.
The following playbook variables were renamed:
- from
host_specific_hostname_identitytomatrix_domain - from
hostname_identitytomatrix_domain - from
hostname_matrixtomatrix_server_fqn_matrix - from
hostname_riottomatrix_server_fqn_riot - from
host_specific_matrix_ssl_lets_encrypt_support_emailtomatrix_ssl_lets_encrypt_support_email
Doing that, we've simplified things, made names less confusing (hopefully) and moved all variable names under the matrix_ prefix.
You can now use the brand new and redesigned Riot.
The new version no longer has a homepage by default, so we've also removed the custom homepage that we've been installing.
However, we still provide you with hooks to install your own home.html file by specifying the matrix_riot_web_embedded_pages_home_path variable (used to be called matrix_riot_web_homepage_template before).
As we're moving toward Synapse v1.0, things are beginning to stabilize. Upgrading from v0.99.0 to v0.99.1 should be painless.
If you've been overriding the default configuration so that you can terminate TLS at the Synapse side (matrix_synapse_no_tls: false), you'll now have to replace this custom configuration with matrix_synapse_tls_federation_listener_enabled: true. The matrix_synapse_no_tls variable is no more.
Matrix is undergoing a lot of changes as it matures towards Synapse v1.0. The first step is the Synapse v0.99 transitional release, which this playbook now supports.
If you've been using this playbook successfully until now, you'd be aware that we've been doing Server Delegation using a _matrix._tcp DNS SRV record (as per Configuring DNS).
Due to changes related to certificate file requirements that will affect us at Synapse v1.0, we'll have to stop using a _matrix._tcp DNS SRV record in the future (when Synapse goes to v1.0 - around 5th of March 2019). We still need to keep the SRV record for now, for backward compatibility with older Synapse versions (lower than v0.99).
What you need to do now is make use of this transitional Synapse v0.99 release to prepare your federation settings for the future. You have 2 choices to prepare yourself for compatibility with the future Synapse v1.0:
-
(recommended) set up Server Delegation via a well-known file, unless you are affected by the Downsides of well-known-based Server Delegation. If you had previously set up the well-known
clientfile, depending on how you've done it, it may be that there is nothing new required of you (besides upgrading). After upgrading, you can run a self-check, which will tell you if you need to do anything extra with regard to setting up Server Delegation via a well-known file. After some time, when most people have upgraded to Synapse v0.99 and older releases have disappeared, be prepared to drop your_matrix._tcpSRV record. -
(more advanced) if the Downsides of well-known-based Server Delegation are not to your liking, as an alternative, you can set up Server Delegation via a DNS SRV record. In such a case, you get to keep using your existing
_matrix._tcpDNS SRV record forever and need to NOT set up a/.well-known/matrix/serverfile. Don't forget that you need to do certificate changes though. Follow the guide at Server Delegation via a DNS SRV record.
Now that the nginx Docker image has added support for TLS v1.3, we have enabled that protocol by default.
When using:
-
the integrated nginx server: TLS v1.3 support might not kick in immediately, because the nginx version hasn't been bumped and you may have an older build of the nginx Docker image (currently
nginx:1.15.8-alpine). Typically, we do not re-pull images that you already have. When the nginx version gets bumped in the future, everyone will get the update. Until then, you could manually force-pull the rebuilt Docker image by running this on the server:docker pull nginx:1.15.8-alpine. -
your own external nginx server: if your external nginx server is too old, the new configuration we generate for you in
/matrix/nginx-proxy/conf.d/might not work anymore, because it mentionsTLSv1.3and your nginx version might not support that. You can adjust the SSL protocol list by overriding thematrix_nginx_proxy_ssl_protocolsvariable. Learn more in the documentation page for Using your own webserver, instead of this playbook's nginx proxy -
another web server: you don't need to do anything to accommodate this change
Devon Maloney (@Plailect) has done some great work bringing IRC bridging support via matrix-appservice-irc. To learn more, see the Setting up Appservice IRC documentation page.
To improve security, this playbook no longer starts container processes as the root user.
Most containers were dropping privileges anyway, but we were trusting them with root privileges until they would do that.
Not anymore -- container processes now start as a non-root user (usually matrix) from the get-go.
For additional security, various capabilities are also dropped (see why it's important) for all containers.
Additionally, most containers now use a read-only filesystem (see why it's important). Containers are given write access only to the directories they need to write to.
A minor breaking change is the matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size variable having being renamed to matrix_nginx_proxy_proxy_matrix_client_api_client_max_body_size_mb (note the _mb suffix). The new variable expects a number value (e.g. 25M -> 25).
If you weren't customizing this variable, this wouldn't affect you.
While we would have preferred to stay with Postfix, we found out that it cannot run as a non-root user. We've had to replace it with Exim (via the devture/exim-relay container image).
The internal matrix-mailer service (running in a container) now listens on port 8025 (used to be 587 before).
The playbook will update your Synapse and mxisd email settings to match (matrix-mailer:587 -> matrix-mailer:8025).
Using the devture/exim-relay container image instead of panubo/postfix also gives us a nice disk usage reduction (~200MB -> 8MB).
The following change affects people running a more non-standard setup - external Postgres or using our roles in their own other playbook.
Most users don't need to do anything, besides becoming aware of the new glue variables file group_vars/matrix-servers.
Because people like using the playbook's components independently (outside of this playbook) and because it's much better for maintainability, we've continued working on separating them. Still, we'd like to offer a turnkey solution for running a fully-featured Matrix server, so this playbook remains important for wiring up the various components.
With the new changes, all roles are now only dependent on the minimal matrix-base role. They are no longer dependent among themselves.
In addition, the following components can now be completely disabled (for those who want/need to):
matrix-coturnby usingmatrix_coturn_enabled: falsematrix-mailerby usingmatrix_mailer_enabled: falsematrix-postgresby usingmatrix_postgres_enabled: false
The following changes had to be done:
-
glue variables had to be introduced to the playbook, so it can wire together the various components. Those glue vars are stored in the
group_vars/matrix-serversfile. When overriding variables for a given component (role), you need to be aware of both the role defaults (role/ROLE/defaults/main.yml) and the role's corresponding section in thegroup_vars/matrix-serversfile. -
matrix_postgres_use_externalhas been superceeded by the more consistently namedmatrix_postgres_enabledvariable and a few othermatrix_synapse_database_variables. See the Using an external PostgreSQL server (optional) documentation page for an up-to-date replacement. -
Postgres tools (
matrix-postgres-cliandmatrix-make-user-admin) are no longer installed if you're not enabling thematrix-postgresrole (matrix_postgres_enabled: false) -
roles, being more independent now, are more minimal and do not do so much magic for you. People that are building their own playbook using our roles will definitely need to take a look at the
group_vars/matrix-serversfile and adapt their playbooks with the same (or similar) wiring logic.
For better maintainability, the playbook logic (which all used to reside in a single matrix-server role)
has been split out into a number of different roles: matrix-synapse, matrix-postgres, matrix-riot-web, matrix-mxisd, etc. (see the roles/ directory).
To keep the filesystem more consistent with this separation, the Postgres data had to be relocated.
The default value of matrix_postgres_data_path was changed from /matrix/postgres to /matrix/postgres/data. The /matrix/postgres directory is what we consider a base path now (new variable matrix_postgres_base_path). Your Postgres data files will automatically be relocated by the playbook (/matrix/postgres/* -> /matrix/postgres/data/) when you run with --tags=setup-all (or --tags=setup-postgres). While this shouldn't cause data-loss, it's better if you do a Postgres backup just in case. You'd need to restart all services after this migration (--tags=start).
To be more flexible and to support the upcoming mxisd 1.3.0 (when it gets released), we've had to redo how mxisd gets configured.
The following variables are no longer supported by this playbook:
matrix_mxisd_ldap_enabledmatrix_mxisd_ldap_connection_hostmatrix_mxisd_ldap_connection_tlsmatrix_mxisd_ldap_connection_portmatrix_mxisd_ldap_connection_baseDnmatrix_mxisd_ldap_connection_baseDnsmatrix_mxisd_ldap_connection_bindDnmatrix_mxisd_ldap_connection_bindDnmatrix_mxisd_ldap_connection_bindPasswordmatrix_mxisd_ldap_filtermatrix_mxisd_ldap_attribute_uid_typematrix_mxisd_ldap_attribute_uid_valuematrix_mxisd_ldap_connection_bindPasswordmatrix_mxisd_ldap_attribute_namematrix_mxisd_ldap_attribute_threepid_emailmatrix_mxisd_ldap_attribute_threepid_msisdnmatrix_mxisd_ldap_identity_filtermatrix_mxisd_ldap_identity_mediummatrix_mxisd_ldap_auth_filtermatrix_mxisd_ldap_directory_filtermatrix_mxisd_template_config
You are encouraged to use the matrix_mxisd_configuration_extension_yaml variable to define your own mxisd configuration additions and overrides.
Refer to the default variables file for more information.
This new way of configuring mxisd is beneficial because:
- it lets us support all mxisd configuration options, as the playbook simply forwards them to mxisd without needing to care or understand them
- it lets you upgrade to newer mxisd versions and make use of their features, without us having to add support for them explicitly
Due to the way we manage cronjobs now, you can no longer configure the schedule they're invoked at.
If you were previously using matrix_ssl_lets_encrypt_renew_cron_time_definition or matrix_nginx_proxy_reload_cron_time_definition
to set a custom schedule, you should note that these variables don't affect anything anymore.
If you miss this functionality, please open an Issue and let us know about your use case!
The playbook now lets you decide between 3 different SSL certificate retrieval methods:
- (default) obtaining free SSL certificates from Let's Encrypt
- generating self-signed SSL certificates
- managing SSL certificates manually
Learn more in Adjusting SSL certificate retrieval.
For people who use Let's Encrypt (mostly everyone, since it's the default), you'll also have to rename a variable in your configuration:
- before:
host_specific_matrix_ssl_support_email - after:
host_specific_matrix_ssl_lets_encrypt_support_email
mxisd has bee upgraded to version 1.2.2, which supports multiple base DNs.
If you were configuring this playbook's matrix_mxisd_ldap_connection_baseDn variable until now (a string containing a single base DN), you'll need to change to configuring the matrix_mxisd_ldap_connection_baseDns variable (an array containing multiple base DNs).
Example change:
- before:
matrix_mxisd_ldap_connection_baseDn: OU=Users,DC=example,DC=org - after:
matrix_mxisd_ldap_connection_baseDns: ['OU=Users,DC=example,DC=org']
Synapse has been upgraded to 0.34.0 and now uses Python 3. Based on feedback from others, running Synapse on Python 3 is supposed to decrease memory usage significantly (~2x).
You can now customize some parts of the Riot homepage (or even completely replace it with your own custom page).
See the matrix_riot_web_homepage_ variables in roles/matrix-riot-web/defaults/main.yml.
The LDAP identity store for mxisd can now be configured easily using playbook variables (see the matrix_mxisd_ldap_ variables in roles/matrix-server/defaults/main.yml).
- matrix-remove-all allows to uninstall everything with a single command
- matrix-make-user-admin allows to upgrade a user's privileges
The playbook can now install and configure LDAP auth support for you.
Additional details are available in Setting up the LDAP authentication password provider module.
The playbook now lets you enable public registration for users (controlled via matrix_synapse_enable_registration).
By default, public registration is forbidden.
You can also make people automatically get auto-joined to rooms (controlled via matrix_synapse_auto_join_rooms).
By default, @riot-bot:matrix.org is used to welcome newly registered users.
This can be changed to something else (or disabled) via the new matrix_riot_web_welcome_user_id variable.
The playbook now allows you to set the log levels used by Synapse. The default logging levels remain the same.
You can now override following variables with any of the supported log levels listed here: https://docs.python.org/3/library/logging.html#logging-levels
matrix_synapse_log_level: "INFO"
matrix_synapse_storage_sql_log_level: "INFO"
matrix_synapse_root_log_level: "INFO"
You can now customize some parts of Riot's config.json. These playbook variables, with these default values, have been added:
matrix_riot_web_disable_custom_urls: true
matrix_riot_web_disable_guests: true
matrix_riot_web_integrations_ui_url: "https://scalar.vector.im/"
matrix_riot_web_integrations_rest_url: "https://scalar.vector.im/api"
matrix_riot_web_integrations_widgets_urls: "https://scalar.vector.im/api"
matrix_riot_web_integrations_jitsi_widget_url: "https://scalar.vector.im/api/widgets/jitsi.html"
This now allows you use a custom integrations manager like Dimesion. For example, if you wish to use the Dimension instance hosted at dimension.t2bot.io, you can set the following in your vars.yml file:
matrix_riot_web_integrations_ui_url: "https://dimension.t2bot.io/riot"
matrix_riot_web_integrations_rest_url: "https://dimension.t2bot.io/api/v1/scalar"
matrix_riot_web_integrations_widgets_urls: "https://dimension.t2bot.io/widgets"
matrix_riot_web_integrations_jitsi_widget_url: "https://dimension.t2bot.io/widgets/jitsi"
There's now a new matrix_nginx_proxy_ssl_protocols playbook variable, which controls the SSL protocols used to serve Riot and Synapse. Its default value is TLSv1.1 TLSv1.2. This playbook previously used TLSv1 TLSv1.1 TLSv1.2 to serve Riot and Synapse.
You may wish to reenable TLSv1 if you need to access Riot in older browsers.
Note: Currently the dockerized nginx doesn't support TLSv1.3. See nginxinc/docker-nginx#190 for more details.
The playbook now installs Postgres 11 by default.
If you have have an existing setup, it's likely running on an older Postgres version (9.x or 10.x). You can easily upgrade by following the upgrading PostgreSQL guide.
Due to the large amount of features added to this playbook lately, to keep things manageable we've had to reorganize its configuration variables a bit.
The following playbook variables were renamed:
- from
matrix_docker_image_mxisdtomatrix_mxisd_docker_image - from
matrix_docker_image_mautrix_telegramtomatrix_mautrix_telegram_docker_image - from
matrix_docker_image_mautrix_whatsapptomatrix_mautrix_whatsapp_docker_image - from
matrix_docker_image_mailertomatrix_mailer_docker_image - from
matrix_docker_image_coturntomatrix_coturn_docker_image - from
matrix_docker_image_goofystomatrix_s3_goofys_docker_image - from
matrix_docker_image_riottomatrix_riot_web_docker_image - from
matrix_docker_image_nginxtomatrix_nginx_proxy_docker_image - from
matrix_docker_image_synapsetomatrix_synapse_docker_image - from
matrix_docker_image_postgres_v9tomatrix_postgres_docker_image_v9 - from
matrix_docker_image_postgres_v10tomatrix_postgres_docker_image_v10 - from
matrix_docker_image_postgres_latesttomatrix_postgres_docker_image_latest
The playbook now supports bridging with Whatsapp by installing the mautrix-whatsapp bridge. This playbook functionality is available thanks to @izissise.
Additional details are available in Setting up Mautrix Whatsapp bridging.
The playbook can now help you with Controlling Matrix federation, should you wish to run a more private (isolated) server.
From now on, Riot's configuration setting disable_guests would be set to true.
The homeserver was rejecting guests anyway, so this is just a cosmetic change affecting Riot's UI.
The playbook can now check if services are configured correctly.
The playbook can now enable/disable user presence-status tracking in Synapse, through the playbook's matrix_synapse_use_presence variable (having a default value of true - enabled).
If users participate in large rooms with many other servers, disabling presence will decrease server load significantly.
The playbook now makes the Synapse cache factor configurable, through the playbook's matrix_synapse_cache_factor variable (having a default value of 0.5).
Changing that value allows you to potentially decrease RAM usage or to increase performance by caching more stuff. Some information on it is available here: https://github.com/matrix-org/synapse#help-synapse-eats-all-my-ram
--log-driver=none is used for all Docker containers now.
All these containers are started through systemd anyway and get logged in journald, so there's no need for Docker to be logging the same thing using the default json-file driver. Doing that was growing /var/lib/docker/containers/.. infinitely until service/container restart.
As a result of this, things like docker logs matrix-synapse won't work anymore. journalctl -u matrix-synapse is how one can see the logs.
The playbook now helps you set up service discovery using a /.well-known/matrix/client file.
Additional details are available in Configuring service discovery via .well-known.
The following playbook variables were renamed:
- from
matrix_nginx_riot_web_data_pathtomatrix_riot_web_data_path - from
matrix_riot_web_default_identity_server_urltomatrix_identity_server_url
The playbook now supports bridging with Telegram by installing the mautrix-telegram bridge. This playbook functionality is available thanks to @izissise.
Additional details are available in Setting up Mautrix Telegram bridging.
The playbook now lets you configure Matrix Synapse's event_cache_size configuration via the matrix_synapse_event_cache_size playbook variable.
Previously, this value was hardcoded to "10K". From now on, a more reasonable default of "100K" is used.
The playbook now supports enabling password-peppering for increased security in Matrix Synapse via the matrix_synapse_password_config_pepper playbook variable. Using a password pepper is disabled by default (just like it used to be before this playbook variable got introduced) and is not to be enabled/disabled after initial setup, as that would invalidate all existing passwords.
There's now a new matrix_synapse_report_stats playbook variable, which controls the report_stats configuration option for Matrix Synapse. It defaults to false, so no change is required to retain your privacy.
If you'd like to start reporting statistics about your homeserver (things like number of users, number of messages sent, uptime, load, etc.) to matrix.org, you can turn on stats reporting.
We've been using acmetool (with the willwill/acme-docker Docker image) until now.
Due to the Docker image being deprecated, and things looking bleak for acmetool's support of the newer ACME v2 API endpoint, we've switched to using certbot (with the certbot/certbot Docker image).
Simply re-running the playbook will retrieve new certificates (via certbot) for you. To ensure you don't leave any old files behind, though, you'd better do this:
systemctl stop 'matrix*'- stop your custom webserver, if you're running one (only affects you if you've installed with
matrix_nginx_proxy_enabled: false) mv /matrix/ssl /matrix/ssl-acmetool-delete-later- re-run the playbook's installation
- possibly delete
/matrix/ssl-acmetool-delete-later
The playbook can now install and configure matrix-corporal for you.
Additional details are available in Setting up Matrix Corporal.
The following new variables can now be configured to control Matrix Synapse's rate-limiting (default values are shown below).
matrix_synapse_rc_messages_per_second: 0.2
matrix_synapse_rc_message_burst_count: 10.0The playbook can now install and configure matrix-synapse-shared-secret-auth for you.
Additional details are available in Setting up the Shared Secret Auth password provider module.
The playbook can now install and configure matrix-synapse-rest-auth for you.
Additional details are available in Setting up the REST authentication password provider module.
Shifted Matrix Synapse compression from happening in the Matrix Synapse, to happening in the nginx proxy that's in front of it.
Additionally, riot-web also gets compressed now (in the nginx proxy),
which drops the initial page load's size from 5.31MB to 1.86MB.
The following services are not necessary, so they have been disabled:
- on the federation port (8448): the
clientservice - on the http port (8008, exposed over 443): the old Angular
webclientand thefederationservice
Federation runs only on the federation port (8448) now. The Client APIs run only on the http port (8008) now.
The playbook now sets up an mxisd Identity Server for you by default. Additional details are available in Adjusting mxisd Identity Server configuration.
The playbook now configures an email-sending service (postfix) by default. Additional details are available in Adjusting email-sending settings.
With this, Matrix Synapse is able to send email notifications for missed messages, etc.
The following playbook variables were renamed:
- from
matrix_max_upload_size_mbtomatrix_synapse_max_upload_size_mb - from
matrix_max_log_file_size_mbtomatrix_synapse_max_log_file_size_mb - from
matrix_max_log_files_counttomatrix_synapse_max_log_files_count - from
docker_matrix_imagetomatrix_docker_image_synapse - from
docker_nginx_imagetomatrix_docker_image_nginx - from
docker_riot_imagetomatrix_docker_image_riot - from
docker_goofys_imagetomatrix_docker_image_goofys - from
docker_coturn_imagetomatrix_docker_image_coturn
If you're overriding any of them in your vars.yml file, you'd need to change to the new names.
The command for executing the whole playbook has changed.
The setup-main tag got renamed to setup-all.
Changed the way the Docker containers are linked together. The ones that need to communicate with others operate in a matrix network now and not in the default bridge network.