Skip to content

Commit aba4834

Browse files
raidensakurakhakersTaaku18
authored
Merge develop into stable for v4.2.0 (#23)
* A few optimizations - Reduce Docker image size - Migrate requirements.txt to Pipfile - Remove unused dependencies - Add a few unnecessary files in .dockerignore * Pull modmail-dev#3281 * Pull modmail-dev#3279 * Pull modmail-dev#3278 * Pull modmail-dev#3277 & proper indentation * Squashed commit of the following: commit 2e32d6e Merge: 51a023c b21037c Author: Martin <[email protected]> Date: Sun Apr 9 21:22:08 2023 +0200 Merge branch 'modmail-dev:master' into master commit 51a023c Author: Martin <[email protected]> Date: Mon Apr 3 13:35:56 2023 +0200 Changelog+Small fixes commit f5f25b5 Author: Martin <[email protected]> Date: Mon Apr 3 13:10:35 2023 +0200 confirm_thread_creation Buttons instead of reactions commit 11e094c Merge: ebd38d3 e919304 Author: Taku <[email protected]> Date: Tue Mar 14 05:19:54 2023 -0700 Merge branch 'master' into master commit ebd38d3 Author: Taku <[email protected]> Date: Tue Mar 14 05:18:17 2023 -0700 Fix indentations commit b77e435 Author: Martin <[email protected]> Date: Sun Mar 12 17:22:42 2023 +0100 Reminder Plugin Created a reminder plugin * Pull modmail-dev/modmail#modmail-dev#3271 * Squashed commit of the following: commit 13437cd Author: baptiste0928 <[email protected]> Date: Tue Mar 21 22:29:24 2023 +0100 Format with black commit 8a6ecd6 Merge: d9e364e 02250e8 Author: Taku <[email protected]> Date: Tue Mar 21 13:30:40 2023 -0700 Merge branch 'development' into feat/guild-icon-size Signed-off-by: Taku <[email protected]> commit d9e364e Author: baptiste0928 <[email protected]> Date: Sun Mar 19 20:17:35 2023 +0100 Update changelog commit d0bfeb4 Author: baptiste0928 <[email protected]> Date: Sun Mar 19 20:06:20 2023 +0100 Set size for guild icons on embeds * Format with black * Pull modmail-dev/modmail#modmail-dev#3233 * Squashed commit of the following: commit cf3444a Author: Khakers <[email protected]> Date: Tue Jan 24 14:13:15 2023 -0800 Add dm_channel_id field to log entry * Update changelog, workflow & remove bandit * Pull modmail-dev#3272 + lock dependencies * Add docs boilerplate + bumppython version * Dpy bump, add versions for ?about * Format with black * Update README.md * Fix permission error when installing plugin * Add new mention_message config, black + ruff * Switch from root to app user for Docker * Add support for trailing space in prefix * Add logviewer as built-in plugin * logviewer plugin enhancements * Minor logviewer changes * Update changelog + logviewer pomelo change * add pomelo for logbase * Change logviewer favicon * Refactor loglist object with proper serialization * Remove favicon to untrack later * fix log entry key error * Replace local favicon with remote avatar * Add markdown header support * Switch repo link and branch * Change more repo links * Handle properly when loglist is the index page * toggle off internal message checkbox on load * Remove trailing slash, fix broken log url on index * Fix broken default avatar url * Fix pathing issue on displaying log list in index * Add placeholder for not logged in user * Fix blockquote parsing emote ending * Whoops, remove comment * Fix blockquote parsing + oauth navbar indicator * Update docs * config and docs change * Update FUNDING.yml * Update docs * Update ignores + workflow * Tweak loadcheck * tweak loadcheck (2) * blegh * ✨ * Attempt to fix invalid guild_id on test * Docs update * Terminate process more gracefully * Update dependencies for Python 3.11 * Refactor logviewer pomelo changes * Minify docker image size * Remove unused files * bump black to fix checks * Update dev requirements * Update workflow * More pomelo changes * Replace avatar with local asset * Fix broken author avatar on `logs` command * Implement pomelo on channel name * Format with black * Add pre-commit hook * Update requirements * Overhaul block functionality * Add plugin uninstall alias * Update python badge * Update README.md * Update README.md * Fix block datetime not being timezone aware * Remove cairosvg * Fix copy paste mistake * Another copy paste issue sigh * Fix NoneType on parse * Improve block check and add message cooldown * bump dpy * Update internal dpy version * Update ?info command for pomelo * Bump version * Fix missing package error * Fix index compile error * Fix space in filepath failing to load local plugin * Escape codeblock language in class name * Fix genesis message retrieval failure * Update docs * Don't build image on PR * Blocklist rewrite (#11) * Add blocklist blocklist.py contains the initial classes and code for the new blocklist system. The intention of this system is to unify blocking to one new mongodb collection and vastly simplify bot code required to manage it. Additionally, most simply methods should be side effect free. * Add migrations.py migrations.py contains functions to migrate legacy blocklists to the new blocklist. contains handling for both legacy modmail string and the newer dict storage * Add blocklist bot.py integration * rename BlockedUser to BlocklistItem & add more blocklist methods Adds add_block, unblock_id, and is_id_blocked to blocklist class * migrate block and unblock commands to new blocklist * fix overzealous ide rename * add get_all_blocks method and documentation + cleanup to blocklist.py * refactor blocked command to use new block system * cleanup block command code now with more DRY * migrate bot.py is_blocked method to new blocklist system * migrate whitelist command and refactor is_id_blocked to return tuple is_id_blocked now returns a tuple[bool, Optional[BlocklistItem]] so the calling code can get information about the block easily * rename BlocklistItem to BlocklistEntry * Add deprecation warning logs to legacy block code and docstrings to bot.py block related code * clean up migrations.py code * Add [p]migration command * add final deprecation docstrings and refactor contact command to use blocklist.is_user_blocked() instead of core is_blocked() * remove blocklist print statements * Format with black --------- Co-authored-by: Raiden Sakura <[email protected]> * fix note code upgrade signature of note code state store_user method to match expected discord.py method use a typed checked dict for author instead of making a declaring a class in a for loop that replicates a different type. hopefully a type checker would raise warnings the next time one of these changes * Fix failing tests and docker build * fix blocklist bugs (#13) * fix missing await statements in new block method usage * remove Self type importing and usage * rename placeholder variable name * StrEnum is only in 3.11... --------- Co-authored-by: Raiden Sakura <[email protected]> * oops * Fix object is not subscriptable for 3.8 * Pull 2 changes from upstream (#14) * Fixed unset registry_plugins_only causing non-registry plugins to fail to install * To avoid confusion, config.get() only accepts one pos argument now --------- Co-authored-by: Taku <[email protected]> * Format with black * More blocklist fixes (#15) * update requirements.txt * fix blocklist migration detection * fix migrations.py type hints for python 3.8 * Add index for channel_id Channel_id is the main query field for nearly every log related database operation and for some reason it isn't indexed. This should speed up all log related database reliant operations especially on large collections * Update CHANGELOG.md Update changelog with my changes * Remove CairoSVG dependencies from Dockerfile * Improve is_image detection to actually detect images (excluding videos) and add content_type value * Fix logviewer positional arg error * partially reverts 2f76202 which added lottie sticker support and updates dependency lock (#20) * partially reverts 2f76202 which added lottie sticker support and updates dependency lock * Lint --------- Co-authored-by: Raiden Sakura <[email protected]> * Synchronize thread NSFW status and title with the database (#18) * Changing the thread title or nsfw status now immediately updates in the database !Changes signature of thread.set_title() adds update_nsfw and update_title methods to the mongodb client * Lint --------- Co-authored-by: Raiden Sakura <[email protected]> * Refactor docker workflow to multiplatform build (#21) Upgrades out of date action & adds dependency actions (QEMU & BuildX) for multiplatform build. Sets platforms to Linux/amd64 and Linux/arm64 * Update funding * Remove internal logviewer plugin * Add .gitignore for local plugin * Migrate to poetry * Update workflow for poetry * Update workflow * Oops, forgot to install deps * Update launch command * Attempt to solve image not building arm64 * Add missing libffi * Refactor Dockerfile * Resolve deprecated pkg_resources warning * Update changelog * Bump workflow versions * oops * Add [speed] extra for dpy * Bump dpy, don't exit when dpy version mismatch * Fix sticker replies not sending * Update changelog * Improved join/leave message for multiple servers. From modmail-dev@0086e04 * Fix code scanning alert #5 * Fixed compat with MissingRequiredArgument requiring additional param modmail-dev@9464c5d * Fix plugin update bug on broken plugin https://github.com/modmail-dev/Modmail/pull/3295/files * Update CHANGELOG.md * Fix rate limit issue on raw reaction add/remove events https://github.com/modmail-dev/Modmail/pull/3306/files * Remove Heroku Reference from Debug Command modmail-dev@d891ad1 * Update CHANGELOG.md * Remove unused files * Update pip * Bump aiohttp and use [speedups] * Update .dockerignore * Rename project * Add shell script for pm2 * Update docs * Update env example * Remove app.json * Update dependencies * Update readme * Add config to manage how long logs are stored (modmail-dev#3257) * Add presence intent into .env.example * Formating * Fix missing guild icon url * Revert gyazo image embedding * Fix overly permissive regex * Specify fork on metadata * Update changelog * Bump version --------- Co-authored-by: khakers <[email protected]> Co-authored-by: Taku <[email protected]>
1 parent e46fbb1 commit aba4834

File tree

116 files changed

+2761
-14861
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

116 files changed

+2761
-14861
lines changed

.dockerignore

+4-3
Original file line numberDiff line numberDiff line change
@@ -130,26 +130,27 @@ dmypy.json
130130
package-lock.json
131131
node_modules/
132132

133-
# Modmail
133+
# Unused Modmail files
134134
config.json
135135
plugins/
136136
!plugins/registry.json
137137
!plugins/@local/
138138
temp/
139139
test.py
140140

141-
# Other stuff
141+
# Unused repository files
142142
.env.example
143143
.gitignore
144144
.dockerignore
145145
.github/
146+
.pre-commit-config.yaml
146147
app.json
147148
Procfile
148-
pyproject.toml
149149
*.md
150150
.*.json
151151
Dockerfile
152152
docker-compose.yml
153+
requirements.txt
153154
LICENSE
154155
PRIVACY.md
155156

.env.example

+46-4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,47 @@
1+
# Your Discord bot token
12
TOKEN=MyBotToken
2-
LOG_URL=https://logviewername.herokuapp.com/
3-
GUILD_ID=1234567890
4-
OWNERS=Owner1ID,Owner2ID,Owner3ID
5-
CONNECTION_URI=mongodb+srv://mongodburi
3+
4+
# ID of your Discord server
5+
GUILD_ID=123456
6+
7+
# ID of the server the thread channels will be created in
8+
# MODMAIL_GUILD_ID=123456
9+
10+
# Discord ID of the owners
11+
OWNERS=123456,123456
12+
13+
# Connection URI for your mongodb
14+
CONNECTION_URI=mongod+srv://mongo.net
15+
16+
# The url of logviewer for viewing self-hosted logs
17+
# LOG_URL=https://logviewername.domain.com/
18+
19+
# The path to your log viewer extending from your `LOG_URL`, defaults to "/logs"
20+
# LOG_URL_PREFIX=/logs
21+
22+
# Enable presence intent for the bot, defaults to false
23+
# ENABLE_PRESENCE_INTENT=false
24+
25+
# A github personal access token with the repo scope to access private repository
26+
# GITHUB_TOKEN=abcdef
27+
28+
# If set to true, only plugins that are in the registry can be loaded
29+
# REGISTRY_PLUGINS_ONLY=false
30+
31+
# Client ID for your application, used by logviewer`
32+
# OAUTH2_CLIENT_ID=12345
33+
34+
# Client Secret for your application, used by logviewer
35+
# OAUTH2_CLIENT_SECRET=abcdef
36+
37+
# The redirect URL for logviewer OAuth. You must also add this URL in your application's Redirects section
38+
# OAUTH2_REDIRECT_URI=https://logviewername.herokuapp.com/callback
39+
40+
# The host address logviewer will bind itself to. Defaults to 0.0.0.0
41+
# HOST=0.0.0.0
42+
43+
# The port logviewer plugin will listen to. Defaults to 8000
44+
# PORT=8000
45+
46+
# The number of logs to display per page on logviewer. Defaults to 25
47+
# LOGVIEWER_PAGINATION=25

.github/FUNDING.yml

+1
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
ko_fi: raidensakura
2+
github: raidensakura

.github/workflows/docker-image.yml

+14-8
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,6 @@ on:
88
- 'develop'
99
tags:
1010
- 'v*'
11-
pull_request:
12-
branches:
13-
- 'develop'
1411

1512
env:
1613
REGISTRY: ghcr.io
@@ -25,18 +22,24 @@ jobs:
2522

2623
steps:
2724
- name: Checkout repository
28-
uses: actions/checkout@v3
29-
25+
uses: actions/checkout@v4
26+
27+
- name: Set up QEMU
28+
uses: docker/setup-qemu-action@v3
29+
30+
- name: Set up Docker Buildx
31+
uses: docker/setup-buildx-action@v3
32+
3033
- name: Log in to the Container registry
31-
uses: docker/login-action@v2
34+
uses: docker/login-action@v3
3235
with:
3336
registry: ${{ env.REGISTRY }}
3437
username: ${{ github.actor }}
3538
password: ${{ secrets.GITHUB_TOKEN }}
3639

3740
- name: Extract metadata (tags, labels) for Docker
3841
id: meta
39-
uses: docker/metadata-action@v4
42+
uses: docker/metadata-action@v5
4043
with:
4144
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
4245
tags: |
@@ -51,9 +54,12 @@ jobs:
5154
type=raw,value=latest,enable=${{ github.event.release.prerelease == false }}
5255
5356
- name: Build and push Docker image
54-
uses: docker/build-push-action@v3
57+
uses: docker/build-push-action@v5
5558
with:
5659
context: .
60+
platforms: linux/amd64,linux/arm64
5761
push: true
5862
tags: ${{ steps.meta.outputs.tags }}
5963
labels: ${{ steps.meta.outputs.labels }}
64+
cache-from: type=gha
65+
cache-to: type=gha,mode=max

.github/workflows/lint.yml

+3-3
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,12 @@ jobs:
1212
runs-on: ubuntu-latest
1313
strategy:
1414
matrix:
15-
python-version: ["3.8"]
15+
python-version: ["3.11"]
1616
name: Python ${{ matrix.python-version }} on ubuntu-latest
1717
steps:
18-
- uses: actions/checkout@v3
18+
- uses: actions/checkout@v4
1919
- name: Set up Python
20-
uses: actions/setup-python@v4
20+
uses: actions/setup-python@v5
2121
with:
2222
python-version: ${{ matrix.python-version }}
2323
architecture: x64

.github/workflows/load-check.yml

+12-13
Original file line numberDiff line numberDiff line change
@@ -16,40 +16,39 @@ jobs:
1616

1717
name: Load Modmail - Python ${{ matrix.python-version }}
1818
steps:
19-
- uses: actions/checkout@v3
19+
- uses: actions/checkout@v4
2020
- name: Set up Python ${{ matrix.python-version }}
21-
uses: actions/setup-python@v4
21+
uses: actions/setup-python@v5
2222
with:
2323
python-version: ${{ matrix.python-version }}
24-
25-
- name: Cache venv
26-
id: cache-venv
24+
- uses: Gr1N/setup-poetry@v8
25+
- name: Cache poetry
26+
id: cache-poetry
2727
uses: actions/cache@v3
2828
with:
29-
path: .venv
30-
key: ${{ matrix.python-version }}-${{ hashFiles('Pipfile') }}-${{ secrets.CACHE_V }}
29+
path: ~/.cache/pypoetry/virtualenvs
30+
key: ${{ runner.os }}-poetry-${{ hashFiles('poetry.lock') }}
3131

32-
- name: Maybe make pipenv
32+
- name: Maybe install dependencies
3333
if: steps.cache-venv.outputs.cache-hit != 'true'
34-
run: |
35-
pip install pipenv && PIPENV_VENV_IN_PROJECT=1
36-
pipenv install --deploy
34+
run: poetry install --no-root
3735

3836
- name: Run script loadcheck.py
3937
run: |
40-
pipenv run python .github/workflows/scripts/loadcheck.py
38+
poetry run python .github/workflows/scripts/loadcheck.py
4139
env:
4240
TOKEN: ${{ secrets.DISCORD_TEST_BOT }}
4341
CONNECTION_URI: ${{ secrets.CONNECTION_URI }}
4442
LOG_URL: https://domain.example
4543
GUILD_ID: "616969119685935162"
44+
MODMAIL_GUILD_ID: "1100011945207271494"
4645
OWNERS: "243316261264556032,505386507440488458"
4746
DATA_COLLECTION: off
4847
LOG_LEVEL: DEBUG
4948

5049
- name: Save Modmail output as Artifact
5150
if: always() # still run if prev step failed
52-
uses: actions/upload-artifact@v3
51+
uses: actions/upload-artifact@v4
5352
with:
5453
name: "Modmail log - Python ${{ matrix.python-version }}"
5554
path: modmail.log

.github/workflows/scripts/loadcheck.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414

1515
file = open("modmail.log", "w")
1616
proc = subprocess.Popen(
17-
"pipenv run bot",
17+
"poetry run python bot.py",
1818
stdout=file,
1919
stderr=subprocess.STDOUT,
2020
shell=True,

CHANGELOG.md

+43
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,46 @@ however, insignificant breaking changes do not guarantee a major version bump, s
88

99
# [UNRELEASED]
1010

11+
# v4.2.0
12+
13+
### Breaking
14+
- Completely rewritten blocklist system. Blocklisting now runs off its own mongoDB collection. This once again introduces backwards incompatible schema changes, so a manual migration is required. You may upgrade from both v4.0 and v4.1 using the `[p]migrate blocklist` command. This removes any need to perform the previous migration steps in v4.1.0, you may upgrade directly to this version. After running the command, blocklist functionality will return and legacy config based blocks will have been deleted. You should always back up your config before migration.
15+
- Migrated package manager from pipenv to Poetry. Dockerfile and GitHub Actions are also now set up to install using Poetry.
16+
17+
### Deprecated
18+
- Legacy blocklist properties are deprecated and no longer function. They now log a warning when used and provide no functionality. They have been replaced with methods in blocklist.py
19+
20+
### Added
21+
- Added `content_type` to attachments stored in the database.
22+
- Introduced multiplatform Docker workflow for GitHub Actions.
23+
- Improved join/leave message for multiple servers.
24+
- Added `log_expiration` config to auto-delete old logs on a schedule.
25+
26+
### Changed
27+
- Changing a thread's title or NSFW status immediately updates the status in the database.
28+
29+
### Removed
30+
- The logviewer plugin is no longer included locally.
31+
- Removed lottie sticker support and its dependencies due to unnecessary bulk.
32+
- Removed gyazo image embedding.
33+
34+
### Fixed
35+
- Persistent notes have been fixed after the previous discord.py update.
36+
- `is_image` now is true only if the image is actually an image.
37+
- Resolved deprecated pkg_resources warning in certain Python environment.
38+
- Improved genesis message retrieval checks where it would fail under certain circumstances.
39+
- Fixed failure on installing local plugins when plugin path contains whitespace.
40+
- Fixed sticker replies not being sent in a Modmail thread.
41+
- Fixed plugin update command where it would fail to remove the plugin when the plugin is invalid.
42+
- Fixed MissingRequiredArgument requiring additional param error on certain commands.
43+
- Fixed rate limit issue on raw reaction add/remove events.
44+
45+
### Internal
46+
- Add `update_title` and `update_nsfw` methods to `ApiClient` to update thread title and nsfw status in the database.
47+
- `thread.set_title` now requires `channel_id` to be passed as keyword arguments.
48+
- New `thread.set_nsfw_status` method to set nsfw status of a thread.
49+
- Updated discord.py to 2.3.2, with [speed] extra.
50+
1151
# v4.1.0
1252

1353
### Breaking
@@ -46,6 +86,9 @@ however, insignificant breaking changes do not guarantee a major version bump, s
4686
- Fixed blocked roles improperly saving in `blocked_users` config.
4787
- Fixed `?block` command improperly parsing reason as timestamp.
4888

89+
### Internal
90+
- `ConfigManager.get` no longer accepts two positional arguments: the `convert` argument is now keyword-only.
91+
4992
# v4.0.2
5093

5194
### Breaking

Dockerfile

+25-15
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,37 @@
11
FROM python:3.11-alpine as base
22

33
RUN apk update && apk add git \
4-
# cairosvg dependencies
5-
cairo-dev cairo cairo-tools \
64
# pillow dependencies
7-
jpeg-dev zlib-dev
5+
jpeg-dev zlib-dev && \
6+
pip install --upgrade pip && \
7+
adduser -D -h /home/modmail -g 'Modmail' modmail
88

9-
FROM base as python-deps
9+
WORKDIR /home/modmail
1010

11-
RUN apk add --virtual build-deps build-base gcc libffi-dev
12-
COPY requirements.txt /
13-
RUN pip install --prefix=/inst -U -r /requirements.txt
11+
FROM base as builder
12+
13+
ENV POETRY_NO_INTERACTION=1 \
14+
POETRY_VIRTUALENVS_IN_PROJECT=1 \
15+
POETRY_VIRTUALENVS_CREATE=1 \
16+
POETRY_CACHE_DIR=/tmp/poetry_cache
17+
18+
RUN apk add build-base libffi-dev && \
19+
pip install -U poetry
20+
21+
COPY --chown=modmail:modmail poetry.lock pyproject.toml /home/modmail/
22+
23+
RUN --mount=type=cache,target=$POETRY_CACHE_DIR poetry install --without dev --no-root
1424

1525
FROM base as runtime
1626

17-
ENV USING_DOCKER yes
18-
COPY --from=python-deps /inst /usr/local
27+
ENV VIRTUAL_ENV=/home/modmail/.venv \
28+
PATH="/home/modmail/.venv/bin:$PATH" \
29+
USING_DOCKER=yes
1930

20-
COPY . /modmail
21-
WORKDIR /modmail
31+
COPY --from=builder --chown=modmail:modmail ${VIRTUAL_ENV} ${VIRTUAL_ENV}
2232

23-
CMD ["python", "bot.py"]
33+
COPY --chown=modmail:modmail . .
34+
35+
USER modmail
2436

25-
RUN adduser --disabled-password --gecos '' app && \
26-
chown -R app /modmail
27-
USER app
37+
CMD ["python", "bot.py"]

Pipfile

-35
This file was deleted.

0 commit comments

Comments
 (0)