Merge branch 'develop' into taxii

Author: laciKE

.github/workflows/scripts/setup-full.sh

@@ -15,10 +15,6 @@ sudo chown -R elasticsearch:elasticsearch /etc/default/elasticsearch
 sudo systemctl start elasticsearch
 
 sudo apt update
-if [ $python_version == '3.8' ]; then
-	# for pymssql there are no wheels for 3.8 https://github.com/certtools/intelmq/issues/2539
-	DEBIAN_FRONTEND="noninteractive" sudo -E apt install -y build-essential freetds-dev libssl-dev libkrb5-dev
-fi
 # for psql (used below)
 DEBIAN_FRONTEND="noninteractive" sudo -E apt install -y postgresql-client
 

.github/workflows/unittests.yml

@@ -18,7 +18,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13']
+        python-version: ['3.9', '3.10', '3.11', '3.12', '3.13']
         type: ['full', 'basic']
 
     services:

CHANGELOG.md

@@ -16,6 +16,7 @@ Please refer to the [NEWS](NEWS.md) for a list of changes which have an affect o
 ### Configuration
 
 ### Core
+- Drop support for Python 3.8 (fixes #2616, PR#2617 by Sebastian Wagner).
 
 ### Development
 
@@ -32,6 +33,8 @@ Please refer to the [NEWS](NEWS.md) for a list of changes which have an affect o
 #### Outputs
 
 ### Documentation
+- Fix and refresh links to mailing lists (PR#2609 by Kamil Mańkowski)
+- `Aggregate Bot`: Add illustration graphics (PR#2612 by Sebastian Wagner).
 
 ### Packaging
 - Replace `/opt/intelmq` example paths in bots with variable `VAR_STATE_PATH` for correct paths in LSB-path setups like with packages (PR#2587 by Sebastian Wagner).
@@ -41,6 +44,8 @@ Please refer to the [NEWS](NEWS.md) for a list of changes which have an affect o
 - `.github/workflows/codespell.yml`, `debian-package.yml`, `regexploit.yml`: Upgrade to `ubuntu-latest` runners (PR#2602 by Sebastian Wagner).
 
 ### Tools
+- `intelmq.bin.intelmq_psql_initdb`: Use `JSONB` type by default, Postgres supports it since version 9 (PR#2597 by Sebastian Wagner).
+- `intelmq.bin.rewrite_config_files`: Removed obsolete JSON configuration file rewriter (PR#2613 by Sebastian Wagner).
 
 ### Contrib
 

MANIFEST.in

@@ -10,7 +10,7 @@ graft intelmq/tests
 include COPYRIGHT
 include LICENSE
 include CHANGELOG.md
-recursive-exclude intelmq/bin intelmq_gen_feeds_docs.py intelmq_gen_harm_docs.py rewrite_config_files.py
+recursive-exclude intelmq/bin intelmq_gen_feeds_docs.py intelmq_gen_harm_docs.py
 exclude .*
 global-exclude *~
 global-exclude *.py[co]

NEWS.md

@@ -14,6 +14,7 @@ Please refer to the change log for a full list of changes.
 --------------------------------
 
 ### Requirements
+Python `>=3.9` is now required, which is available on all platforms supported by IntelMQ.
 
 ### Tools
 
@@ -24,6 +25,11 @@ Please refer to the change log for a full list of changes.
 ### Libraries
 
 ### Postgres databases
+To switch to the more efficient data type `jsonb` instead of `json`, use the following SQL statement. Data is preserved. JSONB also has more query and data manipulation features than plain JSON.
+```sql
+ALTER TABLE events
+   ALTER COLUMN "extra" SET DATA TYPE jsonb;
+```
 
 
 3.4.0 Feature release (2025-03-14)

debian/control

@@ -25,7 +25,7 @@ Build-Depends: debhelper (>= 4.1.16),
                python3-pytest-cov,
                findutils,
                sed
-X-Python3-Version: >= 3.7
+X-Python3-Version: >= 3.9
 Standards-Version: 3.9.6
 Homepage: https://github.com/certtools/intelmq/
 

debian/rules

@@ -33,7 +33,6 @@ override_dh_auto_build:
 	# $(MAKE) docs
 
 override_dh_auto_install: $(BOTDOCS)
-	rm intelmq/bin/rewrite_config_files.py
 	sed -i -e '/#!\/usr\/bin\//d' intelmq/bin/*.py
 	sed -i -f debian/sedfile intelmq/etc/* setup.py contrib/logrotate/intelmq contrib/logcheck/logcheck.logfiles
 	dh_auto_install

docs/admin/intro.md

@@ -15,7 +15,7 @@ system to preserve processed events.
 
 ## Base Requirements
 
-The following instructions assume the following requirements. Python versions >= 3.7 are supported.
+The following instructions assume the following requirements. Python versions >= 3.9 are supported.
 
 Supported and recommended operating systems are:
 

docs/dev/intro.md

@@ -34,7 +34,7 @@ Similarly, if code does not get accepted upstream by the main developers, it is
 ## Mailing list
 
 There is a separate mailing list for developers to discuss development topics:
-The [IntelMQ-DevArchive](https://lists.cert.at/pipermail/intelmq-dev/) is public as well.
+The [IntelMQ-DevArchive](https://lists.cert.at/mailman3/hyperkitty/list/intelmq-dev@lists.cert.at/) is public as well.
 
 ## GitHub
 

docs/help.md

@@ -26,7 +26,7 @@ To participate on GitHub, you first need to create an account on the platform.
 ## Mailing list
 
 The most traditional way is to ask your question, make a proposal or discuss a topic on the
-mailing [IntelMQ Users mailing list](https://lists.cert.t/cgi-bin/mailman/listinfo/intelmq-users). You need to subscribe to the mailing list before posting, but the archive is publicly available: [IntelMQ Users Archive](https://lists.cert.at/pipermail/intelmq-users/).
+mailing [IntelMQ Users mailing list](https://lists.cert.at/mailman3/hyperkitty/list/intelmq-users@lists.cert.at/). You need to subscribe to the mailing list before posting, but the archive is publicly available: [IntelMQ Users Archive](https://lists.cert.at/mailman3/hyperkitty/list/intelmq-users@lists.cert.at/latest).
 
 ## Assistance