Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[change] Add support for influxdb 2.0 #274

Open
nemesifier opened this issue Mar 10, 2021 · 10 comments · May be fixed by #584
Open

[change] Add support for influxdb 2.0 #274

nemesifier opened this issue Mar 10, 2021 · 10 comments · May be fixed by #584
Labels
enhancement New feature or request help wanted Extra attention is needed influxdb

Comments

@nemesifier
Copy link
Member

nemesifier commented Mar 10, 2021

I had to pin the inflxudb docker image used for testing to version 1.8.4 because the new version 2.0 breaks the tests:
#273

However, this is not great: we want to test with the latest influxdb version and therefore it would be great to upgrade our code so that the test environment is compatible with the last influxdb version. It should be just a matter of setting the authentication credentials correctly.

I think we have to write a new timeseries backend for Influxdb > 2.0 and keep the current backend for Influxdb <= 1.8.

@nemesifier nemesifier added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed influxdb labels Mar 10, 2021
@nemesifier nemesifier added this to the 0.2 release milestone Apr 6, 2021
@devkapilbansal devkapilbansal self-assigned this Apr 8, 2021
@devkapilbansal
Copy link
Member

Hi @nemesisdesign the current library used here supports InfluxDB 1.x only.
Another library (influxdb-client) only support InfluxDB 1.8+.

Would it be good to drop support for InfluxDB 1.7 or earlier ? 🤔

@nemesifier
Copy link
Member Author

Hi @nemesisdesign the current library used here supports InfluxDB 1.x only.
Another library (influxdb-client) only support InfluxDB 1.8+.

Would it be good to drop support for InfluxDB 1.7 or earlier ?

@devkapilbansal I guess we have no other viable option, influxdb 1.8 works well so if we can support from 1.8 onwards that's good enough, what worries me is the change of library, do you know if the syntax is similar?

@devkapilbansal
Copy link
Member

Yes, as both libraries are maintained by InfluxData, syntax is quite similar

@nemesifier nemesifier added Hacktoberfest Easy issues for attracting Hacktoberfest participants. and removed good first issue Good for newcomers labels Sep 21, 2021
@devkapilbansal devkapilbansal removed their assignment Oct 16, 2021
@nemesifier nemesifier changed the title [CI] Migrate CI build to influxdb 2.0 [change] Migrate CI build to influxdb 2.0 Feb 14, 2022
@nemesifier nemesifier changed the title [change] Migrate CI build to influxdb 2.0 [change] Add support for influxdb 2.0 Feb 14, 2022
@AbhigyaShridhar
Copy link

@nemesisdesign I cannot reproduce this issue. I changed the version of influxdb image to 2.0-alpine in both docker-compose.yml and workflows/ci.yml. All the tests run perfectly.

@devkapilbansal
Copy link
Member

@AbhigyaShridhar how did you checked it? I just checked and issue is reproducible:
https://github.com/devkapilbansal/openwisp-monitoring/runs/5649166266?check_suite_focus=true#step:10:17

Had it not been reproducible, the version have been updated a long time ago

@AbhigyaShridhar
Copy link

@devkapilbansal I ran ./run-qa-checks and ./runtests.py on my local system.

@nemesifier
Copy link
Member Author

@nemesisdesign I cannot reproduce this issue. I changed the version of influxdb image to 2.0-alpine in both docker-compose.yml and workflows/ci.yml. All the tests run perfectly.

I wish it was so simple @AbhigyaShridhar, last time we checked, in order to use Influxdb 2.0, we would have to switch to a different python library, please read the first lines of https://github.com/influxdata/influxdb-python#influxdb-python for more info.

@AbhigyaShridhar
Copy link

@nemesisdesign I cannot reproduce this issue. I changed the version of influxdb image to 2.0-alpine in both docker-compose.yml and workflows/ci.yml. All the tests run perfectly.

I wish it was so simple @AbhigyaShridhar, last time we checked, in order to use Influxdb 2.0, we would have to switch to a different python library, please read the first lines of https://github.com/influxdata/influxdb-python#influxdb-python for more info.

Oh, ok I'll look into that

AbhigyaShridhar added a commit to AbhigyaShridhar/openwisp-monitoring that referenced this issue Apr 4, 2022
To add support for influxdb2.0 and above, a different library,
influxdb_client instead of influxdb has to be used.

Added the code to counter syntax differences between both the libraries
Unfortunately, the _init__ file in db/backends module is buggy after the changes
Working to fix those bugs for testing in the next commit

Adding this commit here so things don't get cluttered
_
AbhigyaShridhar added a commit to AbhigyaShridhar/openwisp-monitoring that referenced this issue Apr 12, 2022
I realized that this is a big issue and will take some time to be resolved completely so this is the initial code for it

Changes made:

 - added another backend in /home/abhigya/Documents/OpenSource/openwisp/monitoring/openwisp-monitoring/openwisp_monitoring/db/backends
 - made changes to the __init__ file of backends to now deal with two possible backends and their different requirements
 - added initial code for the influxb2.x backend client
 - wrote initial code for a method to 'standardize' query results. This will have to be implemented for all the backends in future
AbhigyaShridhar added a commit to AbhigyaShridhar/openwisp-monitoring that referenced this issue Apr 14, 2022
@nemesifier
Copy link
Member Author

One announcement which is surely going to impact this work:
https://www.influxdata.com/blog/influxdb-engine/

@Kevinjil
Copy link

Kevinjil commented Oct 16, 2023

In case anyone else wants to try it, there is an ugly way to get it up and running with InfluxDB 2 and the 1.x compatibility API. I used an NGINX reverse-proxy that filters out the incompatible API requests coming from OpenWISP monitoring. Of course, this means that you need to set up the retention policy mapping to buckets yourself.

The following NGINX configuration filters requests and inserts an InfluxDB v2 API token:

server {
    listen       8086;
    listen  [::]:8086;
    server_name  influxdb;

    location / {
        proxy_pass https://0.0.0.0;
        proxy_ssl_verify on;
        proxy_ssl_name influxdb.example.com;
        proxy_ssl_server_name on;
        proxy_set_header influxdb.example.com;
        proxy_set_header Authorization "Token changeme";
        client_body_buffer_size 128K;

        if ($arg_q ~ "^CREATE\+DATABASE") {
                return 200 '{}';
        }
        if ($arg_q ~ "^CREATE\+RETENTION") {
                return 200 '{}';
        }
        if ($arg_q ~ "^ALTER\+RETENTION") {
                return 200 '{}';
        }
        if ($arg_q ~ "^SHOW\+RETENTION") {
                return 200 '{"results":[{"statement_id":0,"series":[{"columns":["name","duration","shardGroupDuration","replicaN","default"],"values":[["autogen", "0s", "0s", 1, true], ["short", "0s", "0s", 1, false]]}]}]}';
        }
    }
}

@nemesifier nemesifier removed the Hacktoberfest Easy issues for attracting Hacktoberfest participants. label Apr 30, 2024
praptisharma28 added a commit to praptisharma28/openwisp-monitoring that referenced this issue May 31, 2024
@praptisharma28 praptisharma28 linked a pull request May 31, 2024 that will close this issue
3 tasks
praptisharma28 added a commit that referenced this issue May 31, 2024
praptisharma28 added a commit that referenced this issue May 31, 2024
praptisharma28 added a commit that referenced this issue May 31, 2024
praptisharma28 added a commit that referenced this issue May 31, 2024
praptisharma28 added a commit that referenced this issue Jun 1, 2024
praptisharma28 added a commit that referenced this issue Jun 1, 2024
praptisharma28 added a commit that referenced this issue Jun 1, 2024
praptisharma28 added a commit that referenced this issue Jun 1, 2024
Fixes 274
praptisharma28 added a commit that referenced this issue Jun 1, 2024
praptisharma28 added a commit that referenced this issue Jun 1, 2024
Fixes 274
praptisharma28 added a commit that referenced this issue Jun 1, 2024
praptisharma28 added a commit that referenced this issue Jun 1, 2024
praptisharma28 added a commit that referenced this issue Jun 2, 2024
praptisharma28 added a commit that referenced this issue Jun 3, 2024
praptisharma28 added a commit that referenced this issue Jun 8, 2024
praptisharma28 added a commit that referenced this issue Jun 25, 2024
praptisharma28 added a commit that referenced this issue Jun 29, 2024
praptisharma28 added a commit that referenced this issue Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed influxdb
Development

Successfully merging a pull request may close this issue.

4 participants