Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #15696: Added Powerbi filter query #19912

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

akashverma0786
Copy link
Contributor

@akashverma0786 akashverma0786 commented Feb 21, 2025

Describe your changes:

Fixes #15696

Short blurb explaining:

  • What changes did you make?
    Added method to add filter query in powerbi api call to filter out projects on call itself
    A method that changes ProjectFilterPattern(if provided) into odata condition
  • Why did you make them?
    To allow pushing down the filtering to the Power BI level, using a different API with an odata filter.
  • How did you test your changes?
    Test for method

Project Filter pattern:-
Screenshot 2025-02-24 at 18 05 26

Before odata filter:-
Screenshot 2025-02-24 at 17 34 39

After filter:-
Screenshot 2025-02-24 at 17 37 37

I worked on ... because ...

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

Copy link
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Copy link
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Copy link
Contributor

github-actions bot commented Feb 24, 2025

🛡️ TRIVY SCAN RESULT 🛡️

Target: openmetadata-ingestion-base-slim:trivy (debian 11.11)

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Java

Vulnerabilities (46)

Package Vulnerability ID Severity Installed Version Fixed Version
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.11.4 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.11.4 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.13.0 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.13.0 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4
com.google.code.gson:gson CVE-2022-25647 🚨 HIGH 2.2.4 2.8.9
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 2.5.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 2.5.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 2.5.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.3.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.3.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.3.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.7.1 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.7.1 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.7.1 3.25.5, 4.27.5, 4.28.2
com.nimbusds:nimbus-jose-jwt CVE-2023-52428 🚨 HIGH 9.8.1 9.37.2
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.11.0 2.14.0
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.8.0 2.14.0
dnsjava:dnsjava CVE-2024-25638 🚨 HIGH 2.1.7 3.6.0
io.airlift:aircompressor CVE-2024-36114 🚨 HIGH 0.21 0.27
net.minidev:json-smart CVE-2021-31684 🚨 HIGH 1.3.2 1.3.3, 2.4.4
net.minidev:json-smart CVE-2023-1370 🚨 HIGH 1.3.2 2.4.9
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.11.0 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.11.0 1.11.3
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.7.7 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.7.7 1.11.3
org.apache.derby:derby CVE-2022-46337 🔥 CRITICAL 10.14.2.0 10.14.3, 10.15.2.1, 10.16.1.2, 10.17.1.0
org.apache.hadoop:hadoop-common CVE-2022-25168 🔥 CRITICAL 3.3.2 2.10.2, 3.2.4, 3.3.3
org.apache.ivy:ivy CVE-2022-46751 🚨 HIGH 2.5.1 2.5.2
org.apache.mesos:mesos CVE-2018-1330 🚨 HIGH 1.4.3 1.6.0
org.apache.spark:spark-hive-thriftserver_2.12 CVE-2024-23945 🚨 HIGH 3.3.4 3.4.2
org.apache.thrift:libthrift CVE-2019-0205 🚨 HIGH 0.12.0 0.13.0
org.apache.thrift:libthrift CVE-2020-13949 🚨 HIGH 0.12.0 0.14.0
org.apache.zookeeper:zookeeper CVE-2023-44981 🔥 CRITICAL 3.6.2 3.7.2, 3.8.3, 3.9.1
org.xerial.snappy:snappy-java CVE-2023-34455 🚨 HIGH 1.1.8.4 1.1.10.1
org.xerial.snappy:snappy-java CVE-2023-43642 🚨 HIGH 1.1.8.4 1.1.10.4
org.yaml:snakeyaml CVE-2022-1471 🚨 HIGH 1.31 2.0

🛡️ TRIVY SCAN RESULT 🛡️

Target: Node.js

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Python

Vulnerabilities (3)

Package Vulnerability ID Severity Installed Version Fixed Version
Werkzeug CVE-2024-34069 🚨 HIGH 2.2.3 3.0.3
apache-airflow CVE-2024-39877 🚨 HIGH 2.9.1 2.9.3
apache-airflow CVE-2024-45034 🚨 HIGH 2.9.1 2.10.1

🛡️ TRIVY SCAN RESULT 🛡️

Target: /etc/ssl/private/ssl-cert-snakeoil.key

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/extended_sample_data.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/lineage.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data.json

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_data.yaml

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage.json

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /ingestion/pipelines/sample_usage.yaml

No Vulnerabilities Found

Copy link
Contributor

github-actions bot commented Feb 24, 2025

🛡️ TRIVY SCAN RESULT 🛡️

Target: openmetadata-ingestion:trivy (debian 12.5)

Vulnerabilities (36)

Package Vulnerability ID Severity Installed Version Fixed Version
libexpat1 CVE-2024-45491 🔥 CRITICAL 2.5.0-1 2.5.0-1+deb12u1
libexpat1 CVE-2024-45492 🔥 CRITICAL 2.5.0-1 2.5.0-1+deb12u1
libexpat1 CVE-2024-45490 🚨 HIGH 2.5.0-1 2.5.0-1+deb12u1
libpython3.11-minimal CVE-2023-24329 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
libpython3.11-minimal CVE-2023-41105 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
libpython3.11-minimal CVE-2023-6597 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
libpython3.11-minimal CVE-2024-6232 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u4
libpython3.11-minimal CVE-2024-7592 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
libpython3.11-minimal CVE-2024-9287 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
libpython3.11-stdlib CVE-2023-24329 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
libpython3.11-stdlib CVE-2023-41105 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
libpython3.11-stdlib CVE-2023-6597 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
libpython3.11-stdlib CVE-2024-6232 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u4
libpython3.11-stdlib CVE-2024-7592 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
libpython3.11-stdlib CVE-2024-9287 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
libsqlite3-0 CVE-2023-7104 🚨 HIGH 3.40.1-2 3.40.1-2+deb12u1
libsystemd0 CVE-2023-50387 🚨 HIGH 252.22-1~deb12u1 252.23-1~deb12u1
libsystemd0 CVE-2023-50868 🚨 HIGH 252.22-1~deb12u1 252.23-1~deb12u1
libudev1 CVE-2023-50387 🚨 HIGH 252.22-1~deb12u1 252.23-1~deb12u1
libudev1 CVE-2023-50868 🚨 HIGH 252.22-1~deb12u1 252.23-1~deb12u1
openssh-client CVE-2024-6387 🚨 HIGH 1:9.2p1-2+deb12u2 1:9.2p1-2+deb12u3
python3.11 CVE-2023-24329 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
python3.11 CVE-2023-41105 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
python3.11 CVE-2023-6597 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
python3.11 CVE-2024-6232 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u4
python3.11 CVE-2024-7592 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
python3.11 CVE-2024-9287 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
python3.11-minimal CVE-2023-24329 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
python3.11-minimal CVE-2023-41105 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
python3.11-minimal CVE-2023-6597 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u2
python3.11-minimal CVE-2024-6232 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u4
python3.11-minimal CVE-2024-7592 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
python3.11-minimal CVE-2024-9287 🚨 HIGH 3.11.2-6 3.11.2-6+deb12u5
rsync CVE-2024-12084 🔥 CRITICAL 3.2.7-1 3.2.7-1+deb12u1
rsync CVE-2024-12085 🚨 HIGH 3.2.7-1 3.2.7-1+deb12u1
sqlite3 CVE-2023-7104 🚨 HIGH 3.40.1-2 3.40.1-2+deb12u1

🛡️ TRIVY SCAN RESULT 🛡️

Target: Java

Vulnerabilities (46)

Package Vulnerability ID Severity Installed Version Fixed Version
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.11.4 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.11.4 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.11.4 2.12.7.1, 2.13.4
com.fasterxml.jackson.core:jackson-databind CVE-2020-36518 🚨 HIGH 2.13.0 2.13.2.1, 2.12.6.1
com.fasterxml.jackson.core:jackson-databind CVE-2021-46877 🚨 HIGH 2.13.0 2.12.6, 2.13.1
com.fasterxml.jackson.core:jackson-databind CVE-2022-42003 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4.2
com.fasterxml.jackson.core:jackson-databind CVE-2022-42004 🚨 HIGH 2.13.0 2.12.7.1, 2.13.4
com.google.code.gson:gson CVE-2022-25647 🚨 HIGH 2.2.4 2.8.9
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 2.5.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 2.5.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 2.5.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 2.5.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.3.0 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.3.0 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.3.0 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.3.0 3.25.5, 4.27.5, 4.28.2
com.google.protobuf:protobuf-java CVE-2021-22569 🚨 HIGH 3.7.1 3.16.1, 3.18.2, 3.19.2
com.google.protobuf:protobuf-java CVE-2021-22570 🚨 HIGH 3.7.1 3.15.0
com.google.protobuf:protobuf-java CVE-2022-3509 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2022-3510 🚨 HIGH 3.7.1 3.16.3, 3.19.6, 3.20.3, 3.21.7
com.google.protobuf:protobuf-java CVE-2024-7254 🚨 HIGH 3.7.1 3.25.5, 4.27.5, 4.28.2
com.nimbusds:nimbus-jose-jwt CVE-2023-52428 🚨 HIGH 9.8.1 9.37.2
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.11.0 2.14.0
commons-io:commons-io CVE-2024-47554 🚨 HIGH 2.8.0 2.14.0
dnsjava:dnsjava CVE-2024-25638 🚨 HIGH 2.1.7 3.6.0
io.airlift:aircompressor CVE-2024-36114 🚨 HIGH 0.21 0.27
net.minidev:json-smart CVE-2021-31684 🚨 HIGH 1.3.2 1.3.3, 2.4.4
net.minidev:json-smart CVE-2023-1370 🚨 HIGH 1.3.2 2.4.9
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.11.0 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.11.0 1.11.3
org.apache.avro:avro CVE-2024-47561 🔥 CRITICAL 1.7.7 1.11.4
org.apache.avro:avro CVE-2023-39410 🚨 HIGH 1.7.7 1.11.3
org.apache.derby:derby CVE-2022-46337 🔥 CRITICAL 10.14.2.0 10.14.3, 10.15.2.1, 10.16.1.2, 10.17.1.0
org.apache.hadoop:hadoop-common CVE-2022-25168 🔥 CRITICAL 3.3.2 2.10.2, 3.2.4, 3.3.3
org.apache.ivy:ivy CVE-2022-46751 🚨 HIGH 2.5.1 2.5.2
org.apache.mesos:mesos CVE-2018-1330 🚨 HIGH 1.4.3 1.6.0
org.apache.spark:spark-hive-thriftserver_2.12 CVE-2024-23945 🚨 HIGH 3.3.4 3.4.2
org.apache.thrift:libthrift CVE-2019-0205 🚨 HIGH 0.12.0 0.13.0
org.apache.thrift:libthrift CVE-2020-13949 🚨 HIGH 0.12.0 0.14.0
org.apache.zookeeper:zookeeper CVE-2023-44981 🔥 CRITICAL 3.6.2 3.7.2, 3.8.3, 3.9.1
org.xerial.snappy:snappy-java CVE-2023-34455 🚨 HIGH 1.1.8.4 1.1.10.1
org.xerial.snappy:snappy-java CVE-2023-43642 🚨 HIGH 1.1.8.4 1.1.10.4
org.yaml:snakeyaml CVE-2022-1471 🚨 HIGH 1.31 2.0

🛡️ TRIVY SCAN RESULT 🛡️

Target: Node.js

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: Python

Vulnerabilities (10)

Package Vulnerability ID Severity Installed Version Fixed Version
Authlib CVE-2024-37568 🚨 HIGH 1.3.0 1.3.1
Werkzeug CVE-2024-34069 🚨 HIGH 2.2.3 3.0.3
apache-airflow CVE-2024-39877 🚨 HIGH 2.9.1 2.9.3
apache-airflow CVE-2024-45034 🚨 HIGH 2.9.1 2.10.1
mysql-connector-python CVE-2024-21272 🚨 HIGH 8.4.0 9.1.0
setuptools CVE-2024-6345 🚨 HIGH 65.5.1 70.0.0
setuptools CVE-2024-6345 🚨 HIGH 66.1.1 70.0.0
snowflake-connector-python CVE-2025-24793 🚨 HIGH 3.10.0 3.13.1
tornado CVE-2024-52804 🚨 HIGH 6.4 6.4.2
virtualenv CVE-2024-53899 🚨 HIGH 20.26.1 20.26.6

🛡️ TRIVY SCAN RESULT 🛡️

Target: /home/airflow/openmetadata-airflow-apis/openmetadata_managed_apis.egg-info/PKG-INFO

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /etc/ssl/private/ssl-cert-snakeoil.key

No Vulnerabilities Found

🛡️ TRIVY SCAN RESULT 🛡️

Target: /home/airflow/.local/lib/python3.10/site-packages/openmetadata_managed_apis-1.6.0.0.dev0.dist-info/METADATA

No Vulnerabilities Found

Copy link
Contributor

The Python checkstyle failed.

Please run make py_format and py_format_check in the root of your repository and commit the changes to this PR.
You can also use pre-commit to automate the Python code formatting.

You can install the pre-commit hooks with make install_test precommit_install.

@akashverma0786 akashverma0786 changed the title Powerbi filter query Fixes #15696: Added Powerbi filter query Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
safe to test Add this label to run secure Github workflows on PRs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

filter Power BI workspaces before metadata ingestion
2 participants