Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

filter Power BI workspaces before metadata ingestion #15696

Open
Limp-monkey opened this issue Mar 26, 2024 · 1 comment · May be fixed by #19912
Open

filter Power BI workspaces before metadata ingestion #15696

Limp-monkey opened this issue Mar 26, 2024 · 1 comment · May be fixed by #19912
Assignees
Labels
Ingestion P2 Medium priority

Comments

@Limp-monkey
Copy link

Feature
Allow push down of filtering in Power BI ingestion.

Describe the task

Currently, in the Power BI ingestion pipeline, any workspace (aka project in OM) filters are applied AFTER the metadata was ingested. This causes the issue, that if there are many workspaces in a Power BI tenant, then the API used (https://learn.microsoft.com/en-us/rest/api/power-bi/admin/groups-get-groups-as-admin) to ingest the metadata will hit the API limits very quickly, resulting in timeout and failures.

image
image

The feature request is to allow pushing down the filtering to the Power BI level, but using a different API with an odata filter or something similar - https://learn.microsoft.com/en-us/rest/api/power-bi/groups/get-groups#get-a-list-of-workspaces-using-a-filter-example

image

@OnkarVO7 OnkarVO7 self-assigned this Mar 26, 2024
@OnkarVO7 OnkarVO7 moved this to Ingestion - Bugs & Minor Features in Release 1.4.0 Mar 26, 2024
@pmbrull pmbrull removed this from Release 1.4.0 Apr 25, 2024
@harshach harshach assigned SumanMaharana and unassigned OnkarVO7 Aug 10, 2024
@harshach
Copy link
Collaborator

@OnkarVO7 lets work with @SumanMaharana to get this done in 1.6.0

@harshach harshach moved this to Integration in Release 1.7.0 Dec 14, 2024
@harshach harshach added the P2 Medium priority label Dec 14, 2024
@harshach harshach added this to the Feb 25 - Sprint 1 milestone Jan 21, 2025
@akashverma0786 akashverma0786 linked a pull request Feb 24, 2025 that will close this issue
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ingestion P2 Medium priority
Projects
Status: Integration
Development

Successfully merging a pull request may close this issue.

4 participants