"Evaluating Dagster Open Source for Enterprise-Scale Near Real-Time Workloads"

[BKWInsights]

Hi everyone,

I’m currently evaluating whether we can work with a Dagster Open Source solution.

The commercial version of Dagster seems quite expensive. We are planning a near real-time Data Vault solution with around 600 source tables, which could quickly grow to 3000 Vault objects. Loading all of these every 5 minutes seems financially unfeasible with Dagster+.

In this GitHub discussion (#25313), the limitations and capabilities of the Open Source version were also brought up. However, even there, there doesn’t seem to be a clear overview of what the Open Source version can actually do.

This gives me the impression that Dagster Open Source might primarily serve as a “gateway” to their commercial SaaS solution. If that’s the case, it could mean the Open Source version is either heavily limited or will only see slow development in the future.

Has anyone here had experience working with Dagster’s Open Source version? How well does it perform, and is it realistic to maintain an enterprise-grade environment with it?

Thanks in advance for sharing your insights!