"The" or "Their" data platform

[lfrancke]

I stumbled across the use of "the data platform" in this part of the page https://www.datamesh-architecture.com/#ingesting and am wondering if "the" or "their" makes more sense.

I don't think it becomes very clear (not in most other literature either) what "the data platform" should be. It can be many things and it'd be good to add a clarification to it.

What should the data platform team produce and own?
There are multiple options and this is a non-complete list of things it can be interpreted:

• Every domain team can have their own platform (e.g. instance of a database) helped by a data platform team (e.g. a team that provides "vetted" helm charts or whatever), operation is handled by the domain team
• Similar to above but the "ops" part is handled by a central team
• The data platform teams provisions, scales and operates technologies (e.g. databases) and has an easy self-serve way for domain teams to provision abstractions (e.g. tables in this case, buckets or topics in others) on top of these centrally managed technologies

In the Cloud/SaaS world this is a bit simpler but the general question still applies, someone needs to provide the SLA.
I believe all of these alone and in combination are valid implementations so all I'm looking for in this discussion is more opinions and whether you think it makes sense to clarify this a bit more earlier in the page.

Thank you!

[jochenchrist]

Hi lrancke,

Thanks for your discussion.
You are correct, the data platform is not yet introduced in the article at this point. Maybe we should put a link to the platform section.

I think, usually, it is your 3rd option:

• The data platform teams provisions, scales and operates technologies (e.g. databases) and has an easy self-serve way for domain teams to provision abstractions (e.g. tables in this case, buckets or topics in others) on top of these centrally managed technologies

The data platform team manages data platform (data warehouse, data lake) and ensures domain teams can access it and add their analytical data and publish data product. The platform must also provide capabilities to discover other data products (e.g. through a data catalog, which might be a separate software).

From an ops perspective, the platform teams ensures that the platform is available and has enough storage, compute, ... and does not run out of costs. E.g. by prometheus, etc. The domain teams are responsible for their data products and observe data quality by regularly running test SQL statements. Tools are monte carlo, Great Expectations, (again provided by the platform teams, but statements are metrics are defined by domain teams)