What should the new database schemas be called?

[MoralCode]

In an upcoming version of CollectOSS we plan to include a database migration that updates the name of the database schemas.

I wanted to start a community discussion over the new naming we should use.

This change will likely require all downstream that are relying on our core database tables (including 8Knot and probably the grafana monitoring stack) to update some or all of their queries where the schema is being hard-coded.

Many such queries within CollectOSS itself will likely change.

Our current affected schemas are called:

• augur_data (for collected data used as part of analysis)
• augur_operations (for metadata relating to the collection process)

[MoralCode]

Option 1: basic

• collectoss_data / collectoss_operations

Option 2: generic

• collection_data / collection_operations

Option 3: snazzy

• forge_data / collection_operations

Misc other ideas:

• collected_data
• health_data
• core_data

[cdolfi]

I dont have any strong opinions, good with any of the 3!

[shlokgilda]

I prefer 2.

[MoralCode]

collection_data / collection_operations it is

[MoralCode]

On talking with Josh Berkus, his recommendation is not to use a prefix since CollectOSS is so heavy on databases (and IO) at large scales, so it doesnt really make much sense for it to be sharing a database with other applications anyway.

Also given the plans to attempt to codify a separate schema thats explicitly an external interface, i propose this naming scheme:

augur_data -> internal
augur_operations -> operations
new schema for views (materialized or not) that downstream projects use -> data

[cdolfi]

This feels very overcomplicated but maybe im missing something. @jberkus what are your thoughts on this?

[MoralCode]

internally, it is more complicated than a simple rename, but externally (as far as the migration guide/what downstreams will need to do) I think this is the path that allows us to have more control over breaking changes (and being able to avoid or schedule them without delaying the CollectOSS roadmap nearly as much), which is going to be better/more stable for downstreams long term.

Technical questions for josh:

• will using a view that is essentially select * from table automatically pass through any indexes?
• will we be able to logical database replication from this "data" (views) schema?

[MoralCode]

seems like AI suggests that the views will just pass through their indexes because the view isnt its own table, its basically just a proxy to internal tables.

The logical replication will probably need to replicate the underlying tables though so that we can recreate similar views in the new tooling

[jberkus]

Here's what I would do:

• augur_data --> data
• augur_operations --> operations
• spdx --> spdx

The new schema could be called "views" or "interface" which helps clearly indicate what it is.

[MoralCode]

would calling the new views schema "stable" be a good option? I want a naming system that's clear, but also strongly pushes downstreams to use the views if they want a stable set of tables to query

Edit: this was acked by cali on slack

[MoralCode]

ok final final plan

augur_data --> data
augur_operations --> operations
spdx --> spdx

A new schema called stable will also be created to store all the materialized views and will become the home for stable tables/views that will be unit tested for their stability and that downstreams should be strongly encouraged to use (if they want to avoid breaking schema changes that will be coming in future versions

[cdolfi]

lgtm!