Deployable Workspace Libraries

[samshuster]

Goal

For a library like dwh-databricks-common, it would be easiest for users if it was a library available on workspace. This way users could easily install it to any cluster they want.

This is very different then a productionalized interactive cluster or job that needs a predefined set of libraries which is functionality that already exists. The difference here is that it would be used on adhoc clusters that should not be managed programmatically and we want it easy for users to be able to find common code and install the most up to date version.

Requirements

There is functionality for maven-plugin to deploy a library as a standalone library on databricks workspace.

default clusters that it is installed to would be great.

It doesn’t need to restart clusters, I think having it be manual is fine.