A Single place to Discover, Collaborate, and Get your data right

Darkseal

Unlock the value of data assets with an end-to-end metadata platform that includes data discovery, governance, data quality, observability, and people collaboration.

Quickstart

Start From Source

# Clone source
git clone [email protected]:GuinsooLab/darkseal.git

# Package 
mvn clean package -DskipTests

# Prepare database and indexes
./bootstrap/bootstrap_storage.sh create

# Start Server
java -cp catalog-rest-service org.openmetadata.catalog.CatalogApplication server ./conf/openmetadata.yaml

# Start UI
cd openmetadata-ui/src/ui
yarn run start

Start From Built Package

# Clone source
git clone [email protected]:GuinsooLab/darkseal.git

# Package 
mvn clean package -DskipTests

# Change dist dir & un-tar files
cd openmetadata-dist/target
tar zxvf openmetadata-xxx.tar.gz
cd openmetadata-xxx

# Prepare database and indexes
./bootstrap/bootstrap_storage.sh create

# Start Server
./bin/openmetadata.sh start

For more information, please referer to here.

Main Features

• Data Trust
• Data Documentation
• Data Governance
• Discovery & Collaboration
• Teams & Users with Roles and Policies
• Data Glossaries
• Data Insights
• Data Lineage
• Security & Compliance
• Alerts & Notifications

Core Components

Darkseal includes the following:

• Metadata schemas - Defines core abstractions and vocabulary for metadata with schemas for Types, Entities, Relationships between entities. This is the foundation of the Open Metadata Standard.
• Metadata store - Stores metadata graph that connects data assets, user, and tool generated metadata.
• Metadata APIs - For producing and consuming metadata built on schemas for User Interfaces and Integration of tools, systems, and services.
• Ingestion framework - A pluggable framework for integrating tools and ingesting metadata to the metadata store. Ingestion framework already supports well know data warehouses - Google BigQuery, Snowflake, Amazon Redshift, and Apache Hive, and databases - MySQL, Postgres, Oracle, MSSQL, and Guinsoo.
• Metadata User Interface - One single place for users to discover, and collaborate on all data.

Documentation & Supports

• Introduction 😋
• Overview
  • Tutorial
  • Features
  • Releases
  • Roadmap
• Deployment
  • Docker Deployment
  • Bare Metal Deployment
• Service Integrations
  • Database
  • Dashboard
  • Messaging
  • Pipeline
  • ML Model
  • Metadata
  • Ingestion
• Developers
  • Architecture
  • Contribute
  • Web Hooks
• Appendix
  • FAQs

Contributors

We ❤️ all contributions, big and small! Check out our CONTRIBUTING guide to get started and let us know how we can help.

Don't want to miss anything? Give the project a ⭐ 🚀

License

Darkseal is released under Apache License, Version 2.0