Skip to content

Latest commit

 

History

History
464 lines (396 loc) · 42.5 KB

README.md

File metadata and controls

464 lines (396 loc) · 42.5 KB
Raw

Awesome Database Tools Awesome

Community driven list of database tools

Here we will collect information about awesome useful and awesome experimental tools that simplify working with databases for DBA, DevOps, Developers and mere mortals.

Feel free to add information about your own db-tools or your favorite third-party db-tools.

For updates on awesome-db-tools and thoughts/news about databases/tools/SQL follow me at @GraminMaksim

Contents

IDE

  • AnySQL Maestro - Premier multi-purpose admin tool for database management, control and development.
  • Aqua Data Studio - Aqua Data Studio is productivity software for Database Developers, DBAs, and Analysts.
  • Coginiti Pro - Modern IDE for analyst and analytics engineers with proweful script and grid functionality.
  • Database .net - Multiple database management tool with support for 20+ databases.
  • DataGrip - Cross-Platform IDE for Databases & SQL by JetBrains.
  • DataStation - Easily query, script, and visualize data from every database, file, and API.
  • DBeaver - Free universal database manager and SQL client.
  • dbForge Studio for MySQL - Universal IDE for MySQL and MariaDB database development, management, and administration.
  • dbForge Studio for Oracle - Powerful IDE for Oracle management, administration, and development.
  • dbForge Studio for PostgreSQL - GUI tool for managing and developing databases and objects.
  • dbForge Studio for SQL Server - Powerful integrated development environment for SQL Server development, management, administration, data analysis, and reporting.
  • DBHawk - Datasparc offers database security, database management, database governance and data analytics - all in one solution.
  • dbKoda - Modern (JavaScript/Electron framework), open source IDE for MongoDB. It has features to support development, administration and performance tuning on MongoDB databases.
  • IBExpert - Comprehensive GUI tool for Firebird and InterBase.
  • HeidiSQL - A lightweight client for managing MySQL, MSSQL and PostgreSQL, written in Delphi.
  • KeepTool - A professional suite of tools for Oracle Database developers, administrators and advanced application users.
  • MySQL Workbench - MySQL Workbench is a unified visual tool for database architects, developers, and DBAs.
  • Navicat - A database development tool that allows you to simultaneously connect to MySQL, MariaDB, SQL Server, Oracle, PostgreSQL, and SQLite databases from a single application.
  • Oracle SQL Developer - Oracle SQL Developer is a free, integrated development environment that simplifies the development and management of Oracle Database in both traditional and Cloud deployments.
  • pgAdmin - The most popular and feature rich Open Source administration and development platform for PostgreSQL, the most advanced Open Source database in the world.
  • pgAdmin3 - Long Term Support for pgAdmin3.
  • PL/SQL Developer - IDE that is specifically targeted at the development of stored program units for Oracle Databases.
  • PostgreSQL Maestro - Complete and powerful database management, admin and development tool for PostgreSQL.
  • Querybook - Querybook is a Pinterest open-source Big Data Querying UI, combining collocated table metadata and a simple notebook IDE interface.
  • Slashbase - The open-source collaborative IDE for your databases. Connect to your database, browse data, run a bunch of SQL commands or share SQL queries with your team, right from your browser.
  • Sql Server Management Studio - SQL Server Management Studio (SSMS) is an integrated environment for managing any SQL infrastructure, for SQL Server and Azure SQL Databases.
  • Toad - Toad is the premier database solution for developers, admins and data analysts. Manage complex database changes with a single database management tool.
  • Toad Edge - Simplified database development tool for MySQL and Postgres.
  • TOra - TOra is an open source SQL IDE for Oracle, MySQL and PostgreSQL dbs.
  • Valentina Studio - Create, administer, query and explore Valentina DB, MySQL, MariaDB, PostgreSQL and SQLite databases for FREE.
  • WebDB - Free Efficient Database IDE. Featuring Server Discovery, ERD, Data Generator, AI, NoSQL Structure Manager, Database Versioning and many more.

GUI

  • Adminer - Database management in a single PHP file.
  • Another Redis Desktop Manager - Free Open Source Redis Manager. Available on Mac, Linux, Windows, Homebrew, Snap, winget, and more.
  • Azure Data Studio - A data management tool that enables working with SQL Server, PostgreSQL, Azure SQL DB and SQL DW from Windows, macOS and Linux.
  • Beekeeper Studio - Open Source SQL Editor and Database Manager with a privacy commitment in their mission statement.
  • Clidey WhoDB - A lightweight database explorer with next-gen UX for all SQL, NoSQL, Caches, and Queues.
  • DbGate - Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application.
  • DB Lens - Open Source Postgres GUI - Automatic ER diagrams, Internal DB Insights, Disk Utilisation, Performance Metrics, Index Usage, Sequential scan counts and more.
  • DbVisualizer - Universal database tool for developers, DBAs and analysts.
  • JackDB - Direct SQL access to all your data, no matter where it lives.
  • Jailer - Database Subsetting and Relational Data Browsing Tool/Client.
  • Malewicz - Yet Another WEB client for DB schema exploring and performance analysis, but originally created specifically for hacking and extending.
  • MissionKontrol - Modern drag & drop admin panel/client with full user permissions for non-technical users.
  • ocelotgui - For MySQL, MariaDB, and Tarantool. Developed for Linux but can run on Windows.
  • OmniDB - Web tool for database management.
  • Pgweb - Web-based database browser for PostgreSQL, written in Go and works on macOS, Linux and Windows machines.
  • phpLiteAdmin - Web-based SQLite database admin tool written in PHP with support for SQLite3 and SQLite2.
  • phpMyAdmin - A web interface for MySQL and MariaDB.
  • psequel - PSequel provides a clean and simple interface for you to perform common PostgreSQL tasks quickly.
  • PopSQL - Modern, collaborative SQL editor for your team.
  • Postico - A Modern PostgreSQL Client for the Mac.
  • Robo 3T - Robo 3T (formerly Robomongo) is a shell-centric cross-platform MongoDB management tool.
  • Sequel Ace - MySQL/MariaDB database management for macOS.
  • Sequel Pro - Sequel Pro is a fast, easy-to-use Mac database management application for working with MySQL & MariaDB databases.
  • SQLite Expert - Graphical interface supports all SQLite features.
  • sqlite-tui - A TUI for viewing sqlite databases, written in Go.
  • sqlpad - Web-based SQL editor run in your own private cloud.
  • SQLPro - A simple, powerful Postgres manager for macOS.
  • SQuirreL - Graphical SQL client written in Java that will allow you to view the structure of a JDBC compliant database, browse the data in tables, issue SQL commands etc.
  • SQLTools - Database management for VSCode.
  • SQLyog - The most complete and easy to use MySQL GUI.
  • Tabix - SQL Editor & Open source simple business intelligence for Clickhouse.
  • TablePlus - Modern, native, and friendly GUI tool for relational databases: MySQL, PostgreSQL, SQLite & more.
  • TeamPostgreSQL - PostgreSQL Web Administration GUI - use your PostgreSQL databases from anywhere, with rich, lightning-fast AJAX web interface.
  • Query.me - Collaborative SQL editor in Notebook format. Let's you reference query results using JINJA, visualize data, and schedule runs and exports.

CLI

  • ipython-sql - Connect to a database for issue SQL commands within IPython or IPython Notebook.
  • iredis - A Cli for Redis with AutoCompletion and Syntax Highlighting.
  • pgcenter - Top-like admin tool for PostgreSQL.
  • pg_activity - Top-like application for PostgreSQL server activity monitoring.
  • pg_top - Top for PostgreSQL.
  • pspg - Postgres Pager.
  • SQLcl - Oracle SQL Developer Command Line (SQLcl) is a free command line interface for Oracle Database.
  • sqlite-utils - CLI tools for manipulating SQLite database files - inserting data, running queries, creating indexes, configuring full-text search and more.
  • SQLLine - Command-line shell for issuing SQL to relational databases via JDBC.
  • usql - A universal command-line interface for PostgreSQL, MySQL, Oracle Database, SQLite3, Microsoft SQL Server, and many other databases including NoSQL and non-relational databases!

dbcli

  • athenacli - AthenaCLI is a CLI tool for AWS Athena service that can do auto-completion and syntax highlighting.
  • litecli - CLI for SQLite Databases with auto-completion and syntax highlighting.
  • mssql-cli - A command-line client for SQL Server with auto-completion and syntax highlighting.
  • mycli - A Terminal Client for MySQL with AutoCompletion and Syntax Highlighting.
  • pgcli - Postgres CLI with autocompletion and syntax highlighting.
  • vcli - Vertica CLI with auto-completion and syntax highlighting.

Schema

Changes

  • 2bass - Database configuration-as-code tool that utilizes concept of idempotent DDL scripts.
  • Atlas - Inspect and Apply changes to your database schema.
  • Bytebase - Web-based, zero-config, dependency-free database schema change and version control tool for teams.
  • flyway - Database migration tool.
  • gh-ost - Online schema migration for MySQL.
  • liquibase - Database-independent library for tracking, managing and applying database schema changes.
  • migra - Like diff but for PostgreSQL schemas.
  • node-pg-migrate - Node.js database migration management built exclusively for postgres. (But can also be used for other DBs conforming to SQL standard - e.g. CockroachDB.)
  • pg-osc - Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL.
  • Prisma Migrate - Declarative database schema migration tool that uses a declarative data modeling syntax to describe your database schema.
  • Pyrseas - Provides utilities to describe a PostgreSQL database schema as YAML.
  • Reshape - An easy-to-use, zero-downtime schema migration tool for Postgres.
  • SchemaHero - A Kubernetes operator for declarative database schema management (gitops for database schemas).
  • Skeema - Declarative pure-SQL schema management system for MySQL and MariaDB, with support for sharding and external online schema change tools.
  • Sqitch - Sensible database-native change management for framework-free development and dependable deployment.
  • sqldef - Idempotent schema management for MySQL, PostgreSQL, and more.
  • yuniql - Yet another schema versioning and migration tool just made with native .NET Core 3.0+ and hopefully better.

Code generation

  • ddl-generator - Infers SQL DDL (Data Definition Language) from table data.
  • scheme2ddl - Command line util for export Oracle schema to set of ddl init scripts with ability to filter undesirable information, separate DDL in different files, pretty format output.

Diagrams

  • Azimutt - An Entity Relationship diagram (ERD) visualization tool, with various filters and inputs to help understand your database schema.
  • ChartDB - Free and Open-source database diagrams editor, visualize and design your DB with a single query.
  • DrawDB - Free, simple, and intuitive online database design tool and SQL generator.
  • ERAlchemy - Entity Relation Diagrams generation tool.
  • ERD Lab - Free cloud based entity relationship diagram (ERD) tool made for developers.
  • QuickDBD - Simple online tool to quickly draw database diagrams.

Documentations

  • dbdocs - Create web-based database documentation using DSL code.
  • DBML - Database Markup Language, designed to define and document database structures.
  • SchemaCrawler - A free database schema discovery and comprehension tool.
  • Schema Spy - Generating your database to HTML documentation, including Entity Relationship diagrams.
  • tbls - CI-Friendly tool for document a database, written in Go.

Design

  • Database Design - Useful tips for designing robust database schema.
  • DBDiagram - A free, simple tool to draw ER diagrams by just writing code.
  • DbSchema - Universal database designer for out-of-the-box schema management, schema documentation, design in a team, and deployment on multiple databases. DbSchema features tools for writing and executing queries, exploring the data, generating data, and building reports.
  • ERBuilder Data Modeler - Easy-to-use database modeling software for high-quality data models. It's a complete data modeling solution for data modelers and data architects.
  • Moon Modeler - Data modeling tool for both noSQL and relational databases. Available for Windows, Linux and macOS.
  • Navicat Data Modeler - A powerful and cost-effective database design tool which helps you build high-quality conceptual, logical and physical data models.
  • Oracle SQL Developer Data Modeler - Oracle SQL Developer Data Modeler is a free graphical tool that enhances productivity and simplifies data modeling tasks.
  • pgmodeler - Data modeling tool designed for PostgreSQL.
  • WWW SQL Designer - Online SQL diagramming tool.

Samples

API

Building API for your Data

  • Datasette - A tool for exploring and publishing data.
  • DreamFactory - A open source REST API backend for mobile, web, and IoT applications.
  • Graphweaver - Turn multiple data sources into a single GraphQL API.
  • Hasura GraphQL Engine - Blazing fast, instant realtime GraphQL APIs on Postgres with fine grained access control, also trigger webhooks on database events.
  • Oracle REST Data Services - A mid-tier Java application, ORDS maps HTTP(S) verbs (GET, POST, PUT, DELETE, etc.) to database transactions and returns any results formatted using JSON.
  • Prisma - Prisma turns your database into a realtime GraphQL API.
  • PostGraphile - Instantly spin-up a GraphQL API server by pointing PostGraphile at your existing PostgreSQL database.
  • PostgREST - REST API for any Postgres database.
  • prest - Is a way to serve a RESTful API from any databases written in Go.
  • Remult - End-to-end type-safe CRUD via REST API for your database, with fine-grained access control.
  • restSQL - SQL generator with Java and HTTP APIs, uses a simple RESTful HTTP API with XML or JSON serialization.
  • resquel - Easily convert your SQL database into a REST API.
  • sandman2 - Automatically generate a RESTful API service for your legacy database.
  • VulcanSQL - Write templated SQL to automatically exposing RESTful APIs from your database/data warehouse/data lake.

Application platforms

Low-code and no-code platforms for application building

  • Appsmith - Powerful open source low code framework to build internal applications really quickly.
  • Budibase - Low-code platform for creating internal apps in minutes.
  • ILLA Cloud - Low-code internal tool building platform.
  • Nhost - The Open Source Firebase Alternative with GraphQL.
  • Saltcorn - Open source no-code builder for web datatabase applications. Server and drag-and-drop UI builder, data stored in PostgreSQL or SQLite.
  • Tooljet - Open-source low-code platform to build internal tools.

Backup

  • BaRMan - Backup and Recovery Manager for PostgreSQL.
  • pgbackrest - Reliable PostgreSQL Backup & Restore.
  • pgcopydb - Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids).

Cloning

  • Database Lab Engine - Instant thin cloning for Postgres to scale the development process.
  • clone_schema - Postgres clone schema utility without need of going outside of database.
  • Spawn - Cloud service for creating instant database copies for development and CI. No more local db installs, instant recovery to arbitrary save points, isolated copies for each feature branch or test. Instant provisioning regardless of database size.

Monitoring/Statistics/Perfomance

  • ASH Viewer - Provides a graphical view of active session history data within the Oracle and PostgreSQL DB.
  • Metis - Metis provides observability and performance tuning for SQL databases.
  • Monyog - Agentless & Cost-effective MySQL Monitoring Tool.
  • mssql-monitoring - Monitor your SQL Server on Linux performance using collectd, InfluxDB and Grafana.
  • Navicat Monitor - A safe, simple and agentless remote server monitoring tool that is packed with powerful features to make your monitoring effective as possible.
  • Percona Monitoring and Management - Open source platform for managing and monitoring MySQL and MongoDB performance.
  • pganalyze collector - Pganalyze statistics collector for gathering PostgreSQL metrics and log data.
  • pgbadger - A fast PostgreSQL Log Analyzer.
  • pgDash - Measure and track every aspect of your PostgreSQL databases.
  • PgHero - A performance dashboard for Postgres - health checks, suggested indexes, and more.
  • pgmetrics - Collect and display information and stats from a running PostgreSQL server.
  • pgMonitor - All-in-one tool to easily create an environment to visualize the health and performance of your PostgreSQL cluster.
  • pgMustard - A user interface for Postgres explain plans, plus tips to improve performance.
  • pgstats - Collects PostgreSQL statistics, and either saves them in CSV files or print them on the stdout.
  • pgwatch2 - Flexible self-contained PostgreSQL metrics monitoring/dashboarding solution.
  • PostgreSQL Metrics - Service to extract and provide metrics on your PostgreSQL database.
  • Postgres Monitor - An easy-to-use monitoring service for PostgreSQL providing alerts, dashboards, query stats and dynamic recommendations.
  • postgres-checkup - New-generation diagnostics tool that allows users to do a deep analysis of the health of Postgres databases.
  • Promscale - The open-source observability backend for metrics and traces powered by SQL.
  • Telegraf PostgreSQL plugin - Provides metrics for your postgres database.

Prometheus

  • pgSCV - Metrics exporter for PostgreSQL and Postgres-related services.
  • postgres_exporter - Prometheus exporter for PostgreSQL server metrics.
  • pg_exporter - Fully customizable Prometheus exporter for PostgreSQL & Pgbouncer with fine-grained execution control.

Zabbix

  • Mamonsu - Monitoring agent for PostgreSQL.
  • Orabbix - Orabbix is a plugin designed to work with Zabbix Enterprise Monitor to provide multi-tiered monitoring, performance and availability reporting and measurement for Oracle Databases, along with server performance metrics.
  • pg_monz - This is the Zabbix monitoring template for PostgreSQL Database.
  • Pyora - Python script to monitor Oracle Databases.
  • ZabbixDBA - ZabbixDBA is fast, flexible, and continuously developing plugin to monitor your RDBMS.

Testing

  • DbFit - A database testing framework that supports easy test-driven development of your database code.
  • pgTAP - Unit Testing for PostgreSQL.
  • RegreSQL - Regression Testing your SQL queries.
  • SQLancer - Automatically test DBMS in order to find logic bugs in their implementation.

HA/Failover/Sharding

  • Citus - Postgres extension that distributes your data and your queries across multiple nodes.
  • patroni - A template for PostgreSQL High Availability with ZooKeeper, etcd, or Consul.
  • Percona XtraDB Cluster - A High Scalability Solution for MySQL Clustering and High Availability.
  • ShardingSphere - Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
  • stolon - Cloud native PostgreSQL manager for PostgreSQL high availability.
  • pg_auto_failover - Postgres extension and service for automated failover and high-availability.
  • pglookout - PostgreSQL replication monitoring and failover daemon.
  • pgslice - Postgres partitioning as easy as pie.
  • PostgreSQL Automatic Failover - High-Availibility for Postgres, based on industry references Pacemaker and Corosync.
  • postgresql_cluster - PostgreSQL High-Availability Cluster (based on "Patroni" and "DCS(etcd)"). Automating deployment with Ansible.
  • Vitess - Database clustering system for horizontal scaling of MySQL through generalized sharding.

Kubernetes

  • KubeDB - Making running production-grade databases easy on Kubernetes.
  • Postgres operator - The Postgres Operator enables highly-available PostgreSQL clusters on Kubernetes (K8s) powered by Patroni.
  • Spilo - HA PostgreSQL Clusters with Docker.
  • StackGres - Enterprise-grade, Full Stack PostgreSQL on Kubernetes.

Configuration Tuning

  • MySQLTuner-perl - Script written in Perl that allows you to review a MySQL installation quickly and make adjustments to increase performance and stability.
  • PGConfigurator - Free online tool to generate an optimized postgresql.conf.
  • pgtune - PostgreSQL configuration wizard.
  • postgresqltuner.pl - Simple script to analyse your PostgreSQL database configuration, and give tuning advice.

DevOps

  • DBmaestro - DBmaestro accelerates release cycles & supports agility across the entire IT ecosystem.
  • Toad DevOps Toolkit - Toad DevOps Toolkit executes key database development functions within your DevOps workflow —without compromising quality, performance or reliability.

Reporting

  • Chartbrew - Create live dashboards, charts, and client reports from multiple databases and services.
  • Poli - An easy-to-use SQL reporting application built for SQL lovers.

Distributions

  • DBdeployer - Tool that deploys MySQL database servers easily.
  • dbatools - PowerShell module that you may think of like a command-line SQL Server Management Studio.
  • Postgres.app - Full-featured PostgreSQL installation packaged as a standard Mac app.
  • BigSQL - A developer-friendly distribution of Postgres.
  • Elephant Shed - Web-based PostgreSQL management front-end that bundles several utilities and applications for use with PostgreSQL.
  • Pigsty - Battery-Included Open-Source Distribution for PostgreSQL with ultimate observability & Database-as-Code toolbox for developers.

Security

  • Acra - Database security suite. Database proxy with field-level encryption, search through encrypted data, SQL injections prevention, intrusion detection, honeypots. Supports client-side and proxy-side ("transparent") encryption. SQL, NoSQL.
  • Databunker - Special GDPR compliant secure vault for customer records built on top of regular DB.
  • Inspektor - Access control layer for databases. Inspektor leverages open policy agent to make policy decisions.

SQL

Analyzers

  • Holistic.dev - Automatic detection service for database performance, security, and architecture issues.
  • SQLCheck - Automatically detects common SQL anti-patterns.
  • SQLFluff - Dialect-flexible and configurable SQL linter.
  • SQLLineage - SQL Lineage Analysis Tool powered by Python.
  • TSQLLint - A tool for describing, identifying, and reporting the presence of anti-patterns in TSQL scripts.

Extensions

  • PartiQL - SQL-compatible access to relational, semi-structured, and nested data.

Frameworks

  • Apache Calcite - Dynamic data management framework with advanced SQL features.
  • ZetaSQL - Analyzer Framework for SQL.

Formatters

  • CodeBuff - Language-agnostic pretty-printing through machine learning.
  • pgFormatter - A PostgreSQL SQL syntax beautifier.
  • Poor SQL - Instant free and open-source T-SQL formatting.
  • SQL Formatter - JavaScript library for pretty-printing SQL queries.

Games

  • Lost at SQL - A SQL learning game to help you pick up basic SQL skills - so that you can use queries to get information.
  • Schemaverse - A Space-based strategy game implemented entirely within a PostgreSQL database.
  • SQL Island - After the survived plane crash, you will be stuck on SQL Island for the time being. By making progress in the game, you will find a way to escape from this island.
  • SQL Murder Mystery - Designed to be both a self-directed lesson to learn SQL concepts and commands and a fun game for experienced SQL users to solve an intriguing crime.
  • SQL Police Department - In SQLPD, you get to solve crimes while learning SQL at the same time.

Parsers

  • General SQL Parser - Parsing, formatting, modification and analysis for SQL.
  • jOOQ - Parses SQL, translates it to other dialects, and allows for expression tree transformations.
  • JSqlParser - Parses an SQL statement and translate it into a hierarchy of Java classes.
  • libpg_query - C library for accessing the PostgreSQL parser outside of the server environment.
  • More SQL Parsing! - Parse SQL into JSON.
  • sqlparse - Non-validating SQL parser for Python.
  • SQLGlot - Pure Python SQL parser, transpiler, and builder.

Über SQL

Run SQL queries against anything

  • CloudQuery - Extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.
  • csvq - SQL-like query language for CSV.
  • dsq - Commandline tool for running SQL queries against JSON, CSV, Excel, Parquet, and more.
  • MAT Calcite plugin - This plugin for Eclipse Memory Analyzer allows to query heap dump via SQL.
  • OctoSQL - Query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.
  • osquery - SQL powered operating system instrumentation, monitoring, and analytics.
  • Resmo - Audit and evaluate resources using SQL.
  • Steampipe - Use SQL to instantly query your cloud services (AWS, Azure, GCP and more).
  • TextQL - Execute SQL against structured text like CSV or TSV.
  • trdsql - CLI tool that can execute SQL queries on CSV, LTSV, JSON and TBLN.
  • Trino - Trino is a distributed SQL query engine designed to query large data sets distributed over one or more heterogeneous data sources.

Language Server Protocol

Learning

Learning and puzzles for SQL

  • Advanced SQL Puzzles - Difficult set-based SQL puzzles.
  • Hackerrank - Practice coding, prepare for interviews, and get hired.
  • Learn SQL in a Month of Lunches - A book about how to use SQL to retrieve, filter, and analyze data.
  • LeetCode - Enhance your skills, expand your knowledge and prepare for technical interviews.
  • Select Star SQL - Free interactive book which aims to be the best place on the internet for learning SQL.
  • StrataScratch - Data science educational resources.
  • SQL Murder Mystery - Self-directed lesson to learn SQL concepts and commands and a fun game for experienced SQL users to solve an intriguing crime.

Plan

  • pev2 - A VueJS component to show a graphical vizualization of a PostgreSQL execution plan.
  • pg_flame - A flamegraph generator for Postgres EXPLAIN ANALYZE output.

Scripts

Useful SQL-scripts for various purposes

Data

  • dbt - Transform your data by simply writing select statements, while dbt handles turning these statements into tables and views in a data warehouse.
  • QuickTable - Empowers everyone to access, clean, analyze, transform, and model data with no code.

Catalog

  • Amundsen - Metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data.
  • DataHub - The Metadata Platform for the Modern Data Stack.
  • Marquez - Collect, aggregate, and visualize a data ecosystem's metadata.

Lineage

  • Dwh.dev - Nexgen data lineage for Snowflake.

Generation/Masking/Subsetting

  • Benerator - Generate, obfuscate (anonymize / pseudonymize) and migrate data for development, testing and training purposes.
  • dbForge Data Generator for MySQL - Powerful GUI tool for creating massive volumes of realistic test data.
  • dbForge Data Generator for Oracle - Small but mighty GUI tool for populating Oracle schemas with tons of realistic test data.
  • dbForge Data Generator for SQL Server - Powerful GUI tool for a fast generation of meaningful test data for databases.
  • Faker - Generate massive amounts of fake data in the browser and Node.js.
  • Noisia - Harmful workload generator for PostgreSQL.
  • SB Data Generator - Simple and powerful tool to generate and populate selected tables or entire databases with realistic test data for your applications. Generate test data for: Oracle, MS SQL Server, MySQL, PostgreSQL, Firebird, SQLite, Azure SQL Database, Amazon Redshift and Amazon RDS.
  • Synthesized TDK - DevOps' best friend for database masking and generation.

Replication

  • dtle - Distributed Data Transfer Service for MySQL.
  • Litestream - Streaming replication for SQLite.
  • pgsync - Sync Postgres data between databases.
  • pg_chameleon - MySQL to PostgreSQL replica system written in Python 3. The system use the library mysql-replication to pull the row images from MySQL which are stored into PostgreSQL as JSONB.
  • PGDeltaStream - A Golang webserver to stream Postgres changes atleast-once over websockets, using Postgres logical decoding feature.
  • repmgr - The Most Popular Replication Manager for PostgreSQL.

Compare

  • data-diff - Command-line tool and Python library to efficiently diff rows across two different databases.
  • KS DB Merge Tools - GUI to compare and sync DB schema and data. For Oracle Database, MySQL, MariaDB, SQL Server, PostgreSQL, SQLite, MS Access and Cross-DBMS.

Papers

Documents, articles, manifestos and other theoretical materials on database tools

Machine Learning

  • MindsDB - In-database Machine Learning.
  • SQLFlow - Brings SQL and AI together.

Contributing