PyData Global 2024: Scaling Outside the Warehouse Using DuckDB and Python

Welcome to the repository for the PyData Global 2024 talk:
"Scaling Outside the Warehouse Using DuckDB and Python"

This talk explores how DuckDB can process large-scale data efficiently outside traditional warehouse systems, seamlessly integrating with Python pipelines.

Repository Structure

• lambda_ddb_qry_v2/
  Contains the code for an AWS Lambda function that executes SQL queries using DuckDB.

• pydata_global_24/
  A demo notebook illustrating key concepts and workflows demonstrated in the PyData Global 2024 talk.

• utility/
  Includes a utility file with reusable functions to support the demo notebook.