Warning: Some posts on this platform may contain adult material intended for mature audiences only. Viewer discretion is advised. By clicking ‘Continue’, you confirm that you are 18 years or older and consent to viewing explicit content.
initially read my data from whatever source (CSV, relational database somewhere, whatever)
write it to one or more parquet files in a directory
tell duckdb that the directory is my data source
Then duckdb treats the directory just like a databese that you can build indexes on, and since they’re parquet files they’re hella small and have static typing. It was pretty fast and efficient before, and duckdb has really sped up my data wrangling and analysis a ton.
Bruh, this looks cool af. Seriously has me wondering whether I should port my shit from SQLite to duckDB.
I love duckDB, my usual workflow is:
Then duckdb treats the directory just like a databese that you can build indexes on, and since they’re parquet files they’re hella small and have static typing. It was pretty fast and efficient before, and duckdb has really sped up my data wrangling and analysis a ton.