Portable Data Stack
A lightweight, self-contained data analytics environment built with:
- DuckDB - In-process analytical database
- dbt - Data transformation
- Dagster - Orchestration
- Evidence - Dashboards
- Marimo - Interactive data analysis
Quick Start
Run the following commands to start each component:
# Start Dagster server
./speedrun_setup.sh start
# In a new terminal, materialize all assets
./speedrun_setup.sh materialize
# In another terminal, start the Evidence dashboard
./speedrun_setup.sh evidence
# Or start all services in the background
./speedrun_setup.sh -UIComponents
- Dagster UI: http://localhost:3000
- Evidence Dashboard: http://localhost:4000
- Marimo: Opens in your browser when started
Project Structure
.
├── config/ # Configuration files
├── db/ # DuckDB database files
├── data/ # Raw data files
├── logs/ # Log files
└── src/ # Source code
├── dagster/ # Dagster orchestration
│ ├── assets/ # Data generation assets
│ ├── resources/ # I/O managers and resources
│ ├── jobs/ # Job definitions
│ └── sensors/ # Sensors and schedules
├── dbt/ # dbt transformations
│ ├── models/ # dbt models
│ │ ├── staging/ # Staging models
│ │ └── marts/ # Mart models
│ ├── macros/ # dbt macros
│ └── seeds/ # dbt seeds
├── evidence/ # Evidence dashboards
│ ├── pages/ # Dashboard pages
│ └── sources/ # Data sources
└── marimo/ # Marimo notebooks
└── notebooks/ # Interactive notebooks
Package Management
- Python packages: Always use
uvinstead of pip- Example:
uv add pandas
- Example:
- Node.js packages: Always use
buninstead of npm- Example:
bun add d3
- Example: