Skip to content
Data Directory

Query these datasets with Claude

Install a skill and Claude can query this data directly with DuckDB — the catalogs, the schemas and the query patterns, with nothing between it and the files.

For your AI agentRecommended

npx skills add your-org/data-directory

Installs the query-datasets skill — the catalog URLs, the schemas and the query patterns — into Claude Code, Codex or any agent that reads skills.

For your machine

curl -fsSL https://data.bunnytech.app/skill.sh | sh

Checks for the DuckDB CLI, writes a ./datadirectory/ folder with an attach script and per-dataset notes, then connects to all 2 published catalogs and prints the schema it finds. It never installs anything without asking.

Both end up in the same place. Read the script before you run it — it is generated from the published manifests, so it cannot describe data that is not there.

Contribute

Add a dataset

Any open data source can live here. A connector is one small TypeScript package that says where the bytes come from and what shape they are; everything after that — the lakehouse, the sorted Parquet, the manifest, the schedule, the pages on this site — is handled for you.

The repository is written to be read by a coding agent. In practice that means: fork it, paste one prompt, review what comes back.

Three steps

  1. Fork the connector repo

    Everything you need is in it, including the guide the agent reads.

    Fork iosifnicolae2/data-directory-connectors →

    Then, locally
    git clone https://github.com/<your-username>/data-directory-connectors
cd data-directory-connectors
pnpm install

  2. Paste this into Claude

    Run claude in the repo you just cloned, and paste this. Replace the placeholder with the source you want — a URL, an API, a public feed.

    The prompt
    Read CLAUDE.md, then add a connector for <THE DATA SOURCE YOU WANT>.
Follow every step, stop at the licensing gate if the terms are unclear,
and don't open the PR until the checklist passes.

    That is the whole process. The repo’s CLAUDE.md is a complete, self-contained specification — research the source, clear the licensing gate, declare the tables, choose a sort order, keep the transform pure, record fixtures, write the tests, register it, verify it. Claude follows it step by step.

    No agent? CLAUDE.md reads perfectly well as a human guide — it is the same twelve steps either way.

  3. Open a pull request

    Claude runs the checks itself and will tell you when they pass. Say where you found the licence and why you chose the sort order — those are the two things a reviewer always asks about.

    Once it is merged, the dataset is scheduled, published, and appears on the dataset list like every other one here.

What you have to write

Four files, and only two of them contain logic.

src/index.ts
Who the dataset is, where it came from, its licence — and extract(), the only function allowed to touch the network.
src/tables.ts
The tables you publish: every column with a description, a write strategy, and a sort order. The sort order is the only index a lakehouse has, so it is chosen deliberately.
src/transform.ts
Raw payload in, typed rows out. Pure: no clock, no randomness, no filesystem, no network. This is what the tests exercise.
test/
Fixture-driven tests that run with the network switched off, plus one that ingests the same input twice and proves the row count does not move.

The repo ships a working example — usgs-earthquakes — that is exactly this, end to end. Copy its shape.

The rules that get a PR sent back

There are more in CLAUDE.md, but reviewers look hardest at these three.

  1. Licensing is a blocking gate

    A dataset whose terms you cannot determine must not be published. Find the licence, write a credit line a reuser can paste, and say what you are and are not republishing. If you cannot find it, ask — do not guess.

  2. transform must be pure

    No clock, no randomness, no filesystem, no network. Everything time-dependent is handed to it. That is what makes re-running yesterday safe and what makes the tests mean anything.

  3. sortBy is your index

    There are no indexes in a lakehouse — the physical sort order of the Parquet is all the pruning a query gets. Sort by what people filter on, and write down which query it makes fast.

Links