Skip to content

Add differentiable-SQL demos: ARCO-ERA5 and gradient descent#200

Open
alxmrs wants to merge 1 commit into
claude/xarray-sql-autograd-73ovqqfrom
claude/xarray-sql-era5-demo
Open

Add differentiable-SQL demos: ARCO-ERA5 and gradient descent#200
alxmrs wants to merge 1 commit into
claude/xarray-sql-autograd-73ovqqfrom
claude/xarray-sql-era5-demo

Conversation

@alxmrs

@alxmrs alxmrs commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

Stacked demo branch (on the autograd feature) holding the runnable benchmark scripts, kept out of the core branch so it stays reviewable.

  • grad_era5.py: symbolic grad over real ARCO-ERA5 data (wind-speed sensitivity checked exactly; saturation vapour pressure checked against the closed-form Clausius-Clapeyron slope). The queries ORDER BY latitude DESC, longitude to match ERA5's native order, so results line up with the xarray reference with no sorting on either side (single partition, so the order survives to_dataset).
  • grad_descent.py: gradient descent as ONE declarative recursive-CTE query, with grad() inside the recursion. The loss is written once and differentiated in place; AVG(grad(loss, a)) descends the gradient each step. No Python loop and no precompiled rule — grad() is rewritten to SQL before planning, so it works inside the recursive CTE. Fit matches numpy least-squares.

Stacked demo branch (on the autograd feature) holding the runnable benchmark
scripts, kept out of the core branch so it stays reviewable.

* grad_era5.py: symbolic grad over real ARCO-ERA5 data (wind-speed sensitivity
  checked exactly; saturation vapour pressure checked against the closed-form
  Clausius-Clapeyron slope). The queries ORDER BY latitude DESC, longitude to
  match ERA5's native order, so results line up with the xarray reference with
  no sorting on either side (single partition, so the order survives to_dataset).
* grad_descent.py: gradient descent as ONE declarative recursive-CTE query, with
  grad() inside the recursion. The loss is written once and differentiated in
  place; AVG(grad(loss, a)) descends the gradient each step. No Python loop and
  no precompiled rule — grad() is rewritten to SQL before planning, so it works
  inside the recursive CTE. Fit matches numpy least-squares.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant