Skip to content

About

LakeSail + Marimo (MS Aspire Recipe)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

LakeSail + Marimo (MS Aspire Recipe)

This MS Aspire Recipe starts a local LakeSail Spark Connect server and a Marimo next-generation Python notebook editor.

This recipe was inspired by the blog post Bringing Rust-native Spark compatibility to Microsoft OneLake with Sail and LakeSail.

Combining the developer productivity of MS Aspire with LakeSail, Marimo, and OneLake for a super-low-friction data analytics experience.

📓 Marimo workspace examples

The Marimo workspace includes two examples:

  • Hello Sail.py is a simple introduction to running Spark SQL and DataFrame operations on Sail.
  • MS Fabric & OneLake.py is a more advanced example that connects to a Microsoft Fabric Lakehouse, reads a Lakehouse table, and reads a Parquet file from OneLake. It requires a Microsoft Entra service principal and Microsoft Fabric workspace and Lakehouse configuration.

🧰 Setup

🐳 Install Docker and start the Docker engine.

Install dotnetup from Microsoft:

🍎 On macOS or Linux:

curl -fsSL https://aka.ms/dotnetup/get-dotnetup.sh | bash

🪟 On Windows, in PowerShell:

irm https://aka.ms/dotnetup/get-dotnetup.ps1 | iex

🔧 Follow the printed instructions to add dotnetup to your PATH, then open a new terminal. Run the interactive setup:

dotnetup init

Choose No, customize setup, select the 10 SDK channel, and choose shell access mode (or everywhere on Windows).

Open a new terminal after setup, then verify the SDK:

dotnet --version
dotnetup list

📦 Install the Microsoft Aspire CLI version used by this project:

dotnet tool install -g Aspire.Cli --version 13.6.5

Ensure the .NET global tools directory is on your PATH, then verify with aspire --version.

🚀 Run

aspire start

🌐 Open the Microsoft Aspire dashboard URL printed by the CLI and use the marimo resource's ui endpoint to open the editor.

📡 LakeSail OpenTelemetry support is still experimental with OTLP export (logs, metrics and traces) to the local Microsoft Aspire dashboard enabled.

The Spark Connect endpoint is exposed on port 50051.

📚 References

About

LakeSail + Marimo (MS Aspire Recipe)

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages