This MS Aspire Recipe starts a local LakeSail Spark Connect server and a Marimo next-generation Python notebook editor.
This recipe was inspired by the blog post Bringing Rust-native Spark compatibility to Microsoft OneLake with Sail and LakeSail.
Combining the developer productivity of MS Aspire with LakeSail, Marimo, and OneLake for a super-low-friction data analytics experience.
The Marimo workspace includes two examples:
Hello Sail.pyis a simple introduction to running Spark SQL and DataFrame operations on Sail.MS Fabric & OneLake.pyis a more advanced example that connects to a Microsoft Fabric Lakehouse, reads a Lakehouse table, and reads a Parquet file from OneLake. It requires a Microsoft Entra service principal and Microsoft Fabric workspace and Lakehouse configuration.
🐳 Install Docker and start the Docker engine.
Install dotnetup from Microsoft:
🍎 On macOS or Linux:
curl -fsSL https://aka.ms/dotnetup/get-dotnetup.sh | bash🪟 On Windows, in PowerShell:
irm https://aka.ms/dotnetup/get-dotnetup.ps1 | iex🔧 Follow the printed instructions to add dotnetup to your PATH, then open a new terminal.
Run the interactive setup:
dotnetup initChoose No, customize setup, select the 10 SDK channel, and choose shell access
mode (or everywhere on Windows).
Open a new terminal after setup, then verify the SDK:
dotnet --version
dotnetup list📦 Install the Microsoft Aspire CLI version used by this project:
dotnet tool install -g Aspire.Cli --version 13.6.5Ensure the .NET global tools directory is on your PATH, then verify with aspire --version.
aspire start🌐 Open the Microsoft Aspire dashboard URL printed by the CLI and use the marimo resource's ui endpoint to open the editor.
📡 LakeSail OpenTelemetry support is still experimental with OTLP export (logs, metrics and traces) to the local Microsoft Aspire dashboard enabled.
The Spark Connect endpoint is exposed on port 50051.
- Refer to the OneLake catalog guide in the Sail documentation.
- Learn more about Microsoft OneLake.
- Explore Sail on GitHub and the getting started guide.