🔗 Visual Pipeline Builder

A node-based visual pipeline editor with DAG validation, built for the VectorShift Frontend Technical Assessment.

🚀 Live Demo →

Overview • Demo • How to Use • Architecture • Getting Started • API Reference • Implementation Notes

---

Overview

Visual Pipeline Builder is a drag-and-drop, node-based editor for composing data/LLM pipelines in the browser. Users drag nodes onto a canvas, wire them together into a directed graph, and submit the graph to a Python backend that validates its structure — specifically, whether it forms a Directed Acyclic Graph (DAG) — before it would be eligible for execution.

This project was built end-to-end for VectorShift's frontend technical assessment, covering four requirements:

#	Requirement	Status
1	Reusable node abstraction (BaseNode) + 5 new demo nodes	✅
2	Unified visual styling across the canvas and nodes	✅
3	Auto-resizing Text node with dynamic {{variable}} → Handle generation	✅
4	Frontend ⇄ backend integration with DAG validation	✅
—	Deployed live (frontend + backend)	✅

⏱️ Note on cold starts: the backend is hosted on Render's free tier, which sleeps after 15 minutes of inactivity. The first request after idle time can take 20–30 seconds to wake up — this is expected, not a bug. Subsequent requests are fast.

---

Demo

Two Input nodes feeding a Text node with a detected {{input}} variable, chained into an LLM node.

---

How to Use

Try it live at visual-pipeline-builder.netlify.app — no signup, no setup. Here's how to build and test a pipeline, and what each node does:

Building a pipeline

1. Drag a node from the toolbar at the top onto the canvas.
2. Wire nodes together by dragging from an output handle (right edge, colored dot) to an input handle (left edge) on another node.
3. Fill in each node's fields — most nodes have simple text/dropdown inputs.
4. Click Submit Pipeline at the bottom. The frontend sends your current graph to the backend, which returns the node count, edge count, and whether the graph is a valid DAG — shown in a result modal.
5. Clear Canvas resets the board if you want to start over.

Node reference

Node	What it's for	How to use it
Input	Entry point of a pipeline — represents external data coming in	Give it a Name and pick a Type (e.g. Text). Its output handle feeds downstream nodes.
Output	Exit point of a pipeline — where a result would be returned	Connect an upstream node's output into it; represents the final value produced by the pipeline.
Text	A text block that can reference other nodes as variables	Type into the text field. Wrapping a word in {{double curly braces}} (e.g. {{input}}) automatically creates a new input handle on the left, matching that variable name — wire an Input node into it to bind the value. The box also grows as you type.
LLM	Represents a language-model call	Wire a system and/or prompt input into it (from Text or Input nodes); its response output handle can feed an Output node.
Filter	Represents a conditional data-filtering step	Connect an upstream data source; used to demonstrate the node abstraction — configure its field to describe the filter condition.
Condition	Represents branching logic (if/else-style routing)	Wire an input to evaluate; intended to route to different downstream paths depending on the condition.
Transform	Represents a data transformation step	Wire in a value, describe the transformation in its field; passes a transformed output downstream.
API Call	Represents an external API request	Configure the endpoint/field; wire its output into whatever consumes the API response.
Note	A free-floating annotation	Not wired into the graph at all — use it to leave comments or context directly on the canvas.

The last five node types (Filter, Condition, Transform, API Call, Note) exist primarily to demonstrate the BaseNode abstraction from Part 1 of the assessment — they show that adding an entirely new node type is a matter of defining its fields/handles, not rewriting UI code.

For developers reviewing the code

If you want to see the DAG check fail on purpose: wire a node's output back into one of its own upstream inputs to form a cycle, then hit Submit Pipeline — is_dag will return false, since a cycle means no node in that loop ever reaches in-degree 0 (see Implementation Notes below for how that's detected).

---

Architecture & Deployment

flowchart LR
    subgraph Browser["Your Browser"]
        U[User]
    end

    subgraph Netlify["Netlify — Frontend"]
        A[React + ReactFlow Canvas] --> B[submit.js]
    end

    subgraph Render["Render — Backend"]
        C["/pipelines/parse"] --> D[Pydantic validation]
        D --> E["check_is_dag()\n(Kahn's Algorithm)"]
    end

    U --> A
    B -- "POST { nodes, edges }" --> C
    E -- "{ num_nodes, num_edges, is_dag }" --> A

Loading

Live services:

Service	Platform	URL
Frontend	Netlify (static build)	visual-pipeline-builder.netlify.app
Backend	Render (Python web service, free tier)	dag-execution-platform.onrender.com

How the two are connected:

• The frontend build is compiled with REACT_APP_API_URL set to the Render backend URL, so every deployed build knows where to send requests — no hardcoded URLs.
• The backend's CORS policy (ALLOWED_ORIGINS env var on Render) explicitly allowlists the Netlify domain, so the browser is permitted to call it cross-origin.
• Both env vars are configured directly in each platform's dashboard (Netlify → Site settings → Environment variables; Render → Environment tab) — not committed to the repo.

Request/response contract:

// POST /pipelines/parse
// Request body
{
  "nodes": [{ "id": "1", "type": "input", "data": { ... } }, ...],
  "edges": [{ "id": "e1-2", "source": "1", "target": "2" }, ...]
}

// Response
{
  "num_nodes": 4,
  "num_edges": 3,
  "is_dag": true
}

---

Features

• 9 node types — Input, Output, LLM, Text, Filter, Condition, Transform, API Call, Note — all built on one shared BaseNode shell
• Dynamic Text node — typing {{variable_name}} live-scans the input and spawns a corresponding input Handle; deleting the variable removes it
• Auto-resizing node body — the Text node grows in width/height as content is typed, so long prompts stay readable
• Connection guarding — isValidConnection prevents a second edge from overwriting an already-connected input handle
• One-click validation — "Submit Pipeline" sends the current graph to the backend and surfaces num_nodes, num_edges, and is_dag in a friendly alert
• O(V + E) cycle detection — backend uses Kahn's algorithm to detect cycles in linear time
• Deployed live — Netlify (frontend) + Render (backend), connected via env-configured CORS and API URL

---

Tech Stack

Layer	Technology	Purpose
UI Framework	React 18	Component-driven canvas & node rendering
Canvas Engine	ReactFlow 11	Node/edge rendering, drag-and-drop, pan/zoom
Backend Framework	FastAPI	REST endpoint for pipeline validation
Data Validation	Pydantic	Request body schema enforcement
Graph Algorithm	Kahn's Topological Sort	Cycle / DAG detection in O(V + E)
Tooling	Create React App (react-scripts)	Dev server, build pipeline
Hosting	Netlify (frontend), Render (backend)	Free-tier static + web service deployment

---

Project Structure

.
├── frontend/
│   ├── src/
│   │   ├── nodes/
│   │   │   ├── BaseNode.js        # Shared shell all node types extend
│   │   │   ├── InputNode.js
│   │   │   ├── OutputNode.js
│   │   │   ├── LLMNode.js
│   │   │   ├── TextNode.js        # Dynamic {{variable}} → Handle logic
│   │   │   └── ...                # 5 additional demo nodes
│   │   ├── submit.js              # POSTs pipeline to backend (uses REACT_APP_API_URL)
│   │   ├── ui.js                  # ReactFlow canvas + toolbar
│   │   └── store.js                # Shared node/edge state
│   └── package.json
├── backend/
│   ├── main.py                    # FastAPI app + /pipelines/parse
│   ├── requirements.txt
│   └── render.yaml                # Render service config
└── docs/
    └── pipeline-demo.png

---

Getting Started (Local Development)

Want to run it yourself instead of using the live demo? Here's how.

Prerequisites

• Node.js 16+ and npm
• Python 3.9+

Frontend

cd frontend
npm install
npm start

Runs at http://localhost:3000

Backend

cd backend
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt
uvicorn main:app --reload

Runs at http://localhost:8000

With both servers running, build a pipeline on the canvas and click Submit Pipeline — an alert will report the node/edge counts and whether the graph is a valid DAG.

Deploying your own copy

Step	Where
1. Push repo to GitHub	—
2. Deploy backend/ as a Web Service	Render — root directory: backend, build: pip install -r requirements.txt, start: uvicorn main:app --host 0.0.0.0 --port $PORT
3. Set ALLOWED_ORIGINS env var on Render	value = your Netlify URL (no trailing slash)
4. Deploy frontend/	Netlify — base dir: frontend, build: npm run build, publish dir: frontend/build
5. Set REACT_APP_API_URL env var on Netlify	value = your Render backend URL

---

API Reference

GET /

Health check.

{ "Ping": "Pong" }

POST /pipelines/parse

Validates a submitted pipeline.

Field	Type	Description
nodes	array	List of node objects, each with a unique id
edges	array	List of edge objects with source and target node IDs

Response

Field	Type	Description
num_nodes	int	Total nodes in the submitted pipeline
num_edges	int	Total edges in the submitted pipeline
is_dag	bool	true if the graph has no cycles

---

Implementation Notes

Part 1 — Node Abstraction

Every node type extends a single BaseNode component that owns the card layout, header styling, and Handle positioning. New node types are defined declaratively (label, fields, handles) instead of duplicating markup — the five demonstration nodes (Filter, Condition, Transform, API Call, Note) took minutes each to add on top of the original four.

Part 2 — Styling

A single shared stylesheet drives hover/focus states and node theming (each node type has its own accent color), so visual changes propagate everywhere at once instead of being scattered across files.

Part 3 — Text Node Logic

The Text node re-scans its content on every keystroke with a regex matching {{ variableName }}. Each unique match is mapped to a Handle rendered on the node's left edge; removing a variable from the text removes its Handle. The node's width/height also grow with content length so multi-line prompts stay legible.

Part 4 — Backend Integration

submit.js reads the current nodes/edges from state and POSTs them to /pipelines/parse (via REACT_APP_API_URL, so the same code works locally and in production). On the backend, Pydantic's PipelineData model rejects malformed payloads before they reach the graph algorithm. Cycle detection uses Kahn's algorithm:

1. Compute in-degree for every node
2. Seed a queue with all nodes whose in-degree is 0
3. Repeatedly pop a node, increment a processed counter, and decrement its neighbors' in-degrees — pushing any neighbor that reaches 0
4. If processed == total nodes, the graph is a DAG; otherwise a cycle exists (nodes inside a cycle never reach in-degree 0, so they never enter the queue)

This runs in O(V + E) — every node and edge is visited exactly once.

Deployment integration

CORS is handled via an ALLOWED_ORIGINS environment variable on the backend rather than a hardcoded list, so the same codebase works against localhost:3000 in development and the live Netlify domain in production without a code change. The frontend mirrors this pattern with REACT_APP_API_URL, injected at build time by Netlify.

---

Possible Extensions

• Persist canvas state to localStorage so pipelines survive a page refresh
• Execute the validated DAG in topological order against real node logic
• Add undo/redo and a minimap zoom-to-fit shortcut
• Auth + saved pipelines per user

---

Author

Somnath B.Tech CSE, University of Kalyani

If you have feedback on this project, feel free to open an issue or reach out.

License

Released under the MIT License.