Visual Pipeline Node Builder & DAG Validator

An interactive, full-stack workflow automation canvas that allows users to design multi-stage data pipelines using drag-and-drop node configurations, manage complex network states on the frontend, and validate graph topologies via a high-performance backend processing engine.

---

Key Engineering Features

• Interactive Canvas Interface: Built an extensible front-end node grid using React Flow with safe pointer event capture (nodrag nowheel) to enable smooth real-time text input configurations inside active nodes.
• Optimized State Management: Implemented a unified React global application store via Zustand to manage high-frequency additions, removals, and structural updates across nodes and edges without incurring layout thrashing.
• Strict Memory Lifecycle Governance: Memoized dynamic node configuration lookups via React's useMemo hooks, eliminating redundant object re-allocations and stabilizing the canvas component rendering cycles.
• Modern API Contract Architecture: Developed an asynchronous communication layer mapping frontend payloads directly into structured FastAPI Pydantic schemas, replacing antiquated form-string constraints with native application/json data marshalling.
• Topological Sorting Validation Engine: Built a backend graph analysis routine utilizing Kahn's Algorithm (In-Degree/Queue tracking) to evaluate full pipeline topologies in O(V + E) time complexity, ensuring strict Directed Acyclic Graph (DAG) state integrity.

---

Project Architecture & Setup

This repository is split into a localized microservices architecture consisting of an asynchronous Python backend engine and a modular React frontend workspace.

1. Python Backend Service Setup

The backend runs an asynchronous FastAPI application that processes mathematical graph validations.

# Navigate to the backend directory
cd backend

# Create or activate your virtual environment (e.g., venv or conda)

# Install necessary requirements
pip install fastapi uvicorn pydantic

# Launch the hot-reloading development server
uvicorn main:app --reload --port 8000

Interactive Documentation (Swagger UI): Accessible at http://127.0.0.1:8000/docs once the server is live.

2. React Frontend Canvas Setup

The user interface is powered by React, React Flow, and Zustand to maintain full visual data fidelity.

# Open a separate terminal window and navigate to the frontend directory
cd frontend

# Install UI and state management dependencies
npm install

# Launch the local development server
npm start

Active Canvas Workspace: Accessible at http://localhost:3000. Launch in an Incognito window to avoid browser extension script conflicts.

---

Core Data Architecture & Graph Mechanics

JSON Payload Architecture Contract

When a user triggers the "Execute Graph Validation" process, the frontend transforms the visual canvas geometry into a clean relational data model:

{
  "nodes": [
    { "id": "node_1", "type": "apiNode" },
    { "id": "node_2", "type": "codeNode" }
  ],
  "edges": [
    { "id": "edge_1", "source": "node_1", "target": "node_2" }
  ]
}

Kahn's Algorithm Implementation

To verify that the user's custom workflow can execute from start to finish without getting trapped in an infinite cycle, the backend runs a topological sort:

1. Calculates the In-Degree (number of incoming dependency connections) for every single node in the active canvas.
2. Isolates independent roots (In-Degree = 0) and places them inside an execution tracking Queue.
3. Dequeues nodes sequentially, tracking an aggregated visited_count, and decrements the in-degree scores of neighboring destination components.
4. If a neighbor's in-degree reaches 0, it enters the queue for subsequent analysis.
5. DAG Validation Success: Evaluates to True if and only if the final visited_count matches the total number of ingested nodes across the network, confirming a cycle-free environment.