MicrOps — No-Code Cloud DevOps & Container Deployment Platform

MicrOps (microps.in) is a self-hosted PaaS I built solo: paste a Git repository URL and it inspects the repo, builds an OCI image on an isolated build runner, pushes it to Amazon ECR, and deploys it as a serverless container on AWS ECS Fargate behind an Application Load Balancer — with live build logs streamed to the dashboard over SSE.

⚠️ Project Status (read this first)

• The live instance is intentionally stopped. Hosting user deployments on my own AWS account costs real money, so the EC2 box is off and microps.in currently returns a Cloudflare 522. Everything described below is implemented in this repo; it is just not serving traffic right now.
• I am rebuilding MicrOps in Java (Netty + Spring Boot, microservices) as a separate project: codesbysaravana/microps_microservices. This TypeScript repo is the original, working version.
• Known issues are listed at the bottom of this README — including the fact that npm ci and tsc are currently broken on main. This README describes what the code actually does, not what I want it to do.

---

✨ What It Does

Pre-flight repository inspection

• Clones the target repo and detects the runtime (Node.js, Python, Go, Java) and framework (Next.js, Vite, NestJS, Express, FastAPI, Django, Flask, and generic variants) via a pluggable detector chain (backend/src/services/preflight/detectors.service.ts).
• Produces a cost estimate and suggested Fargate CPU/memory spec before anything is built (engine.service.ts — the "cost oracle").

Zero-config container builds

• A BullMQ worker (concurrency 5) generates an inline Dockerfile per runtime — Node 20, Python 3.11, Go 1.21, Maven 3.9 / Gradle 8.2 (JDK 17) — builds the image, and pushes it to a single ECR repo (microps-hq).
• Images are tagged u{userId}-p{projectId}-{buildId} for per-tenant isolation (backend/src/utils/naming.ts). The ECR lifecycle policy in infra/ecr/lifecycle-policy.json expires untagged images after 7 days and keeps only the last 5 tagged images.
• Builds are resilient by design: TypeScript errors and missing build scripts degrade to warnings, Node projects fall back to direct tsx execution, Python projects skip incompatible dependencies.

AI-assisted failure diagnostics

• A diagnostic engine with 16 deterministic rules (peer-dependency conflicts, OOM, missing build/start scripts, port conflicts, etc.) plus an OpenAI fallback analyses failed build logs and returns a root cause with a confidence score (backend/src/services/diagnostics/diagnostic.engine.ts).
• Fixes are applied with one click through POST /api/v1/build/apply-fix, and the worker automatically retries a failed build up to 2 times with the fix applied.

Serverless deployment on ECS Fargate

• Each deployment provisions an on-demand Fargate task (256 CPU / 512 MB) and attaches it to an ALB target group. Target groups, services and task families get SHA-256 hash-based names so similar project names never collide.
• Tenant URLs follow tenant-{userId}-{projectName}.{BASE_DOMAIN}.
• A background job checks every 5 minutes and scales services idle for more than 15 minutes (configurable via IDLE_THRESHOLD_MINUTES) down to zero tasks to save cost.

Bring Your Own Cloud (BYOC) via OIDC

• Users can deploy to their own AWS account without handing over access keys: MicrOps injects a GitHub Actions workflow into their repo and vendored short-lived (15-minute) credentials via sts:AssumeRoleWithWebIdentity, verified against GitHub's JWKS (backend/src/controllers/build.sts.controller.ts).

Billing, auth and platform bits

• Stripe billing with 4 plans (Free / Pro / Ultra / Enterprise), webhook signature verification and an idempotency table.
• Email/password JWT auth plus GitHub and Google OAuth.
• Per-user environment variables encrypted at rest with AES-256-GCM (backend/src/utils/encryptEnv.ts).
• Rate limiting in two layers: nginx edge limits (10 req/s general, 5 req/s auth) and Express limits (100 req/15 min general, 10 req/15 min auth), plus helmet security headers.
• Transactional email via Resend.
• Live build logs stream to the dashboard over Server-Sent Events, filtered per tenant.

CLI

microps-cli is published on npm:

npm i -g microps-cli
microps login
microps projects
microps link

Commands: login, projects, repos, link, deploy (cli/src/commands/).

---

🏛️ Architecture

graph TD
    User((User))
    WebUI[MicrOps Dashboard<br/>React 19 + Vite]
    CLI[MicrOps CLI<br/>microps-cli]

    GitHub[GitHub API / OAuth]
    Stripe[Stripe Billing]
    OpenAI[OpenAI Diagnostics]
    Postgres[(PostgreSQL)]
    Redis[(Redis / BullMQ)]

    subgraph Backend ["MicrOps Backend (EC2, PM2)"]
        API[Express API]
        BuildWorker[BullMQ Build Worker]
        DiagSvc[Diagnostic Engine]
    end

    subgraph AWS ["AWS (ap-southeast-2)"]
        ECR[Amazon ECR<br/>microps-hq]
        ALB[Application Load Balancer]
        ECS[ECS Fargate Cluster]
    end

    User --> WebUI
    User --> CLI
    WebUI <-->|REST / SSE| API
    CLI <-->|REST| API
    API <--> BuildWorker
    API <--> DiagSvc
    API <--> Postgres
    BuildWorker <--> Redis
    DiagSvc <--> OpenAI
    API <--> Stripe
    API <--> GitHub
    BuildWorker -->|Build & Push| ECR
    BuildWorker -->|Provision Task| ECS
    ALB -->|Routes Traffic| ECS

Loading

Production serving: nginx on the EC2 box serves the built React SPA from /var/www/microps-frontend and proxies /api/ to the Express process on port 8000 (PM2). The frontend is also synced to an S3 bucket (microps-client). Both deploy via GitHub Actions on push to main (.github/workflows/).

---

📁 Repository Structure

microps/
├── backend/                  # Express 4 + TypeScript API (port 8000)
│   ├── src/
│   │   ├── app.ts            # Express setup: helmet, CORS, rate limiting
│   │   ├── server.ts         # Entrypoint; runs the schema setup on boot
│   │   ├── controllers/      # Auth, GitHub/Google OAuth, projects, build, billing, webhooks, STS
│   │   ├── routes/v1/        # /api/v1 routers
│   │   ├── services/
│   │   │   ├── build/        # BullMQ builder, ECS deployer, GitHub Actions provider
│   │   │   ├── diagnostics/  # 16-rule engine + OpenAI fallback
│   │   │   └── preflight/    # Runtime/framework detectors + cost oracle
│   │   ├── repository/       # Postgres access (pg pool)
│   │   ├── scripts/          # init_enterprise_schema.ts (authoritative schema)
│   │   ├── jobs/             # Idle scale-to-zero job
│   │   ├── utils/            # ECR/TG naming, event bus, AES-GCM encryption
│   │   └── __tests__/        # Jest tests (see Testing for current state)
│   └── jest.config.js
├── frontend/                 # React 19 + Vite SPA
│   ├── src/
│   │   ├── components/
│   │   │   ├── deploy/       # Deploy flow: RepoInput, ConfigPanel, PreflightResults,
│   │   │   │                 # PipelineView, LogConsole (SSE), DiagnosticBanner, etc.
│   │   │   ├── dashboard/    # Overview + deployment control center
│   │   │   ├── landing/      # Landing page (ogl/WebGL Plasma visual)
│   │   │   ├── billing/      # 4-plan pricing
│   │   │   └── settings/
│   │   ├── pages/            # Landing, Login, Signup, Dashboard, Docs, Reviews, Policy, Integrations
│   │   ├── services/         # API clients
│   │   ├── store/            # Zustand auth store
│   │   └── lib/api.ts        # HTTP client (VITE_API_URL, JWT Bearer)
│   └── index.html            # Loads Tailwind via the Play CDN script
├── cli/                      # microps-cli (published on npm)
├── infra/
│   ├── nginx/                # Production nginx config + notes
│   ├── ecr/                  # ECR lifecycle policy
│   └── github-runner-vault-workflow/  # Workflow installed on the self-hosted build runner
└── .github/workflows/        # deploy-backend.yml (EC2/PM2), deploy-frontend.yml (S3 + EC2)

🧰 Stack

• Backend: Node.js 22, Express 4, TypeScript 5, BullMQ + ioredis, pg (PostgreSQL), Stripe, AWS SDK v3 (ECS, ELBv2, STS), Zod, helmet, express-rate-limit
• Frontend: React 19, Vite 8, TypeScript, react-router 7, Zustand, react-hook-form + Zod, Tailwind CSS (Play CDN), ogl (WebGL)
• Infra: AWS EC2 + PM2, ECS Fargate, ECR, ALB, S3, nginx, GitHub Actions, self-hosted build runner
• CLI: Node.js, Commander (published as microps-cli)

---

🚀 Local Development

Prerequisites: Node.js 22, Docker (for Redis), PostgreSQL.

git clone https://github.com/codesbysaravana/microps.git
cd microps

# Redis for BullMQ
docker run -d --name microps-redis -p 6379:6379 redis:latest

# Install backend + frontend (use install, not ci — see Known Issues)
npm run install:all

# Run both (backend on :8000, frontend on :5173)
npm run dev

Backend environment variables (.env in backend/):

Variable	Purpose
DATABASE_URL (or PGHOST/PGUSER/PGPASSWORD/PGDATABASE)	PostgreSQL connection
REDIS_HOST / REDIS_PORT	BullMQ queue
JWT_SECRET	Auth tokens (required, no default)
STRIPE_SECRET_KEY + webhook secret	Billing
OPENAI_API_KEY	Diagnostic fallback
AWS_REGION, AWS credentials, ECR/ECS/ALB identifiers	Builds and deploys
BASE_DOMAIN	Tenant subdomains (defaults to microps.in in most of the code)
RESEND_API_KEY, RESEND_SENDER_EMAIL	Transactional email
GitHub / Google OAuth client IDs and secrets	OAuth login

The frontend reads VITE_API_URL (defaults to https://microps.in/api/v1, so set it to http://localhost:8000/api/v1 for local dev).

Database schema: npm run migrate in backend/ runs src/scripts/init_enterprise_schema.ts, which is the authoritative schema (also executed on server boot).

---

🧪 Testing — honest current state

Jest is the test runner (npm test in backend/). As of the latest commit on main:

• 42 tests total: 37 passing, 5 failing.
• One suite (workflow.test.ts) does not run at all — it imports vitest, but the project runs Jest.
• CI does not run the tests; the deploy workflows only install and build.

An earlier version of this README claimed "38 automated tests, zero failures, passing in CI." That was not true, and claiming it was my mistake.

---

⚠️ Known Issues (verified against the code, not yet fixed)

1. npm ci fails on main. vitest was added to runtime dependencies and conflicts with @types/node 20 — npm crashes resolving the tree. The deploy workflows run npm ci, so pushes to main currently cannot build or deploy. Workaround locally: npm install / npm ci --legacy-peer-deps.
2. tsc fails on main. Two ioredis copies in the lockfile produce an incompatible-type error where the BullMQ worker is constructed in builder.service.ts.
3. 5 of 17 Jest tests fail, and workflow.test.ts is written for Vitest while the repo uses Jest.
4. Plan quotas are not enforced. requireQuota middleware and a quota service exist, but no route mounts them — plan limits are displayed, not applied.
5. The GitHub build-completion webhook has no authentication/signature check (the Stripe webhook in the same router does verify signatures).
6. GitHub/Google OAuth tokens are stored unencrypted in the users table (env vars get AES-256-GCM; these don't).
7. Schema management is ad-hoc: the full schema setup runs on every server start, and there are no versioned migrations.
8. JWT is accepted as a ?token= query param on the SSE stream endpoint, which means tokens land in nginx access logs.
9. nginx HTTPS block is commented out — traffic to the box is plain HTTP unless Cloudflare is in front.
10. BASE_DOMAIN falls back to microps.com in deploy.service.ts but microps.in everywhere else.
11. The backend depends on ogl (a WebGL library) and microps-cli — both almost certainly belong only to the frontend/CLI.

---

📚 Existing docs in this repo

• infra/nginx/README.md — nginx commands and deployment notes
• infra/ecr/lifecycle-policy.json — ECR image retention rules

(Previous versions of this README linked nine documentation files that do not exist. Those links have been removed.)

---

Built solo as a working proof that deployment tooling can be a product, not a pile of YAML. Now being rebuilt properly in Java: microps_microservices.