DataGuardAI — Autonomous Multi-Agent SRE Platform

An autonomous LLM-powered SRE platform that detects Airflow DAG failures, performs root cause analysis using AWS Bedrock / OpenAI, and auto-generates remediation PRs — reducing MTTR from hours to minutes.

---

Architecture

┌────────────────────────────────────────────────────────────────────┐
│                     MONITORING LAYER                                │
│   AWS CloudWatch Logs  |  Airflow REST API  |  Pipeline Metrics    │
└─────────────────────────────┬──────────────────────────────────────┘
                              │  Failure Event
                              ▼
┌────────────────────────────────────────────────────────────────────┐
│                   LANGGRAPH AGENT ORCHESTRATOR                      │
│                                                                     │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────────┐ │
│  │ Triage Agent │→ │  RCA Agent   │→ │ Remediation Agent        │ │
│  │ (classify    │  │ (AWS Bedrock │  │ (generates PR fix,       │ │
│  │  failure)    │  │  log analysis│  │  confidence-scored)      │ │
│  └──────────────┘  └──────────────┘  └──────────────────────────┘ │
│          │                │                      │                  │
│  ┌───────▼────────────────▼──────────────────────▼───────────────┐ │
│  │  State Graph: DETECT → ANALYSE → REMEDIATE → VALIDATE → DONE  │ │
│  └───────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────┘
                              │
              ┌───────────────┼───────────────┐
              ▼               ▼               ▼
   ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐
   │  PostgreSQL  │  │    Redis     │  │  GitHub PR API        │
   │  (state,     │  │  (agent      │  │  (auto-generated fix  │
   │   audit log) │  │   cache)     │  │   pull requests)      │
   └──────────────┘  └──────────────┘  └──────────────────────┘
                              │
                              ▼
                   ┌──────────────────────┐
                   │   Next.js Dashboard  │
                   │   MTTR metrics,      │
                   │   incident timeline, │
                   │   confidence scores  │
                   └──────────────────────┘

---

Key Features

Feature	Detail
8 LangGraph agents	Triage → RCA → Remediation → Validation → PR → Notification → Escalation → Audit
AWS Bedrock / OpenAI	Confidence-scored RCA from CloudWatch logs
Auto PR generation	Generates and opens GitHub PRs with suggested DAG fixes
MTTR dashboard	Real-time Next.js dashboard — incident timeline, resolution rate
Production-ready	Docker + GitHub Actions CI/CD, full test coverage, release-tagged
Validated outputs	Each agent output validated before passing to next stage

---

Results

• MTTR: Reduced mean time to resolution from ~4 hours → ~5 minutes for common DAG failure patterns
• Recovery rate: ~70% of automated remediations accepted without manual intervention
• Coverage: Handles DAG import errors, upstream failures, SLA breaches, schema drift in pipeline outputs

---

Agent Graph

START
  │
  ▼
[triage_agent]     — Classifies failure: IMPORT_ERROR | UPSTREAM_FAILURE |
  │                  SLA_BREACH | SCHEMA_DRIFT | UNKNOWN
  ▼
[rca_agent]        — Queries CloudWatch (60-day window), sends log context
  │                  to Bedrock/OpenAI, returns structured RCA with confidence
  ▼
[remediation_agent]— Generates diff / DAG fix based on RCA output
  │
  ▼
[validation_agent] — Validates proposed fix against known patterns
  │
  ├─(high confidence)→ [pr_agent] → opens GitHub PR
  │
  └─(low confidence) → [escalation_agent] → pages on-call, creates Jira ticket
  │
  ▼
[audit_agent]      — Writes full incident record to PostgreSQL
  │
  ▼
END

---

Technology Stack

Component	Technology
Agent orchestration	LangGraph
LLM providers	AWS Bedrock (Claude), OpenAI GPT-4o
API	FastAPI
State persistence	PostgreSQL (LangGraph checkpointing)
Cache	Redis
Frontend	Next.js
Containerisation	Docker, Docker Compose
CI/CD	GitHub Actions

---

Quick Start

git clone https://github.com/ritesxh/DataGuardAI.git
cd DataGuardAI

# Copy and configure environment
cp .env.example .env
# Edit .env: set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, OPENAI_API_KEY, GITHUB_TOKEN

# Start all services
docker compose up -d

# Run the agent on a sample failure
python -m src.pipeline.run --dag-id etl_daily_trades --run-id 2025-01-15T00:00:00+00:00

---

Project Structure

DataGuardAI/
├── src/
│   ├── agents/
│   │   ├── triage.py          # Failure classification agent
│   │   ├── rca.py             # Root cause analysis (Bedrock/OpenAI)
│   │   ├── remediation.py     # Fix generation agent
│   │   ├── validation.py      # Output validation agent
│   │   ├── pr_agent.py        # GitHub PR creation
│   │   ├── escalation.py      # On-call escalation
│   │   └── audit.py           # PostgreSQL audit trail
│   ├── pipeline/
│   │   ├── graph.py           # LangGraph state machine definition
│   │   └── run.py             # CLI entry point
│   └── api/
│       └── main.py            # FastAPI webhook receiver
├── tests/
├── docker-compose.yml
├── .github/workflows/ci.yml
└── pyproject.toml

---

License

MIT