Aegis AI: Security Analytics & Threat Detection Pipeline

Aegis AI is an automated, local Security Information and Event Management (SIEM) and Security Operations Center (SOC) analytics platform. It parses raw cybersecurity logs, executes SQL-based detection queries for known attacks, runs unsupervised machine learning (Isolation Forest) models to catch anomalous behavioral patterns (UEBA), and displays all alerts in an interactive dashboard for forensic analysis and report generation.

---

🚀 Key Features

1. Enterprise Log Simulation: Simulates realistic authentication, web server, and firewall logs complete with normal traffic fluctuations and injected cyberattacks (Brute Force, SQL Injection, Data Exfiltration, and Insider Threats).
2. Hybrid Detection Engine:
   • Static Rules (SQL): Analyzes database logs with advanced SQL queries to flag immediate indicators of compromise (IOCs).
   • AI/ML Analytics (Scikit-Learn): Uses an unsupervised IsolationForest model to detect high-entropy anomalies (e.g., data exfiltration volumes, off-hours access, or unusual external IP activities).
3. Streamlit SOC Dashboard:
   • Overview Tab: Ingestion speed lines, alert severity donuts, and network traffic port breakdown charts.
   • Triage Center Tab: Investigate active alerts, check matching log segments, update status, and generate detailed Markdown Forensic Reports.
   • Threat Hunting Tab: Live SQL Query Editor allowing analysts to run custom database queries.

---

📁 Project Structure

• generator.py: Simulated security log generator. Injects attack scenarios (brute force, SQLi scanning, data exfil, and time anomalies).
• database.py: SQLite database interface. Parses JSON logs, structures tables (auth_logs, web_logs, firewall_logs, alerts), and runs SQL detection queries.
• detector.py: The Machine Learning anomaly detection engine. Extracts features and scales them using Scikit-Learn to fit unsupervised threat classifiers.
• app.py: Streamlit visualization frontend, SQL playground, and forensic report downloader.
• requirements.txt: Package dependencies (Streamlit, Pandas, Scikit-learn, Plotly, NumPy).

---

🛠️ Setup & Running Instructions

1. Install Dependencies

Ensure you have Python 3.9+ installed. Run the following command in your terminal:

pip install -r requirements.txt

2. Generate and Analyze Logs

Run the backend pipeline to generate test logs and ingest them into the SIEM database:

# Step A: Simulate 7 days of logs
python generator.py --days 7

# Step B: Parse logs, initialize SQLite database, and run detection rules
python database.py

# Step C: Execute ML Anomaly Detection model (Isolation Forest)
python detector.py

Note: All logs will be stored in a local ./logs/ folder and the structured database will be created as siem.db in the root folder.

3. Launch the Streamlit Dashboard

Run the following command to start the web-based visualization server:

streamlit run app.py

Once launched, open your web browser and navigate to http://localhost:8501.

---

🕵️ Simulated Cyberattack Scenarios

The generator script automatically injects the following scenarios to test the platform:

1. Brute Force Attempt: Attacker IP 198.51.100.45 initiates 45 failed login attempts followed by a single successful compromise within a 5-minute window.
2. SQL Injection Scanning: Web scanner from IP 203.0.113.88 requesting URL patterns with ' OR '1'='1 and UNION SELECT payloads.
3. Data Exfiltration: Internal host 192.168.1.105 uploading ~5 GB of data to a foreign IP on port 443.
4. Insider Threat (Time Anomaly): Employee account siddharth logging in at 3:14 AM from an external IP and executing network transfers over SSH.