🛡️ PDF Malware Detection System

Machine Learning + AI-Powered PDF Risk Analyzer

Spring Boot Microservices | FastAPI ML Model | Local Ollama LLM | Streamlit Frontend

---

📌 Overview

This system analyzes PDF files for malware using a hybrid approach that combines:

1. Machine Learning Model (Python/FastAPI) → predicts malicious / benign
2. PDF structure feature extraction using PDFBox (Java)
3. AI LLM-based analysis generated via a local Ollama model
4. Streamlit Frontend → uploads files and displays the HTML report

The system produces:

• ML prediction
• Confidence score
• Extracted PDF features
• AI-generated malware analysis report (HTML)
• Beautiful report rendered inside the Streamlit UI

---

⭐ Features

• 📄 PDF feature extraction (metadata, objects, scripts, encryption, page data)
• 🧠 ML-based malware prediction (malicious, benign)
• 🤖 LLM-generated HTML analysis via Ollama
• 🔐 Base64-encoded HTML response for security
• 🎨 Streamlit UI with fixed light background for dark mode users
• ⚡ Optimized for < 15 second response time using lightweight local models
• 🧱 Modular, scalable, extendable architecture
• ☸️ Production-ready Kubernetes deployment with HPA, Ingress, ConfigMaps & Secrets
• 🔧 Multi-environment config support (local, development, kubernetes, staging, production)

---

🏗️ System Architecture

                          ┌─────────────────────┐
                          │     Ingress          │
                          │  pdf-malware.local   │
                          └─────────┬────────────┘
                                    │
                          ┌─────────▼────────────┐
                          │   Frontend (Streamlit)│
                          │   Port: 8501          │
                          │   NodePort: 30501     │
                          └─────────┬────────────┘
                                    │ POST /api/v1/scan
                          ┌─────────▼────────────┐
                          │  Backend (Spring Boot)│
                          │  Port: 8081           │
                          │  ClusterIP            │
                          └──┬──────────────┬─────┘
                             │              │
            POST /predict   │              │  POST /api/v1/ai/analyze
                             │              │
                  ┌──────────▼──┐    ┌──────▼──────────┐
                  │  ModelApi    │    │  AIService       │
                  │  (FastAPI)   │    │  (Spring Boot)   │
                  │  Port: 8000  │    │  Port: 8082      │
                  │  ClusterIP   │    │  ClusterIP       │
                  └─────────────┘    └───────┬──────────┘
                                             │
                                    ┌────────▼──────────┐
                                    │   Ollama LLM      │
                                    │   (via ngrok /     │
                                    │    localhost:11434)│
                                    └───────────────────┘

---

📂 Project Structure

Pdf-Malware-Detection-System/
├── Backend/                          # Spring Boot - Main orchestrator
│   ├── src/main/java/com/pict/
│   │   ├── controller/
│   │   │   └── PdfScanController.java
│   │   ├── service/
│   │   │   ├── PdfFeatureExtractor.java
│   │   │   ├── PythonPredictClient.java
│   │   │   ├── AIAnalysisBuilderService.java
│   │   │   ├── AiService.java
│   │   │   └── CombineService.java
│   │   ├── dtos/
│   │   └── config/
│   │       └── WebFluxConfig.java
│   ├── src/main/resources/
│   │   ├── application.yml            # Base config (port 8081)
│   │   ├── application-local.yml      # Local profile (localhost URLs)
│   │   └── application-k8s.yml        # K8s profile (service DNS URLs)
│   ├── deploy/                        # Kubernetes manifests
│   │   ├── configmap.yml
│   │   ├── secret.yml
│   │   ├── service.yml
│   │   ├── deployment.yml
│   │   ├── hpa.yml
│   │   └── deploy.sh
│   └── Dockerfile
│
├── AIService/                         # Spring Boot - LLM AI analysis
│   ├── src/main/java/edu/pict/
│   │   ├── controller/
│   │   │   └── AIController.java
│   │   ├── service/
│   │   └── dtos/
│   ├── src/main/resources/
│   │   ├── application.yml            # Base config (port 8082)
│   │   ├── application-local.yml      # Local profile
│   │   └── application-k8s.yml        # K8s profile
│   ├── deploy/                        # Kubernetes manifests
│   │   ├── configmap.yml
│   │   ├── secret.yml
│   │   ├── service.yml
│   │   ├── deployment.yml
│   │   ├── hpa.yml
│   │   └── deploy.sh
│   └── Dockerfile
│
├── ModelApi/                          # FastAPI - ML prediction
│   ├── app.py
│   ├── config/
│   │   ├── settings.py                # Environment-aware config loader
│   │   ├── local.env
│   │   ├── dev.env
│   │   ├── kubernetes.env
│   │   ├── staging.env
│   │   └── production.env
│   ├── deploy/                        # Kubernetes manifests
│   │   ├── configmap.yml
│   │   ├── secret.yml
│   │   ├── service.yml
│   │   ├── deployment.yml
│   │   ├── hpa.yml
│   │   └── deploy.sh
│   ├── pdf_malware_detector.json      # XGBoost model file
│   └── Dockerfile
│
├── Frontend/                          # Streamlit - Web UI
│   ├── app.py
│   ├── config/
│   │   ├── settings.py                # Environment-aware config loader
│   │   ├── local.env
│   │   ├── dev.env
│   │   ├── kubernetes.env
│   │   ├── staging.env
│   │   └── production.env
│   ├── deploy/                        # Kubernetes manifests
│   │   ├── configmap.yml
│   │   ├── secret.yml
│   │   ├── service.yml
│   │   ├── deployment.yml
│   │   ├── hpa.yml
│   │   ├── ingress.yml
│   │   └── deploy.sh
│   └── Dockerfile
│
├── Model/                             # ML model training scripts
├── TestingWithMalciousFile/           # Test PDF samples
├── build.sh                           # Build all Docker images
├── deployment.sh                      # Deploy all services to K8s
├── docker-compose.yml                 # Local Docker Compose setup
└── push.sh                            # Push images to registry

---

⚙️ Technologies Used

Component	Technology
ML Model	Python, XGBoost, FastAPI
LLM	Ollama (Gemma3, Phi3, Qwen, Mistral)
Backend	Spring Boot (Java 21)
AI Service	Spring Boot (Java 21)
PDF Parsing	Apache PDFBox
Networking	Spring WebClient (Reactive)
Frontend	Streamlit
Containerization	Docker (multi-stage builds)
Orchestration	Kubernetes (Deployments, Services, HPA, Ingress)
Config Management	ConfigMaps, Secrets, Spring Profiles, Python env files

---

🚀 Setup & Installation

1️⃣ Install Ollama

Download: https://ollama.com/download

Pull a recommended model:

ollama pull gemma3:latest

Run the model:

ollama run gemma3

---

2️⃣ Clone Repository

git clone https://github.com/shrihari7396/Pdf-Malware-Detection-System.git
cd Pdf-Malware-Detection-System

---

3️⃣ Choose Your Deployment Method

Option A: Run Locally (Without Containers)

Start Ollama

ollama serve   # Runs on http://localhost:11434

Start ModelApi (Python ML Server)

cd ModelApi
pip install -r requirements.txt
python app.py   # Runs on http://localhost:8000

Start Backend (Spring Boot)

cd Backend
./mvnw spring-boot:run   # Runs on http://localhost:8081

Start AIService (Spring Boot)

cd AIService
./mvnw spring-boot:run   # Runs on http://localhost:8082

Start Frontend (Streamlit)

cd Frontend
pip install -r requirements.txt
streamlit run app.py   # Opens at http://localhost:8501

---

Option B: Run with Docker Compose

docker-compose up --build

Access the app at http://localhost:8501

---

Option C: Deploy to Kubernetes (Recommended for Production)

See the Kubernetes Deployment section below.

---

🔧 Environment Configuration

Spring Boot Services (Backend & AIService)

Both Spring Boot services use Spring Profiles for environment-specific configuration:

Profile	Activated By	Config File	Use Case
local	Default / SPRING_PROFILES_ACTIVE=local	application-local.yml	Local development
k8s	SPRING_PROFILES_ACTIVE=k8s	application-k8s.yml	Kubernetes deployment

Environment Variables & Defaults

Environment Variable	Description	Local Default	K8s Default
SPRING_PROFILES_ACTIVE	Active Spring profile	local	k8s
AI_ANALYSIS_URL	AIService endpoint	http://localhost:8082/api/v1/ai/analyze	http://ai-service:8082/api/v1/ai/analyze
PYTHON_PREDICT_URL	ModelApi endpoint	http://localhost:8000/predict	http://model-api:8000/predict
OLLAMA_BASE_URL	Ollama LLM endpoint	http://localhost:11434	Configured via ConfigMap (ngrok URL)
OLLAMA_MODEL	Ollama model name	gemma3:latest	gemma3:latest

Python Services (Frontend & ModelApi)

Python services use an environment-aware config loader (config/settings.py) that:

1. Reads APP_ENV environment variable (defaults to local)
2. Loads the matching config/{APP_ENV}.env file
3. Falls back to hardcoded defaults if env vars are missing

Environment	APP_ENV Value	Env File
Local	local	config/local.env
Development	development	config/dev.env
Kubernetes	kubernetes	config/kubernetes.env
Staging	staging	config/staging.env
Production	production	config/production.env

---

☸️ Kubernetes Deployment

📋 Prerequisites

• A running Kubernetes cluster (e.g., Minikube, Kind, or cloud-managed K8s)
• kubectl CLI configured to communicate with your cluster
• Docker installed locally
• Ollama running locally (exposed via ngrok for K8s access)

🏗️ 1. Build Container Images

The build.sh script builds all 4 Docker images. If using Minikube, it automatically points Docker to Minikube's internal daemon — no need to push to a registry:

chmod +x build.sh
./build.sh

This builds:

Service	Image Name
ModelApi	pdf-malware-system-model-api:latest
AIService	ai-service-backend:latest
Backend	pdf-malware-backend:latest
Frontend	pdf-malware-system-frontend:latest

🔌 2. Expose Ollama via ngrok

Since Ollama runs on your local machine with GPU, expose it to the K8s cluster via ngrok:

# Start Ollama
ollama serve

# In another terminal, expose port 11434 via ngrok
ngrok http 11434

Copy the ngrok URL (e.g., https://abc123.ngrok-free.app) and update the AIService ConfigMap:

# AIService/deploy/configmap.yml
data:
  OLLAMA_BASE_URL: "https://your-ngrok-url.ngrok-free.app"

🚀 3. Deploy All Services

chmod +x deployment.sh
./deployment.sh

This deploys services in the correct dependency order:

1. ModelApi → model-api ClusterIP service on port 8000
2. AIService → ai-service ClusterIP service on port 8082
3. Backend → backend ClusterIP service on port 8081
4. Frontend → frontend NodePort service on port 8501 / NodePort 30501 + Ingress

Kubernetes Resources Per Service

Resource	Frontend	Backend	AIService	ModelApi
ConfigMap	✅	✅	✅	✅
Secret	✅	✅	✅	✅
Deployment	✅	✅	✅	✅
Service	NodePort	ClusterIP	ClusterIP	ClusterIP
HPA	✅ (1-5 pods)	✅ (1-5 pods)	✅ (1-5 pods)	✅ (1-5 pods)
Ingress	✅	—	—	—

🔎 4. Verify the Deployment

# Check all resources
kubectl get all

# Check pods are running
kubectl get pods

# Check services
kubectl get svc

# Check ingress
kubectl get ingress

# View logs
kubectl logs -l app=backend --tail=100
kubectl logs -l app=ai-service --tail=100
kubectl logs -l app=model-api --tail=100
kubectl logs -l app=frontend --tail=100

🌐 5. Access the Application

Option 1: Via Ingress (Recommended)

Enable the Ingress controller (Minikube):

minikube addons enable ingress

Add hostname to /etc/hosts:

echo "$(minikube ip) pdf-malware.local" | sudo tee -a /etc/hosts

Access at: http://pdf-malware.local

Option 2: Via NodePort

minikube service frontend

Option 3: Via Port Forwarding

kubectl port-forward svc/frontend 8501:8501

Access at: http://localhost:8501

---

🔍 API Documentation

📌 1. /scan — Main Scan Endpoint (Backend)

POST http://localhost:8081/api/v1/scan
Content-Type: multipart/form-data

Request:

Upload a PDF file with key file.

Response (PredictionResponse):

{
  "prediction": "malicious",
  "confidence": 0.998,
  "features": { ... },
  "explanation": { ... },
  "htmlAnalysis": "BASE64_ENCODED_HTML_STRING"
}

---

📌 2. /ai/analyze — AI Analysis Endpoint (AIService)

POST http://localhost:8082/api/v1/ai/analyze
Content-Type: application/json

Request Body (AIAnalysisRequest):

{
  "prediction": "malicious",
  "confidence": 0.58,
  "features": { ... },
  "extractedText": "...",
  "metadata": { ... },
  "fileName": "sample.pdf",
  "fileSize": 12345
}

Response:

{
  "htmlReport": "<div>... Full HTML analysis report ...</div>"
}

---

📌 3. /predict — ML Prediction Endpoint (ModelApi)

POST http://localhost:8000/predict
Content-Type: application/json

Request Body:

{
  "features": {
    "pdfsize": 12345,
    "metadata size": 200,
    "pages": 3,
    "isEncrypted": 0,
    "JS": 1,
    ...
  }
}

Response:

{
  "prediction": "malicious",
  "confidence": 0.998,
  "features": { ... }
}

---

⚡ Performance Optimization

✔ Recommended fast local models:

• gemma3:latest (recommended)
• phi3:3b
• qwen2.5:3b

✔ Enable GPU for Ollama:

# Linux
export OLLAMA_USE_CUDA=1

# Windows
set OLLAMA_USE_CUDA=1

✔ Reduce extracted text to 300–400 characters

✔ Use shorter, focused prompts

⚡ Result:

• ❌ Before: ~2 minutes
• ✅ After: 6–15 seconds

---

🛠 Troubleshooting

❌ Slow LLM inference (> 60s)

➡ Switch to smaller models + trim prompt
➡ Enable CUDA / GPU acceleration

❌ Dark-mode unreadable text in Streamlit

➡ HTML is wrapped with white background container automatically

❌ Pods stuck in CrashLoopBackOff

➡ Check logs: kubectl logs -l app=<service-name>
➡ Verify ConfigMap values are correct: kubectl describe configmap <name>

❌ AIService cannot reach Ollama

➡ Ensure ngrok is running: ngrok http 11434
➡ Update OLLAMA_BASE_URL in AIService/deploy/configmap.yml
➡ Re-apply: kubectl apply -f AIService/deploy/configmap.yml
➡ Restart pods: kubectl rollout restart deployment/ai-service

❌ Frontend cannot reach Backend

➡ Verify Backend service exists: kubectl get svc backend
➡ Check Backend pods are running: kubectl get pods -l app=backend

❌ Ingress not working

➡ Enable Ingress addon: minikube addons enable ingress
➡ Check Ingress controller: kubectl get pods -n ingress-nginx
➡ Verify /etc/hosts has the Minikube IP mapping

---

🏁 Conclusion

This system is:

✔ Fully modular microservices architecture
✔ Fast and optimized with GPU-accelerated LLM
✔ Local and privacy-preserving (no cloud API calls)
✔ Produces professional HTML malware analysis reports
✔ Multi-environment support (local, Docker Compose, Kubernetes)
✔ Production-ready with HPA auto-scaling, Ingress, and ConfigMap-based configuration
✔ Scalable for enterprise cybersecurity workflows

---