AfriCompute

AfriCompute is a research prototype that simulates a federated shared AI-compute infrastructure and evaluates scheduling policies for it. It is not a real cloud platform, and no real institutions are currently federated -- every node, job, and workload in this repository is synthetic.

Who this is for

• AI/ML engineers -- a worked example of evaluating a scheduling policy the same way you'd evaluate a model: multiple seeds, multiple scenarios, multiple metrics, honestly-reported trade-offs.
• Systems engineers -- a small, readable discrete-time simulation engine (src/simulator.py) plus four pluggable scheduling policies behind one interface (BaseScheduler.select_jobs).
• Researchers / reviewers -- reproducible experiments (fixed seeds, deterministic tie-breaking, deep-copied workloads per comparison) with results traceable line-by-line to results/experiment_summary.csv.
• Recruiters -- a self-contained, tested (16 passing unit tests), documented project demonstrating simulation design, experiment methodology, and honest reporting of a novel-but-modest research contribution.

What the simulation represents

The broader vision this prototype explores: multiple institutions (universities, research centres, innovation hubs) retain ownership of their GPU infrastructure but can make unused capacity available to a federation. A scheduler coordinates the federation's available resources against submitted workloads. Institutions are heterogeneous (different total GPU capacity), demand fluctuates and can exceed supply, some institutions historically consume more compute than others, and node owners can reclaim or withdraw their infrastructure at any time.

The research question is: how should a federation scheduler balance efficiency and fairness under these conditions, and how should that balance change as resources become scarce?

AFAS: Adaptive Fair Allocation Scheduler

AFAS is this project's scheduling-policy contribution. Unlike a fixed policy, AFAS shifts its own priority weights based on how scarce GPU capacity currently is:

• Low scarcity -- favors efficient allocation and responsiveness (many small jobs get through quickly).
• High scarcity -- favors fairness: long-waiting jobs and under-served institutions are pushed to the front.

Every job's priority score is a weighted sum of four normalized (0-1) factors -- waiting time, fairness need (relative to historical usage), urgency, and an efficiency factor -- and the weights themselves move smoothly between a documented "low scarcity" vector and a documented "high scarcity" vector as measured scarcity increases. The full methodology (scarcity measure, each factor's normalization, the exact weight vectors, and the determinism guarantee) is documented in the AFASScheduler docstring in src/schedulers.py -- that docstring is the canonical methodology reference, not a summary of it.

Novelty disclaimer: adaptively weighting fairness against efficiency based on scarcity is a known idea in the scheduling literature in general. The contribution here is the specific, documented policy and its evaluation in this simulated, heterogeneous, federated environment -- not a claim that scarcity-adaptive weighting is itself a new idea, and not a claim that AFAS is a universally superior scheduler (see Results and Limitations below).

Architecture

At a high level, one simulation run flows as:

Workload generation
        ↓
Shared compute federation
        ↓
Scheduler
  ├── FCFS
  ├── Priority
  ├── FairShare
  └── AFAS
        ↓
Simulation
        ↓
Metrics
        ↓
Results / plots

The module layout implementing that flow:

src/
    models.py            ComputeNode, Job, RunningJob, InfrastructureEvent
    workload_generator.py Synthetic, seeded job generation
    schedulers.py        FCFS, Priority, FairShare, and AFAS
    simulator.py          Discrete time-step simulation engine
    metrics.py            Completion/waiting/fairness/starvation metrics
    federation.py         The 4-node simulated federation
    scenarios.py          The 4 reproducible experiment scenarios
    experiment_runner.py  Runs every scheduler x scenario x seed, writes CSVs
    visualization.py      Renders charts from the CSVs

tests/                    pytest unit tests
results/                  experiment_results.csv, institution_results.csv,
                          experiment_summary.csv, plots/
main.py                   Runs the full pipeline end to end

The simulator advances in discrete time steps. At each step it: applies any scheduled infrastructure event, releases GPUs from jobs that just finished, admits newly-submitted jobs into the waiting queue, asks the active scheduler to rank the waiting queue, allocates GPUs to as many ranked jobs as fit (one node per job -- see Limitations), and records utilization for that step.

The federation

Four simulated nodes, heterogeneous by design:

Node id	Name	GPUs
UNI_A	Large University	12
UNI_B	Medium University	6
HUB_C	Innovation Hub	4
RES_D	Research Centre	8

Total theoretical capacity: 30 GPUs.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Running the experiments

python main.py

This runs all 4 schedulers (FCFS, Priority, FairShare, AFAS) against all 4 scenarios across 5 fixed seeds (42-46) -- 4 x 4 x 5 = 80 simulation runs in total, taking well under a minute on a standard laptop -- then writes CSV results and PNG charts.

Run just the test suite:

pytest

Reproducing results

Every run is seeded: WorkloadGenerator(seed=...) is the only source of randomness, and the same generated workload (deep-copied so no scheduler can mutate another's view of it) is used for every scheduler within a (scenario, seed) pair. Infrastructure events are fixed, hand-specified configuration, not randomly injected. Re-running python main.py reproduces the CSVs exactly.

Experimental scenarios

All four are defined in src/scenarios.py, with the sizing rationale documented inline.

1. Normal demand -- concurrent GPU demand (~24) sits just under the federation's 30-GPU capacity. Baseline behaviour with resources relatively available.
2. High demand -- 250 jobs arrive within a 20-unit window (the original baseline experiment). Concurrent demand (~120) is roughly 4x capacity, producing substantial but fully-resolvable queueing.
3. Extreme scarcity -- 400 larger jobs arrive within a 15-unit window; concurrent demand (~335) is over 10x capacity. Evaluated within a bounded 120-step horizon specifically so that completion_rate stays below 1.0 and starvation is actually observable (see the docstring in scenarios.py for why a longer horizon would trivially hide this effect).
4. Dynamic infrastructure -- a high-demand-like workload with a fixed sequence of infrastructure events: at t=30 Medium University reclaims half its GPUs (6->3), at t=50 Innovation Hub goes fully offline, at t=70 Innovation Hub recovers.

Metrics

Computed in src/metrics.py from the simulator's final state:

• Completion rate -- completed / submitted jobs.
• Average / max / p95 waiting time -- over every job that managed to start (running or completed), not only completed jobs, so a job stuck in queue until the very end of the simulation still counts.
• Average GPU utilization -- averaged per-time-step utilization, where each step's denominator is the capacity of currently-online nodes only (an offline node's GPUs are not "idle capacity", they are not part of the federation at that moment).
• Starvation rate -- fraction of jobs whose wait (elapsed wait, if a job never started by the end of the run) exceeds a starvation_threshold. This threshold is scenario-dependent and configurable (see Scenario.starvation_threshold); it is chosen relative to each scenario's submission window and typical job duration, not a universal constant.
• Institutional GPU-hours -- cumulative GPU-hours actually consumed per institution.
• Fairness index -- Jain's Fairness Index. Reported two ways:
  • fairness_index (primary, demand-aware): Jain's index computed over each institution's served ratio -- allocated GPU-hours divided by that institution's own submitted GPU-hours demand. Equal absolute GPU-hours is not fair if institutions submitted very different amounts of demand, so this measures how evenly the federation satisfies each institution's own demand.
  • fairness_index_raw_usage: the classic Jain's index over raw GPU-hours per institution, reported for comparison with prior scheduling literature, but should be read with the caveat above.
• Waiting time by institution -- per-institution mean wait, in results/institution_results.csv.

Results

Results in this section are taken directly from results/experiment_summary.csv (mean across seeds 42-46); see that file and results/experiment_results.csv for the full per-seed data, and results/plots/ for the corresponding charts.

Scenario	Scheduler	Completion	Avg wait	Utilization	Fairness (demand-aware)
normal_demand	FCFS	1.00	1.25	0.215	1.000
normal_demand	AFAS	1.00	1.21	0.215	1.000
high_demand	FCFS	1.00	27.67	0.265	1.000
high_demand	Priority	1.00	24.10	0.265	1.000
high_demand	FairShare	1.00	24.02	0.265	1.000
high_demand	AFAS	1.00	23.55	0.265	1.000
extreme_scarcity	FCFS	0.728	44.69	0.980	0.987
extreme_scarcity	Priority	0.707	37.35	0.968	0.986
extreme_scarcity	FairShare	0.746	41.02	0.968	0.982
extreme_scarcity	AFAS	0.765	38.06	0.959	0.983
dynamic_infrastructure	FCFS	1.00	29.35	0.301	1.000
dynamic_infrastructure	Priority	1.00	23.38	0.301	1.000
dynamic_infrastructure	FairShare	1.00	23.53	0.301	1.000
dynamic_infrastructure	AFAS	1.00	24.79	0.301	1.000

Plots

Rendered by src/visualization.py from the CSVs above, under results/plots/:

• all_scenarios_overview.png -- all four scenarios side by side.
• completion_rate_by_scheduler.png
• average_waiting_time_by_scheduler.png
• gpu_utilization_by_scheduler.png
• fairness_index_by_scheduler.png
• waiting_time_by_institution.png -- referenced directly in the extreme-scarcity discussion below.

Interpretation (honest, not cherry-picked)

• Under normal and high demand, everyone eventually finishes. All four schedulers reach completion_rate 1.0, so the demand-aware fairness index is trivially 1.0 for all of them -- with a long enough horizon and no persistent overload, every institution's full demand is eventually served regardless of ordering policy. The differences that do show up here are in average waiting time: FairShare and AFAS reduce average wait versus FCFS by roughly 12-15% in the high-demand scenario, and static Priority (which ignores fairness entirely) is close behind them, occasionally ahead on raw average wait -- ordering by urgency alone is already a reasonable heuristic when everyone eventually gets served.
• Extreme scarcity is where the schedulers actually diverge. Within the bounded 120-step horizon, AFAS achieves the highest completion rate (76.5% vs. FCFS's 72.8%) and a materially lower average wait than FCFS and FairShare, while remaining close to Priority on wait. It does not, however, post the best demand-aware fairness index in this scenario -- FCFS's is marginally higher (0.987 vs. AFAS's 0.983). This is a genuine, reported trade-off, not a shortcoming to hide: AFAS is tuned to reduce starvation and improve throughput of long-waiting/under-served jobs under scarcity, and in this run that shows up more clearly in completion rate and waiting time than in the aggregate fairness index. The institution-level chart (results/plots/waiting_time_by_institution.png) shows AFAS compresses the spread of per-institution average waits compared to FCFS in the extreme-scarcity panel.
• Dynamic infrastructure: all schedulers still fully complete the workload (this scenario uses a high-demand-like, not extreme-scarcity-like, arrival profile), so fairness is again trivially 1.0. AFAS and FCFS show the largest and smallest average waits respectively swap compared to the high-demand scenario -- FCFS is hit hardest by the capacity reduction/offline period (29.35 avg wait, up from 27.67 in plain high-demand), while FairShare and Priority are least affected here. This suggests AFAS's efficiency-weighted behaviour under lower scarcity is not automatically an advantage when node capacity fluctuates -- a candidate area for future tuning, not a result to paper over.
• Utilization is essentially identical across schedulers within each scenario (as expected: total throughput is capped by total submitted demand and total available GPU-time, not by ordering policy, whenever completion approaches 1.0).

Bottom line: AFAS does not uniformly dominate the three baselines on every metric in every scenario. It shows a clear, consistent advantage in reducing average waiting time under sustained queueing (high demand) and in raising completion rate specifically under severe, bounded-horizon scarcity -- the exact condition it was designed for -- while occasionally trading a small amount of aggregate fairness-index score for that improvement, and it is not the best performer during the dynamic-infrastructure scenario tested here. These are preliminary findings from one workload family and five seeds, not a definitive verdict on scheduling policy for real federated infrastructure.

Limitations

• This is a simulation. No real African universities, research centres, or innovation hubs are currently connected to any system described here.
• All workloads are synthetic, generated by a fixed statistical model (src/workload_generator.py); results depend on that model's assumptions (job-size distributions, arrival patterns, institution mix) and may not transfer to real workload traces.
• Jobs must fit on a single node. AfriCompute does not split a job's GPUs across institutions. This is a deliberate simplification, but it means a job can be stranded even when the federation's aggregate free capacity would be sufficient (e.g. two nodes with 5 free GPUs each cannot serve an 8-GPU job). Node selection among nodes that do fit uses a worst-fit heuristic (most-free-GPUs first), applied identically for every scheduler so it cannot bias the comparison -- but it is not necessarily the fragmentation-optimal choice.
• Historical usage never decays. Cumulative GPU-hours are tracked for the entire simulation with no time-windowing or decay, so an institution's very first jobs affect its scheduling priority for the rest of the run. FairShareScheduler and AFAS both use this same undecayed measure, so the comparison between them is apples-to-apples, but a decayed/windowed variant is a natural extension.
• AFAS is a proposed policy, not a proven universal solution -- see the Results section above for where it does and does not lead on the measured metrics.
• Starvation threshold is a configuration choice, not derived from first principles; different thresholds would shift the reported starvation_rate values (though not the underlying waiting-time distributions they're computed from).