Skip to content

feat(akida2): ImageNet MobileNet V1/V2 eval-only example (HUB-19) - #114

Open
dmclellandbc wants to merge 8 commits into
mainfrom
content/HUB-19-akida2-imagenet-mobilenet
Open

dmclellandbc wants to merge 8 commits into
mainfrom
content/HUB-19-akida2-imagenet-mobilenet

Conversation

@dmclellandbc

Copy link
Copy Markdown
Collaborator

Adds akida2/model_zoo/imagenet_mobilenet, an eval-only Akida 2 example: the ImageNet-pretrained MobileNetV1 (alpha 0.25, 0.5, 0.75, 1.0) and MobileNetV2 (alpha 0.35, 0.5, 0.75, 1.0) models from tf_keras.applications, at 224 × 224, prepared for Akida 2, quantized to 8 bits with post-training quantization (PTQ) only, converted, evaluated on ImageNet and benchmarked on the Akida 2 FPGA.

Refs HUB-19

What's in it

  • imagenet_mobilenet_model.py: build_mobilenet(), the reference model definition. It adds a Rescaling layer up front (uint8 input) and removes the softmax. MobileNetV2 keeps its GAP → Dense head; MobileNetV1's 1 × 1 conv classifier becomes the equivalent Dense layer (same weights), and its Dropout is kept, since conversion removes it. The docstring explains each choice. Every rebuilt model reproduces the original Keras logits (max abs difference ~2e-5).
  • Data module with PTQ calibration samples drawn from the ImageNet train split (1,024 images), eval and FPGA benchmark scripts, a reproduction script (imagenet_mobilenet_eval.sh), two notebooks, and a summary plot.
  • 16 committed models (LFS): the quantized .h5 and converted .fbz for each of the 8 models. The float models are rebuilt from the Keras weights.
  • Akida 2 landing table: the five models that fit on the FPGA. Root README: link to the example.
  • Knowledge base: every pass holds at least two layers and a layer is never split across passes; models that don't fit in one sequence can run as several (hw_only=False); on the 6-node FPGA, the 3 FNP-capable NPs double as CNPs.

Results

Accuracy on the full 50,000-image validation set (top-1), from docs/metrics.json:

Model Float 8-bit PTQ Akida
V1 0.25 51.58% 47.89% 47.89%
V1 0.5 64.42% 62.81% 62.77%
V1 0.75 68.42% 67.47% 67.47%
V1 1.0 70.62% 69.25% 69.23%
V2 0.35 60.98% 58.98% 58.98%
V2 0.5 65.74% 63.55% 63.55%
V2 0.75 70.08% 68.07% 68.08%
V2 1.0 71.74% 70.81% 70.83%

Conversion is lossless (within 0.04 points); PTQ costs 0.9–2.2 points, and 3.7 for V1 0.25.

Mapping on the 6-node FPGA: V1 0.25 / 0.5 / 0.75 and V2 0.35 / 0.5 fit in a single hardware sequence and are benchmarked in all three modes (HwPr fastest on each). V1 1.0 (conv_pw_13, 22 NPs, leaving no FNP for the classifier) and V2 0.75 / 1.0 (block_1_depthwise, 21 NPs) don't fit in one sequence; they fit in one on a 12-node device, and run on the FPGA as 2–3 sequences with hw_only=False, with outputs identical to the software backend. They have accuracy figures but no latency.

How the numbers were produced

  • Accuracy: imagenet_mobilenet_eval.py -v <v> -a <a> --variant {float,quantized,akida} --save-metrics for all 24 combinations, on an RTX 3090 (Keras) and the Akida software backend (CPU). The models were produced by bash imagenet_mobilenet_eval.sh <v> <a> (about 6 min for V1 0.25 alone).
  • Hardware: imagenet_mobilenet_benchmark.py -v <v> -a <a> --save-metrics on the Akida 2 FPGA (6 nodes, 25 MHz), akida 2.19.3, 100 inferences of the 10-image sample pack; 13–26 min per model.
  • Both notebooks executed with python -m brainchip_utils.notebooks: evaluation on the full validation set (9.5 min), benchmark on the FPGA (25.5 min).
  • check_content.py: no findings in this change.

Not in this PR

  • No per-layer benchmark for MobileNetV2: per_layer_benchmark cuts the model after each layer, and a cut inside a residual block can't be mapped. The script and notebook skip it for MobileNetV2 and say why.
  • No latency for the three multi-sequence models: the benchmark helper maps in a single sequence only.
  • No QAT: 8-bit PTQ only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN

dmclellandbc and others added 8 commits October 20, 2026 12:03
…ample scripts

Model definitions (Keras ImageNet MobileNets with in-model rescaling, no
softmax, dense classifier), data loading with train-split PTQ calibration
samples, evaluation, FPGA benchmark and the reproduction script. Models,
metrics, README and notebooks to follow.

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
…FPGA and why

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
…e base: passes hold at least two layers

The quantized (.h5) and converted (.fbz) models for MobileNetV1 alpha 0.25,
0.5, 0.75, 1.0 and MobileNetV2 alpha 0.35, 0.5, 0.75, 1.0, produced by
imagenet_mobilenet_eval.sh.

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
… summary plot and README generator

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
…-layer step, skip it for residual models

Per-layer benchmarking cuts the model after each layer, which can't be mapped
inside a MobileNetV2 residual block. Also avoid tf.function retracing warnings
on the variable-size sample-pack images.

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
… notebooks

Accuracy for all eight models on the full ImageNet validation set, and FPGA
latency for the five that map onto it. README generated from the template; the
Akida 2 landing table lists the five FPGA models. The evaluation notebook is
executed; the benchmark notebook follows.

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
…sequence wording; root README link

The three larger models don't fit on the FPGA in a single hardware sequence,
which the benchmark needs, but they do run on it as two or three sequences
(hw_only=False), with outputs matching the software backend. Say so throughout,
and record it in the knowledge base. Link the example from the root README.

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TWHEyepfKmZYMDwveUd1AN
…esults first

- Model card: keep only the one-line description; bold the Akida top-1 column; the
  "Fits the FPGA" explanation becomes a table footnote.
- Drop "Reading the table"; the quantization paragraphs become a short 8-bit PTQ note in
  the Pipeline section.
- Benchmark: the summary figure comes first, and its bullet list is dropped.
- Note that these are the original Keras ImageNet weights, citing the improved training
  recipes (MobileNetV4 paper, timm baselines), with a planned timm/PyTorch example.
- Root README lists ImageNet / MobileNetV1 and MobileNetV2 separately.
- Knowledge base: "every pass holds at least two layers" also holds on AKD1500.

Refs HUB-19

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WXK4W9Y5CQgk79Cr2NaET7
Comment thread .claude/knowledge/akida.md Resolved
Comment thread README.md Resolved

@mtorresBrainChip mtorresBrainChip left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants