Skip to content

feat(akida2/portrait128): person segmentation example with trained models - #119

Open
mtorresBrainChip wants to merge 5 commits into
mainfrom
content/HUB-43-akida2-portrait128
Open

mtorresBrainChip wants to merge 5 commits into
mainfrom
content/HUB-43-akida2-portrait128

Conversation

@mtorresBrainChip

Copy link
Copy Markdown
Collaborator

Refs HUB-43

Adds an Akida 2 example for Portrait128 person segmentation, ported from
akida_models (akida_unet_portrait128): an AkidaNet 0.5 encoder with a
transposed-convolution decoder, the first Akida 2 example that upsamples.

Training run

  • Float: bash portrait128_train.sh, 50 epochs from ImageNet encoder weights,
    6 h 51 min on an RTX 2080 Ti
  • 8-bit: PTQ, then 1 epoch of QAT at 3e-6 (458 s on an RTX 2080 Ti)
  • Results (binary IoU / pixel accuracy, validation split):
    • float: 0.8883 / 94.31 %
    • 8-bit QAT: 0.9185 / 95.96 % in Keras, identical on Akida
  • The loss Keras reports during QAT (about 9.5) is dominated by the decoder's L2
    weight penalty as computed on the quantized layers; the cross-entropy itself is
    0.10 (float: 0.13).

FPGA benchmark (Akida 2, 25 MHz; projected at the 1 GHz AKD2500 target)

  • Minimal: 67 NPs (66 + input processor), 3 passes, 396.0 ms → 9.90 ms projected
  • AllNps: 69 NPs, 3 passes, 395.6 ms → 9.89 ms projected
  • HwPr: 187 NPs, 8 passes, 254.6 ms → 6.37 ms projected (fastest)
  • Mean activation sparsity 41 %; the encoder's first three convolutions take about
    half the latency. Benchmark notebook executed on the FPGA (about 35 min).

Decisions

  • 8-bit only: on Akida 2, DepthwiseConv2DTranspose needs 8-bit inputs and weights,
    so a 4-bit model falls back to software. BrainChip's model zoo also publishes this
    model at 8 bits only. Added to .claude/knowledge/akida.md.
  • The 8-bit model gets one epoch of QAT after PTQ, matching the model zoo's method.

Repo-wide changes (please review)

  • check_content.py: exempt Jupytext notebook mirrors from the copyright check
    (they must start with their YAML header). New scripts without a header are still flagged.
  • .gitignore: ignore wget-log* and nohup.out.

Open items

  • Training notebook "Run Time with training skipped" not yet measured.
  • Follow-ups: MobileNet backbone; ReLU between depthwise and pointwise layers;
    mixed-precision 4-bit (unconfirmed, see the knowledge base entry); more float epochs
    (QAT improved on the float model).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant