independent · scientific ML · efficient inference · edge AI · Android
EdgeGenBench is a reproducible aircraft-design ML benchmark connecting model accuracy, uncertainty, constrained optimization, numerical equivalence, quantization, runtime selection, native inference, and hardware-oriented deployment.
The benchmark uses 4,200 training, 900 validation, and 900 held-out test cases generated from a deterministic aircraft-design model. The compact 10→64→32→16→6 PyTorch surrogate reaches 0.996956 mean held-out R² and 0.050425 mean NRMSE, outperforming the strongest classical baseline while remaining only 3,414 trainable parameters.
The trained network is exported to ONNX and checked for numerical equivalence. FP16 reduces serialized size while preserving predictive quality, but is not automatically faster in the measured Core ML provider setup. A mixed INT8/FP32 configuration reduces size by 33.21% and improves the measured batch-256 CPU latency, while remaining slower at smaller batches.
The dynamic FP32 ONNX source is compiled into a QNN Context Binary with batch-1, batch-32, and batch-256 graphs for a Snapdragon 8 Elite QRD. All nine profiled layers are placed on the Hexagon v79 NPU. AI Hub profile latency is 38 μs, 34 μs, and 57 μs for batches 1, 32, and 256. On all 900 held-out rows, mean R² remains 0.996953 with maximum normalized deployment drift of 0.003636.
A Qualcomm-native INT8/QDQ candidate retained high predictive quality, but its maximum normalized deployment drift exceeded the frozen 0.01 limit. The fail-closed deployment policy rejected it instead of promoting the smaller model simply because it was quantized.
The repository now includes a C++17 inference boundary with tensor validation, explicit failure modes, CMake/CTest coverage, JSON latency output, and equivalent preprocessing paths. A Kotlin/JNI Android harness exercises that boundary and includes an opt-in Qualcomm QNN route that fails closed when the required runtime or model contract is unavailable.
Android work now goes beyond an APK build. The project records physical-device reference captures on a Samsung SM-A356E, retains repeated-run results, exports self-describing evidence with app version, Git revision, device/OS/ABI identity and runtime metadata, validates those exports, and binds release evidence with SHA-256 checksums. CI separately verifies the native benchmark, APK, and 16 KiB alignment boundary.
The Android reference captures are not presented as QNN/NPU measurements when they use the deterministic reference backend, and power remains unmeasured without a named measurement tool. AI Hub QNN profile measurements are kept distinct from end-to-end Android application latency. The browser app runs on iPhone Safari, but signed native-iPhone latency and energy measurements are still not claimed.