Post Quantum Signature Benchmark
Report about PQ-signature benchmark on constrained devices
Targets
Hardware Targets
This benchmark target-devices are a total of four, consisting of two Raspberry Pis running Ubuntu, a consumer smartphone running a terminal emulator called Termux, and a microcontroller programmed with RiotOS.
Termux is a terminal emulator and Linux environment for Android, it provides a minimal base system with a package manager and provides no root access.
RiotOS is an open source operating system for microcontrollers. It provides features such as cross compiling and flashing tools. It has real time capabilities and easier access to random number generation, while remaining energy and memory efficient.
| Device Name | RAM | Chip | Chip-Architecture | OS |
|---|---|---|---|---|
| Raspberry Pi5 | 16GB | BCM2712 | ARM-64 | Ubuntu 25.10 |
| Samsung S10e | 6GB | Exynos 9820 | ARM-64 | Ubuntu 25.10 |
| Raspberry Pi3 B+ | 1GB | BCM2837B0 | ARM-64 | Ubuntu 25.10 |
| ESP-32 NodeMCU | 520KB | ESP32-WROOM 32 | 32-bit LX6 | RIOT 2026.07 |
Algorithm Targets
Results
ML-DSA
bla bla
Falcon
The two variants of Falcon stay consistent with each other. During key-pair generation Falcon-1024 takes roughly three times as long, while during both signing and verification Falcon-512 is close to twice as fast. This is best seen in the Raspberry Pi5’s signing time, with it being 0.304 ms for Falcon-512, and 0.608ms for Falcon-1024.
The two versions of Falcon, Falcon-512 and Falcon-1024 had by far the worst Key-pair generation time of all algorithms, with 37.7 ms and 103.4 ms on our Raspberry Pi 3 B+ respectively. They, however, boasted the best verification time, even consistently beating ECDSA. It also performed well in singing time and has by far the smallest signatures size out of all of the benchmarked quantum safe algorithms. A similar pattern is visible in Falcon’s standard deviation being very consistent in verification and signing, and very inconsistent in its key generation time. For these reasons, it can be concluded that Falcon would make for a fitting option for consumer applications.
There are some anomalies in the data that need to be discussed. Falcon performed noticeably worse on the Samsung S10e, especially when compared to any other device. While the verification performed mostly as expected, both key-pair time and especially signing time underperformed. The most likely cause of this is either the chip architecture of the Exynos 9820 or Termux. further testing would be required to say for certain.
SLH-DSA-SHA2
SLH-DSA-SHA2 s
SLH-DSA-small performed by far the worst out of all the benchmarked algorithms. While it beat Falcon in key-pair generation time, it was still several times slower than all other algorithms. It managed to beat its fast-signing counterpart in verification times but still falls well short of ML-DSA or Falcon. By far the worst outcome is the signing time, where SLH-DSA-192s was consistently more than 20 times slower than its fast counterpart who was itself up to over 100 times slower than ML-DSA and Falcon.
The supposed benefit it has over its fast counterpart of smaller signature sizes, put SLH-DSA-128s still well in last place with 7856 Bytes. This more than anything puts SLH-DSA out of question for any consumer application. At seven times the signature size relative to its message, SLH-DSA is not just unpractical, but also uneconomical.
Once again there is some nuance to the data. While SLH-DSA-192s was noticeably slower than the 128 variant, the 256 variant consistently outperformed 192 in signing and even outperformed 128 in key generation. This result is consistent across all our devices, which ruled out many possible causes. Additionally, using different message sizes was tried to no change in the outcome. Since SLH-DSA was recently implemented and has many variants, it is assumed this problem most likely originated within the implementation by liboqs.
SLH-DSA-SHA2 f
SLH-DSA-fast while performing much better than its small counterpart in most regards, still underperformed greatly. It fails to compete with Falcon and ML-DSA in verification and signing. Together with its signature size, which is double the size of SLH-DSA-small, it brings little to the table that would make it an attractive option.
The signing process is by far the most consistent and follows a rough pattern. A 1.6x jump from 128 to 192 and a doubling from 192 to 256. This is also the area where it outperformed the small version the most, saving an average of 631ms per iteration on the Raspberry Pi5.
There are a lot of nuances in the data that lead to more questions than answers. For one, the same pattern that was missing in the small version’s verification results, where 256 outperforms 192, is once again present in the fast variants verification results, providing more evidence to the hypothesis that there are some issues in liboqs’ implementation.
Furthermore, in key-pair generation 192 performs much better than expected, almost reaching parity with 128 on the Samsung S10e. In all likelihood SLH-DSA-fast suffers from the same code problems as its small counterpart, it would be best to run this experiment again if and when SLH-DSA is more standardized and further developed.
Results Table
The following is the table of results from this benchmark, shortened down for this Wiki-entry. The full version can be found on the GitLab.
Set-Up Guide
Linux Systems
da guide here
Microcontrollers
da guide also here