This is a benchmark tool to test the performance of different barcode reading SDKs. It is designed to test a dataset of barcode images with ground truth. It uses a B/S architecture with Python Flask as its backend.
SDKs included:
- Dynamsoft Barcode Reader
- Scandit
- Zxing-cpp
- Zbar
- EAN-13 Reader
- OpenCV 1D Reader
- ZXing
- OpenCV Wechat QR Detector
- Boofcv
- Accusoft BarcodeXpress
- Aspose.Barcode
- Google ML Kit
- Apple Vision Framework
There is also a JavaScript version which runs purely in the browser: https://github.com/tony-xlh/barcode-reading-benchmark
pip install -r requirements.txt
Optional:
-
Integration using ZeroMQ
Some barcode reading SDKs don't provide a Python library. For such cases, we can create a command line tool using the provided SDKs and communicate with the server with ZeroMQ.
This is how Scandit, Accusoft and ZXing are integrated. You can see the example here: https://github.com/xulihang/BarcodeReader_CommandLine/
You have to put the commandline executive files under the
commandlinefolder. -
Integration using HTTP
For mobile SDKs, we can run an HTTP server on the phone and make HTTP requests for decoding. The app's source code: https://github.com/xulihang/Mobile-Barcode-SDK-Aggregator.
The app can run on a simulator/emulator for a better performance.
-
Start the server.
flask runor:
python app.py -
Visit http://127.0.0.1:5000/. Specify the path of the dataset's folder and create a session.
The folder should have the images and the txt files which contain the ground truth in JSON format.
An example of the ground truth file:
[{"attrib": {"Type": "EAN13"}, "text": "9785699128013", "value_attrib": {}}]
Every session has a unique ID. We can check them out later with its ID. Previous sessions will be listed on the homepage.
You can also assign a name to a session.
-
You will be directed to a session page where you can start decoding with a specified SDK and see the results.
There are several metrics: accuracy, precision, elapsed time, average time, etc.
You can also click the image's link to check out the image. Detected barcodes will be marked out in green polygons.
PS: If the ground truth file does not exist and the SDK detected barcodes, the detected one will be considered as correct. If the ground truth is provided, the program will examine whether the ground truth exists in the barcode decoding results.
-
You can compare the results in the comparison page.
You can list images detected/undetected by some SDK and filter the results. It can also draw a comparison chart.
There are some existing public datasets available. Scripts to generate ground truth files are provided in the utils folder of this repo.