Few-Shot Symbol Classification via Self-Supervised Learning and Nearest Neighbor

Full text available here.

About • How To Use • Citations • Acknowledgments • License

About

We propose a three-stage process for unlabelled self-supervised learning (SSL) symbol classification:

1. Extraction of isolated symbols from unlabelled documents. Symbols are automatically extracted from unlabelled documents using a sliding-window approach. The documents are divided into patches, which are then converted to grayscale and binarized. The entropy value of each patch is calculated, and patches with an entropy value greater than a user-defined threshold are considered potential symbols.

2. Training of a neural feature extractor using SSL. A CNN is trained using the Variance-Invariance-Covariance Regularization SSL method.

3. Symbol classification using a k-nearest neighbours (kNN) classifier. Firstly, a query and a labelled set of symbol images are mapped to the representation space defined by the CNN obtained in Stage 2. Secondly, the kNN rule is applied to classify the query based on the labels of its k closest neighbours.

How To Use

The datasets used in this work, namely Capitan, TKH, Egyptian, and GRPOLY-DB, are available upon request. After obtaining these datasets, please place them in the datasets folder.

To run the code, you'll need to meet certain requirements which are specified in the Dockerfile. Alternatively, you can set up a virtual environment if preferred.

Once you have prepared your environment (either a Docker container or a virtual environment), you are ready to begin. Execute the experiments/run.py script to replicate the experiments from our work:

python experiments/run.py

Citations

@article{alfaro2023few,
  title     = {{Few-Shot Symbol Classification via Self-Supervised Learning and Nearest Neighbor}},
  author    = {Alfaro-Contreras, Mar{\'\i}a and R{\'\i}os-Vila, Antonio and Valero-Mas, Jose J and Calvo-Zaragoza, Jorge},
  journal   = {{Pattern Recognition Letters}},
  volume    = {167},
  pages     = {1--8},
  year      = {2023},
  publisher = {Elsevier},
  doi       = {10.1016/j.patrec.2023.01.014},
}

@inproceedings{rios2022few,
  title     =   {{Few-Shot Music Symbol Classification via Self-Supervised Learning and Nearest Neighbor}},
  author    =   {R{\'\i}os-Vila, Antonio and Alfaro-Contreras, Mar{\'\i}a and Valero-Mas, Jose J and Calvo-Zaragoza, Jorge},
  booktitle =   {{Proceedings of the 3rd International Workshop Pattern Recognition for Cultural Heritage}},
  pages     =   {93--107},
  year      =   {2022},
  publisher =   {Springer},
  address   =   {Montréal, Canada},
  month     =   aug,
  doi       =   {10.1007/978-3-031-37731-0_8},
}

Acknowledgments

This work is part of the I+D+i PID2020-118447RA-I00 (MultiScore) project, funded by MCIN/AEI/10.13039/501100011033. Computational resources were provided by the Valencian Government and FEDER funding through IDIFEDER/2020/003.

License

This work is under a MIT license.