Deep Learning based-framework for Math Formulas Understanding
Kacem Echi, Afef (University of Tunis (Tunísia))
Ben Aïcha, Takwa (University of Tunis (Tunísia))
Khazri Ayeb, Kawther (University of Tunis (Tunísia))

Date:	2024
Abstract:	Extracting mathematical formulas from images of scientific documents and converting them into structured data for storage in a database is essential for their further use. However, recognizing and extracting math formulas automatically, rapidly, and effectively can be challenging. To handle this problem, we have proposed a system, with a deep learning architecture, that uses the formula combination features to train the YOLOv8 model. This system can detect and classify the formula inside and outside the text. Once extracted, we built a robust end-to-end math formula recognition system that automatically identifies and classifies math symbols, using the faster R-CNN object detection, then a Convolution Graphical Neural network (ConvGNN) to analyze the math formula layout, as the formula is better represented as a graph with complex relationships and object interdependency. ConvGNN can predict formula linkages without resorting to laborious feature engineering. Experimental results on the IBEM and CROHME 2019 datasets reveal that the proposed approach can accurately extract isolated formulas with mAP of 99. 3\%, embedded formulas with mAP of 80. 3%, detect symbols with mAP of 87. 3%, and analyze formula layout with an accuracy of 92%. We also showed that our system is competitive with related work.
Rights:	Aquest document està subjecte a una llicència d'ús Creative Commons. Es permet la reproducció total o parcial, la distribució, i la comunicació pública de l'obra, sempre que no sigui amb finalitats comercials, i sempre que es reconegui l'autoria de l'obra original. No es permet la creació d'obres derivades.
Language:	Anglès
Document:	Article ; recerca ; Versió publicada
Subject:	Formula extraction ; Formula recognition ; Symbol detection ; Symbol classification ; Formula layout analysis ; YOLOv8 ; ConvGNN ; Faster R-CNN
Published in:	ELCVIA. Electronic letters on computer vision and image analysis, Vol. 23 Núm. 2 (2024) (Regular Issue) , ISSN 1577-5097

Adreça original: https://elcvia.cvc.uab.cat/article/view/1833
Adreça alternativa: https://raco.cat/index.php/ELCVIA/article/view/980000001030
DOI: 10.5565/rev/elcvia.1833

28 p, 4.1 MB

The record appears in these collections:
Articles > Published articles > ELCVIA
Articles > Research articles