Project description
The main goal of this research project is to study, develop and validate algorithms and methods for the recognition of handwritten mathematical expressions. This is an important and challenging problem in the field of Pattern Recognition. The variety of symbols to be recognized, the variations in writing style, the need to analyze 2D spatial arrangement of the symbols, different notations, intrinsic ambiguities, among other issues make this a non trivial problem.
An efficient recognition system would be useful in several situations: it would simplify input of mathematical notation into computer systems, it would allow efficient digitalization of handwritten documents, it could assist visually impaired person to read mathematical notations, and so on.
In particular, with the advent of touch screen based devices, online recognition of handwriting from digital ink data is currently an active research topic.
This project has received support from CNPq and FAPESP through research project grants and scholarship grants.

Subprojects
Graph grammar based approach for math expression recognition: this is a current PhD project (by Frank D. JulcaAguilar), a project with cosupervision by Prof. Christian ViardGaudin and Prof. Harold Mouchere (Univ. of Nantes, France).
References:
 ICFHR 2014


ExpressMatch: is a system that helps creation of groundtruth annotated dataset of handwritten mathematical expressions. The main idea consists in building a set of model expressions in which important substrucutures are annotated with groundtruth data. Then, users are invited to transcribe those expressions, and an expression matching method is used to match each symbol in the transcribed expression to the respective symbol in the model expression. Once the matching is computed, all groundtruth data in the model expression can be automatically transferred to the transcribed expression. This method allows rapid generation of a large number os samples of groundtruth annotated handwritten math expresssions. Details of the matching method are described in the paper [3] below.
The matching method and the ExpressMatch system that implements the data collecting and annotation processes are described in the first two references below. Both the code and the generated dataset are avaliable at http://code.google.com/p/expressmatch/
References:
 Hirata, Nina S. T. ; Honda, W. Y. Automatic Labeling of Handwritten Mathematical Symbols via Expression Matching. In: 8th IAPRTC15 International Workshop on GraphBased Representations in Pattern Recognition. Lecture Notes in Computer Science  GraphBased Representations in Pattern Recognition, 2011. v. 6658. p. 295304.
 F. D. J. Aguilar and N. S. T. Hirata, ExpressMatch: A System for Creating GroundTruthed Datasets of Online Mathematical Expressions, 10th IAPR International Workshop on Document Analysis Systems (DAS 2012), pp.155159, 2012.
 N. S. T. Hirata and F. D. J. Aguilar, Matching based groundtruth annotation for online handwritten mathematical expressions Pattern Recognition, In press.

Brief development log
The activities related to this project started in 2006, initially as Graduation Projects or undergraduate level projects. Below is a brief description of main outcomes, in reverse cronological order.

People
 Nina S. T. Hirata (coordinator)
 Current: Frank (PhD candidate), Davi (MSc candidate)
 Former: Marcelo (MSc 2014), Breno and Ricardo; Ricardo Sider (bolsista CNPq/IC); Cristiano Perez Garcia (bolsista CNPq/ITI  2008); Fábio Hirano (exbolsista CNPq/IC); Ana Paula, Fábio Eiji, Leonardo Ka Wah, Eduardo Komatsu (equipe MathPicasso, 2007); Eduardo Gusmão Cáceres Pires, Pedro Henrique Simões de Oliveira, Ricky Ye Lun Chow (equipe SisTREO, 2006).

(Page created on 06/August/2007)
(Last update on 21/November/2014)
   