Evaluation Protocol of Scene Text Detection Based on Character Number

Abstract:
The evaluation metrics of scene text detection algorithms are mostly based on the Intersection-over-Union (IoU), but such metrics do not take into account the characteristics of text and will have the following drawbacks: 1) The integrity of the characters is not considered; 2) Unable to solve the problem of different label granularity. 3) A reasonable score cannot be given to text lines with different character scales. Therefore, this paper proposes an evaluation protocol based on the number of characters. This method uses the ratio of the number of detected characters to the total number of characters to give a score. For the datasets without character-level annotation, we trained a character detection model to support the evaluation metric. The result proves that this method can evaluate the scene text detection algorithm more fairly.