SmartDoc 2015 – Challenge 1 Results

We are very pround to announce the final results of the challenge 1.  Congratulations to all participants, and thank you for your kind interest in the competition!

FOR DETAILED RESULTS PLEASE SEE ICDAR 2015 PROCEEDINGS.

Results summary

Ranking	Method	Jaccard Index	Confidence Interval
1	LRDE	0.9716	[0.9710, 0.9721]
2	ISPL-CVML	0.9658	[0.9649, 0.9667]
3	SmartEngines	0.9548	[0.9533, 0.9562]
4	NetEase	0.8820	[0.8790, 0.8850]
5	A2iA run 2	0.8090	[0.8049, 0.8132]
6	A2iA run 1	0.7788	[0.7745, 0.7831]
7	RPPDI-UPE	0.7408	[0.7359, 0.7456]
7	SEECS-NUST	0.7393	[0.7353, 0.7432]

Methods overview
A2iA run 1C. Kermorvant, A. Semenov, S. Sashov and V. Anisimov from A2iA St. Petesburg and Paris and Teklia Paris, FranceTheir method starts with a Canny edge detector in the RGB space followed by an interpolation of the detected contours by Bezier curves. Some contours are selected based on their contrast and then quadrangles are selected depending on their squareness. If such steps fail to detect a valid quadrangle, a set of similar steps are applied to a denoised binary version of the input image.
A2iA run 2C. Kermorvant, A. Semenov, S. Sashov and V. Anisimov from A2iA St. Petesburg and Paris and Teklia Paris, FranceThe second method from A2iA is the same of the first run without the low contrast contour removal.
ISPL-CVMLS. Heo, H.I. Koo and N.I. Cho from Seoul National University and Ajou University, South KoreaTheir method starts by applying the Line Segment Dectector (LSD) presented in~\cite{Gioi10} to down-sampled images. Document boundaries are then generated by selecting two horizontal and vertical segments that minimize a cost function exploiting color and edge features. The final document boundaries are refined in the original high resolution image.
LRDEE. Carlinet and T. Géraud from EPITA Research and Development Laboratory, FranceTheir method relies on a hierarchical representation of the image named Tree of Shapes. In each frame of the video, an energy on the tree is computed in order to select the shape that looks the most like a papersheet. The energy involves two terms measuring how the shape fits a quadrilateral and if it has sub-contents like lines or images. Two Trees of Shapes are computed on the Lab components of the frame (converted in the Lab space). Shapes having the highest energies in both trees are retained as candidate objects and the location of the detection in the previous frame is used to finally select the right shape among the candidate components.
NetEaseP. Li, Y. Niw and X. Li from NetEase, ChinaTheir method starts by extracting line segments by the LSD method, such segments are then grouped and quadrangles are formed by selecting two horizontal and vertical segment groups. The final quadrangle is selected based on its aspect-ratio, area and inner angles.
RPPDI-UPEB.L.D. Bezerra, L. Leal and A. Junior from University of Pernambuco and Document Solutions, BrazilTheir method starts by using the HSV color space and filtering the hue channel in order to make the document pages stand from the background. Morphological operations followed by a Canny edge detector and a Hough transform yields a set of candidate polygons. Such polygons are then filtered according to their shape and position.

SEECS-NUSTS.A. Siddiqui, F. Asad, A.H. Khan and F. Shafait from School of Electrical Engineering and Computer Science and National University of Science and Technology, PakistanTheir method applies a Canny edge detection on the gray-level image to get a first estimate of the document position. A subsequent analysis of the different color channels is used to determine in which channel there is a higher contrast between document and background followed by a probabilistic Hough Transform to obtain the accurate document segmentation.
SmartEnginesA. Zhukovsky, D. Nikolaev, V. Arlazarov, V. Postnikov, D. Polevoy, N. Skoryukina, T. Chernov, J. Shemiakina, A. Mukovozov, I. Konovalenko and M. Povolotsky from Moscow Institute for Physics and Technologies, National University of Science and Technology, Institute for Systems Analysis, of Russian Academy of Sciences and Institute for Information Transmission Problems of Russian Academy of Sciences, RussiaTheir method starts with a segment extraction step by means of the LSD algorithm followed by a graph construction of segments. A quadrangle selection is done on such graph after applying several size and angle filters. The final candidate quadrangle is selected by fitting a motion model by using a Kalman filter powered by an inter-frame matching strategy of local descriptors based on SURF and BRIEF.