Recent Changes - Search:


edit SideBar


Evaluation of Face datasets as a tool for Assessing the Performance of Face Recognition Methods.

Lior Shamir, IJCV, 79,3, 225-230, 2008.

The generation of test data is a time consuming process which was once restricted to large well funded laboratories. The introduction of standard datasets for testing algorithms was intended to improve the quality of algorithmic assessment making data available to all. However, this availability has led to a new trend in computer vision, the easy combination of down loaded code and datasets for a quick evaluation and rapid publication. Often the combination of methods used are not fully understood and no effort is made to account (characterise) performance. Instead, researchers concentrate on getting a high performance on an ROC curve rather that trying to understand the workings of the system.

This paper should serve as a warning. It appears that good results can often be obtained from these data even when the objects in question are entirely removed from the images. This is because computers can extract detailed information from processes which have nothing to do with the intended application. The author here suggests this may result in evaluation bias. We might interpret these problems as due to the thoughtless combination of black-boxes and an insufficient attempt to understand the real reasons for good and bad performance.

NAT 12/7/2013

Edit - History - Print - Recent Changes - Search
Page last modified on July 12, 2013, at 12:00 PM