We have 10 years of academic experience in theoretical and applied pattern recognition. Because our close involvement with research continues, PR Sys Design provides state-of-the-art solutions to industrial problems.
ICPR 2008, accepted publication
Variance estimation for two-class and multi-class ROC analysis using operating point averaging.
P. Paclik, C. Lai, J. Novovicova, R.P.W.Duin.
To be presented at the 19th International Conference on Pattern Recognition, December 2008, Tampa, USA.
Slides from the SIMPLI'08 (PDF, 1.3MB) presentation in the ICT group at TU Delft.
Abstract:
Receiver Operating Characteristic (ROC) analysis enables fine-tuning of a trained classifier to a desired performance trade-off situation. ROC estimated from a finite test set is, however, insufficient for the sake of classifier comparison as it neglects performance variances. This research presents a practical algorithm for variance estimation at individual operating points of ROC curves or surfaces. It generalizes the threshold averaging of Fawcett et.al. to arbitrary operating point definition including the weighting-based formulation used in multi-class ROC analysis. The statistical test comparing performance differences between operating points of the same curve is illustrated for two-class and multi-class ROC.
Selected publications
- P.Paclik, J.Novovicova, R.P.W.Duin, Building road sign classifiers using trainable similarity measure , IEEE Transactions on Intelligent Transportation Systems, vol. 7, no. 3, pp. 309-321, September 2006.
- T.C.W.Landgrebe, D.M.J.Tax, P.Paclik, R.P.W.Duin, The interaction between classification and reject performance for distance-based reject-option classifiers , Pattern Recognition Latters, vol.27, pp.908-917, 2006
- P.Paclik, R.P.W.Duin, Dissimilarity-based classification of spectra: computational issues , Real Time Imaging, vol.9, num.4, pp.237-244, 2003
- P.Paclik, R.P.W.Duin, G.M.P.van Kempen, R.Kohlus, Segmentation of multi-spectral images using the combined classifier approach , Image and Vision Computing, vol. 21, num. 6, pp. 473-482, June 2003
- E.Pekalska, P.Paclik, R.P.W.Duin, A Generalized Kernel Approach to Dissimilarity Based Classification , Journal of Machine Learning Research, num.2, pp.175-211, 2001
- M.Svitek, J.Prchal, P.Paclik, M.Karny, Parameter Reduction Method for Transport Network Control , Neural Network World, vol.2, num.1, pp.17-26, 2001
- P.Paclik, J.Novovicova, P.Somol, P.Pudil, Road Sign Classification using Laplace Kernel Classifier , Pattern Recognition Letters, vol.21, num.13-14, pp.1165-1173, 2000
- P.Somol, J.Novovicova, P.Pudil, P.Paclik, Adaptive Floating Search Methods in Feature Selection , Pattern Recognition Letters, vol.21, num.11-12, pp.1157-1163, 1999
- T.C.W.Landgrebe, A.Bradley, P.Paclik, R.P.W.Duin, Precision-recall operating characteristic (P-ROC) curves in imprecise environments , proc.of ICPR'06 conference, Hong Kong, August 2006
- P.Paclik, S.Verzakov, R.P.W.Duin, Improving the maximum-likelihood co-occurrence classifier: a study on classification of inhomogeneous rock images , in proc.of SCIA05 conference, Joensuu, Finland, June 2005
- P.Paclik and R.P.W.Duin, Designing multi-modal classifiers of spectra: a study on industrial sorting application , in Proc. of the 2nd Spectral Imaging workshop in Villach, Austria, September 2005
- T.C.W.Landgrebe, P.Paclik, D.M.J.Tax, R.P.W.Duin, Optimising two-stage recognition systems , in proc.of Multiple Classifier Systems conference, 2005
- P.Paclik, D.M.J.Tax, S.Verzakov, R.P.W.Duin, Simplifying the model-based classifiers for multi-modal problems in classification of spectra , in proceedings of PRASA'05 conference (Langebaan, South Africa), November 2005