Visualization
- Interactive scatter plot
Flip through features, analyze multiple sample groupings (labels, object id, material types,...) or define new sample groupings
- Interactive label painting
Define new classes, name plausible clusters and identify outliers by painting in scatter plots
- Visualizing algorithm decisions
Enhance the scatter plots with backdrop of the algorithm decisions or soft outputs in 2D feature spaces
ROC Analysis
- Two-class and multi-class
In adition to standard two-class ROC analysis, optimize classifier operating points in multi-class situations.
- Thresholding-based
Allows for designing detectors (distance-based rejection, one-class classification)
- Weighting-based
For two-class or multi-class discrimination.
- Standard performance measures
Class errors, TPr, FPr, TNr, FNr, precision, specificity, sensitivity, positive fraction, any field of confusion matrix.
- Custom performance measures
Define custom performance measures based on confusion matrix.
- Access to full confusion matrices
Get the raw data out and perform customized cost-based optimization or classification with constraints.
- ROC variance estimation
Variance information allows for testing of statistical significance regarding performance differences between different operating points on the same ROC. General operating point averaging approach is adopted applicable to two-class, multi-class, arbitrary operating point definition and arbitrary ROC optimizer.
- Interactive ROC plot
Inspect and select operating points interactively in ROC spaces. Flip between multiple performance measures.
Evaluation
- Rotation or randomization
Use stratified N-fold rotation or randomization cross-validation to estimate performance of your algorithms.
- Random seed for repeatability
Fix the random seed so that the experiments are repeatable.
- Support for replicas
Support for cross-validation keeping multiple replicas (measurements) of one sample either in the training set or in the test set.
- Multiple operating points
Run cross-validation at a set of operating points i.e. estimate ROC with variances at apriori fixed operating points.
- Stacked generalization
Estimate unbiased classifier outputs using cross-validation with one command. Practical technique for definition of operating points in ROC analysis or training multi-stage systems such as trainable combiners.
- Access to per-fold data
Inspect the training and test sets for a specific fold and per-fold trained algorithms.
- Decision-only algorithms
Cross-validate complex systems such as detector-classifier cascades where each stage was already ROC optimized and the whole system only returns crisp decisions.