A new Brainer postprocessing function for the evaluation of classification models is implemented beginning with the current development release, which is called Graphical Confusion Matrix.

The disadvantage of the traditional Confusion Matrix is the fact that it is not a graphical method. For the comparison of several models all the numerical values have to be collected from very different confusion tables and compared individually. For this reason we were looking for a new, simpler and more intuitive display of the corresponsing information, with which

• classification performance of a model can be assessed within one single view, and
• the quality of several models can be compared within one clear graphical chart.

In our opinion a Graphical Confusion Matrix is the best solution for this. Beside the number values also circles are drawn, for which the radius correlates with the number of samples from the corresponding class. Circles are drawn for each combination of model targets and outputs. Training, validation and test data are plotted in blue, green, and red.

Good classification models are showing big circles in the diagonal and small ones off the diagonal. In the following plot a good model is shown:

In contrast, the following plot shows how a bad model may look like:

Here the circles off the diagonal are significantly bigger, indicating an increased number of wrong classifications.

Some classification models also need a balanced number of training samples within each class. One can check this by comparing the circle sizes within the diagonal, which should have similar sizes if the data is balanced.

A further advantage of the given plot is the possibility to compare several models with each other. In the shown variation of the plot, the circles for the sample numbers of different models are shown in different subplots for the training, validation, test and overall samples. Different models are indicated by colour and line style. When clicking at a circle, the information about the model is shown. The following plot shows an example:

Again the better models have bigger circles in the diagonal and smaller circles off the diagonal.

The old function of the classification scatter plot will not be maintaned anymore and will be removed in some of the next releases.

Conclusion: when working with classification problems a suitable evaluation method is necessary allowing the evaluation of a model at one single view. Hereby some problems with models can be identified quickly, like unbalanced weights, generalization and specialization, systematic errors, and so on. With the Graphical Confusion Matrix the performance of classification models can be evaluated and compared quickly at a glance.