kaggle-metrics API

Classification

log_loss(y_true, y_pred) Logarithmic loss
mean_consequential_error(y_true, y_pred) Mean consequential error
hamming_loss(y_true, y_pred) Hamming loss
mean_utility(y_true, y_pred, weights) Mean utility
matthews_correlation_coefficient(y_true, y_pred) Matthews Correlation Coefficient
roc_auc(y_true, y_pred[, jump]) Area under ROC (Receiver Operating Characteristics) curve
gini(y_true, y_pred) Gini

Regression

root_mean_squared_error(y_true, y_pred) Root mean squared error.
root_mean_squared_logarithmic_error(y_true, …) Root mean squared logarithmic error.
mean_absolute_error(y_true, y_pred) Mean absolute error.
weighted_mean_absolute_error(y_true, y_pred, …) Weighted mean absolute error.
mean_absolute_percentage_error(y_true, y_pred) Mean absolute percentage error
mean_percentage_error(y_true, y_pred) Mean percentage error
mean_absolute_percentage_deviation(y_true, …) Mean absolute percentage error

Order-based

average_precision_at_k(true_positive) Average precision at position k
average_precision(true_positive) Average precision
mean_average_precision(true_positive) Mean average precision

Other

intersection_over_union(y_true, y_pred) Intersection over union
kaggle_metrics.log_loss(y_true, y_pred)[source]

Logarithmic loss

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class probability
Returns:

score – Logarithmic loss score
Return type:

float

References

[1]https://www.kaggle.com/wiki/LogLoss
[2]http://www.exegetic.biz/blog/2015/12/making-sense-logarithmic-loss/
[3]http://wiki.fast.ai/index.php/Log_Loss
kaggle_metrics.mean_consequential_error(y_true, y_pred)[source]

Mean consequential error

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class predictions (0 or 1 values only)
Returns:

score – Mean consequential error score
Return type:

float

References

[1]https://www.kaggle.com/wiki/MeanConsequentialError
[2]http://www.machinelearning.ru/wiki/images/5/59/PZAD2016_04_errors.pdf (RU)

Notes

The higher the better.

kaggle_metrics.hamming_loss(y_true, y_pred)[source]

Hamming loss

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class predictions (0 or 1 values only)
Returns:

score – Hamming loss score
Return type:

float

References

[1]https://www.kaggle.com/wiki/HammingLoss
[2]https://en.wikipedia.org/wiki/Multi-label_classification

Notes

The smaller the better.

kaggle_metrics.mean_utility(y_true, y_pred, weights)[source]

Mean utility

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class predictions (0 or 1 values only)
Returns:

score – Mean utility score
Return type:

float

References

[1]https://www.kaggle.com/wiki/MeanUtility
[2]https://en.wikipedia.org/wiki/Multi-label_classification

Notes

The higher the better.

kaggle_metrics.matthews_correlation_coefficient(y_true, y_pred)[source]

Matthews Correlation Coefficient

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class predictions (0 or 1 values only)
Returns:

score – Matthews Correlation Coefficient score
Return type:

float

References

[1]https://lettier.github.io/posts/2016-08-05-matthews-correlation-coefficient.html
[2]https://en.wikipedia.org/wiki/Matthews_correlation_coefficient
kaggle_metrics.roc_auc(y_true, y_pred, jump=0.01)[source]

Area under ROC (Receiver Operating Characteristics) curve

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class probability

References

[1]https://developers.google.com/machine-learning/crash-course/classification/roc-and-auc
Returns:roc_auc_score – ROC AUC score
Return type:float
kaggle_metrics.gini(y_true, y_pred)[source]

Gini

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class probability
Returns:

gini_score – Gini score
Return type:

float

References

[1]https://en.wikipedia.org/wiki/Receiver_operating_characteristic#Area_under_the_curve
[2]https://aichamp.wordpress.com/2017/10/19/calculating-auc-and-gini-model-metrics-for-logistic-classification/
kaggle_metrics.mce(y_true, y_pred)

Mean consequential error

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class predictions (0 or 1 values only)
Returns:

score – Mean consequential error score
Return type:

float

References

[1]https://www.kaggle.com/wiki/MeanConsequentialError
[2]http://www.machinelearning.ru/wiki/images/5/59/PZAD2016_04_errors.pdf (RU)

Notes

The higher the better.

kaggle_metrics.mcc(y_true, y_pred)

Matthews Correlation Coefficient

Parameters:
  • y_true (numpy.ndarray) – Targets
  • y_pred (numpy.ndarray) – Class predictions (0 or 1 values only)
Returns:

score – Matthews Correlation Coefficient score
Return type:

float

References

[1]https://lettier.github.io/posts/2016-08-05-matthews-correlation-coefficient.html
[2]https://en.wikipedia.org/wiki/Matthews_correlation_coefficient
kaggle_metrics.root_mean_squared_error(y_true, y_pred)[source]

Root mean squared error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Root mean squared error
Return type:

float

References

[1]https://www.kaggle.com/wiki/RootMeanSquaredError
kaggle_metrics.root_mean_squared_logarithmic_error(y_true, y_pred)[source]

Root mean squared logarithmic error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Root mean squared logarithmic error
Return type:

float

References

[1]https://www.kaggle.com/wiki/RootMeanSquaredLogarithmicError
[2]https://www.slideshare.net/KhorSoonHin/rmsle-cost-function
kaggle_metrics.mean_absolute_error(y_true, y_pred)[source]

Mean absolute error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Mean absolute error
Return type:

float

References

[1]https://www.kaggle.com/wiki/MeanAbsoluteError
kaggle_metrics.weighted_mean_absolute_error(y_true, y_pred, weights)[source]

Weighted mean absolute error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Weighted mean absolute error
Return type:

float

References

[1]https://www.kaggle.com/wiki/WeightedMeanAbsoluteError
kaggle_metrics.mean_absolute_percentage_error(y_true, y_pred)[source]

Mean absolute percentage error

Parameters:
  • y_true (ndarray) –
  • truth (Ground) –
  • y_pred (ndarray) –
  • of predictions (Array) –
Returns:

mean_absolute_percentage_error – Mean absolute percentage error
Return type:

float

References

[1]https://en.wikipedia.org/wiki/Mean_absolute_percentage_error
kaggle_metrics.mean_percentage_error(y_true, y_pred)[source]

Mean percentage error

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

mean_percentage_error – Mean percentage error
Return type:

float

References

[1]https://en.wikipedia.org/wiki/Mean_percentage_error
kaggle_metrics.mean_absolute_percentage_deviation(y_true, y_pred)

Mean absolute percentage error

Parameters:
  • y_true (ndarray) –
  • truth (Ground) –
  • y_pred (ndarray) –
  • of predictions (Array) –
Returns:

mean_absolute_percentage_error – Mean absolute percentage error
Return type:

float

References

[1]https://en.wikipedia.org/wiki/Mean_absolute_percentage_error
kaggle_metrics.rmse(y_true, y_pred)

Root mean squared error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Root mean squared error
Return type:

float

References

[1]https://www.kaggle.com/wiki/RootMeanSquaredError
kaggle_metrics.rmsle(y_true, y_pred)

Root mean squared logarithmic error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Root mean squared logarithmic error
Return type:

float

References

[1]https://www.kaggle.com/wiki/RootMeanSquaredLogarithmicError
[2]https://www.slideshare.net/KhorSoonHin/rmsle-cost-function
kaggle_metrics.mae(y_true, y_pred)

Mean absolute error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Mean absolute error
Return type:

float

References

[1]https://www.kaggle.com/wiki/MeanAbsoluteError
kaggle_metrics.wmae(y_true, y_pred, weights)

Weighted mean absolute error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Weighted mean absolute error
Return type:

float

References

[1]https://www.kaggle.com/wiki/WeightedMeanAbsoluteError
kaggle_metrics.mape(y_true, y_pred)

Mean absolute error.

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

rmsle – Mean absolute error
Return type:

float

References

[1]https://www.kaggle.com/wiki/MeanAbsoluteError
kaggle_metrics.mpe(y_true, y_pred)

Mean percentage error

Parameters:
  • y_true (ndarray) – Ground truth
  • y_pred (ndarray) – Array of predictions
Returns:

mean_percentage_error – Mean percentage error
Return type:

float

References

[1]https://en.wikipedia.org/wiki/Mean_percentage_error
kaggle_metrics.mapd(y_true, y_pred)

Mean absolute percentage error

Parameters:
  • y_true (ndarray) –
  • truth (Ground) –
  • y_pred (ndarray) –
  • of predictions (Array) –
Returns:

mean_absolute_percentage_error – Mean absolute percentage error
Return type:

float

References

[1]https://en.wikipedia.org/wiki/Mean_absolute_percentage_error
kaggle_metrics.average_precision_at_k(true_positive)[source]

Average precision at position k

Parameters:true_positive (numpy.ndarray) – True positive for ordered values in query
Returns:score – A vector of average precision score for every k-th point
Return type:numpy.ndarray

References

[1]https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52
[2]https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173
kaggle_metrics.average_precision(true_positive)[source]

Average precision

Parameters:true_positive (numpy.ndarray) – True positive for ordered values in query
Returns:score – A vector of average precision score
Return type:numpy.ndarray

References

[1]https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52
[2]https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173
kaggle_metrics.mean_average_precision(true_positive)[source]

Mean average precision

Parameters:true_positive (numpy.ndarray) – True positive values for n queries (n_queries, answers)
Returns:score – Mean average precision score
Return type:float

References

[1]https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52
[2]https://en.wikipedia.org/wiki/Information_retrieval#Mean_average_precision
[3]https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173
kaggle_metrics.ap(true_positive)

Average precision

Parameters:true_positive (numpy.ndarray) – True positive for ordered values in query
Returns:score – A vector of average precision score
Return type:numpy.ndarray

References

[1]https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52
[2]https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173
kaggle_metrics.ap_at_k(true_positive)

Average precision at position k

Parameters:true_positive (numpy.ndarray) – True positive for ordered values in query
Returns:score – A vector of average precision score for every k-th point
Return type:numpy.ndarray

References

[1]https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52
[2]https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173
kaggle_metrics.map(true_positive)

Mean average precision

Parameters:true_positive (numpy.ndarray) – True positive values for n queries (n_queries, answers)
Returns:score – Mean average precision score
Return type:float

References

[1]https://towardsdatascience.com/breaking-down-mean-average-precision-map-ae462f623a52
[2]https://en.wikipedia.org/wiki/Information_retrieval#Mean_average_precision
[3]https://medium.com/@jonathan_hui/map-mean-average-precision-for-object-detection-45c121a31173
kaggle_metrics.intersection_over_union(y_true, y_pred)[source]

Intersection over union

Parameters:
  • y_true (numpy.ndarray) – Ground truth
  • y_pred (numpy.ndarray) – Prediction
Returns:

iou_score – Intersection over union score
Return type:

float