skeval.utils

skeval.utils.check_is_fitted(model: Any) → None[source]

Check if the model has been fitted.

This method uses sklearn.utils.validation.check_is_fitted to verify that the underlying model has been fitted. It raises a RuntimeError if the model is not fitted.

Parameters:

model (object) – The model to check.

Return type:

None

Raises:

RuntimeError – If the model has not been fitted yet.

skeval.utils.get_cv_and_real_scores(model: Any, scorers: Mapping[str, Callable[[Any, Any], float]], train_data: Tuple[Any, Any], test_data: Tuple[Any, Any]) → Dict[str, Dict[str, float]][source]

Compute cross-validation and real scores for a given model and datasets.

Parameters:

  • evaluator (object) – An evaluator instance with an estimate method.

  • model (object) – A fitted model with fit and predict methods.

  • x_train (array-like) – The training features.

  • y_train (array-like) – The training target labels.

  • x_test (array-like) – The test features.

  • y_test (array-like) – The test target labels.

Returns:

A dictionary containing ‘cv_scores’ and ‘real_scores’.

Return type:

dict

skeval.utils.print_comparison(scorers: Mapping[str, Any], cv_scores: Mapping[str, float], estimated_scores: Mapping[str, float], real_scores: Mapping[str, float]) → None[source]

Print a formatted comparison between cross-validation (intra-domain), estimated, and real performance scores for a set of metrics, and display the absolute errors of the CV and estimated scores with respect to the real scores.

Parameters:

  • scorers (Mapping[str, Any]) – A mapping whose keys are metric names (strings). Only the keys are used to determine which metrics to display; values (e.g., scorer callables) are not inspected by this function.

  • cv_scores (Mapping[str, float]) – Mapping from metric name to the cross-validation (intra-domain) score.

  • estimated_scores (Mapping[str, float]) – Mapping from metric name to the estimated cross-domain score.

  • real_scores (Mapping[str, float]) – Mapping from metric name to the observed real score on the target domain.

Return type:

None