water_benchmark_hub

water_benchmark_hub.benchmark_resource

Module provides a base class for benchmark resources such as datasets, .inp files, control environments, etc.

class water_benchmark_hub.benchmark_resource.BenchmarkResource

Bases: object

Base class for benchmark resources such as datasets, .inp files, control environments, etc.

static get_meta_info() → dict

Gets the meta information of this resource.

Returns:

Meta info.

Return type:

dict

water_benchmark_hub.meta_data

This module contains a class for querying the database.

class water_benchmark_hub.meta_data.MetaData

Bases: object

Class for querying the database (“webpage/static/database.json”) and retrieving the meta info of each benchmark resource.

static get_meta_info(resource_id: str) → dict

Gets the meta information of a specific resource.

Returns:

Meta info.

Return type:

dict

water_benchmark_hub.meta_data.meta_data(benchmark_id: str) → Any

Decorator for making the meta data available in the benchmark resource class.

Parameters:

benchmark_id (str) – ID of the benchmark resource.

Returns:

Extended class.

Return type:

Any

water_benchmark_hub.benchmarks

Module provides functions for registering and loading benchmarks.

water_benchmark_hub.benchmarks.load(res_name: str) → BenchmarkResource

Loads a registered benchmark.

Parameters:

res_name (str) – Name of the benchmark.

Returns:

Benchmark.

Return type:

BenchmarkResource

water_benchmark_hub.benchmarks.register(res_name: str, res: BenchmarkResource) → None

Registers a new benchmark under a given name.

Parameters:

  • res_name (str) – Name of the benchmark – must be unique among all benchmarks.

  • res (BenchmarkResource) – Benchmark.

water_benchmark_hub.metrics

This module provides different metrics for evaluation.

water_benchmark_hub.metrics.accuracy_score(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the accuracy of a classification.

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

Accuracy score.

Return type:

float

water_benchmark_hub.metrics.f1_micro_score(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the F1 score using for a multi-class classification by counting the total true positives, false negatives and false positives.

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

F1 score.

Return type:

float

water_benchmark_hub.metrics.f1_score(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the F1-score for a binary classification.

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

F1-score.

Return type:

float

water_benchmark_hub.metrics.mape(y_pred: numpy.ndarray, y: numpy.ndarray, epsilon: float = 0.05) → float

Computes the Mean Absolute Percentage Error (MAPE).

Parameters:

  • y_pred (numpy.ndarray) – Predicted outputs.

  • y (numpy.ndarray) – Ground truth outputs.

  • epsilon (float, optional) –

    Small number added to predictions and ground truth to avoid division-by-zero.

    The default is 0.05

Returns:

MAPE score.

Return type:

float

water_benchmark_hub.metrics.mase(y_pred: numpy.ndarray, y: numpy.ndarray, epsilon: float = 0.05) → float

Computes the Mean Absolute Scaled Error (MASE).

Parameters:

  • y_pred (numpy.ndarray) – Predicted outputs.

  • y (numpy.ndarray) – Ground truth outputs.

  • epsilon (float, optional) –

    Small number added to predictions and ground truth to avoid division-by-zero.

    The default is 0.05

Returns:

MASE score.

Return type:

float

water_benchmark_hub.metrics.precision_score(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the precision of a classification.

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

Precision score.

Return type:

float

water_benchmark_hub.metrics.roc_auc_score(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the Area Under the Curve (AUC) of a classification.

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

ROC AUC score.

Return type:

float

water_benchmark_hub.metrics.running_mse(y_pred: numpy.ndarray, y: numpy.ndarray)

Computes the running Mean Squared Error (MSE).

Parameters:

  • y_pred (numpy.ndarray) – Predicted outputs.

  • y (numpy.ndarray) – Ground truth outputs.

Returns:

Running MSE.

Return type:

float

water_benchmark_hub.metrics.smape(y_pred: numpy.ndarray, y: numpy.ndarray, epsilon: float = 0.05) → float

Computes the Symmetric Mean Absolute Percentage Error (SMAPE).

Parameters:

  • y_pred (numpy.ndarray) – Predicted outputs.

  • y (numpy.ndarray) – Ground truth outputs.

  • epsilon (float, optional) –

    Small number added to predictions and ground truth to avoid division-by-zero.

    The default is 0.05

Returns:

SMAPE score.

Return type:

float

water_benchmark_hub.metrics.true_negative_rate(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the true negative rate (also called specificity).

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

True negative rate.

Return type:

float

water_benchmark_hub.metrics.true_positive_rate(y_pred: numpy.ndarray, y: numpy.ndarray) → float

Computes the true positive rate (also called sensitivity).

Parameters:

  • y_pred (numpy.ndarray) – Predicted labels.

  • y (numpy.ndarray) – Ground truth labels.

Returns:

True positive rate.

Return type:

float