metrics

evaluation

class castle.metrics.evaluation.MetricsDAG(B_est, B_true)[source]

Bases: object

Compute various accuracy metrics for B_est. true positive(TP): an edge estimated with correct direction. true nagative(TN): an edge that is neither in estimated graph nor in true graph. false positive(FP): an edge that is in estimated graph but not in the true graph. false negative(FN): an edge that is not in estimated graph but in the true graph. reverse = an edge estimated with reversed direction.

fdr: (reverse + FP) / (TP + FP) tpr: TP/(TP + FN) fpr: (reverse + FP) / (TN + FP) shd: undirected extra + undirected missing + reverse nnz: TP + FP precision: TP/(TP + FP) recall: TP/(TP + FN) F1: 2*(recall*precision)/(recall+precision) gscore: max(0, (TP-FP))/(TP+FN), A score ranges from 0 to 1

Parameters

B_est: np.ndarray: [d, d] estimate, {0, 1, -1}, -1 is undirected edge in CPDAG.
B_true: np.ndarray: [d, d] ground truth graph, {0, 1}.

class castle.metrics.evaluation.StructureMetrics(est_graph, true_graph)[source]

Bases: object

Compute various accuracy metrics for est_graph

true positive(TP): an edge estimated with correct direction. true nagative(TN): an edge that is neither in estimated graph nor in true graph. false positive(FP): an edge that is in estimated graph but not in the true graph. false negative(FN): an edge that is not in estimated graph but in the true graph. reverse = an edge estimated with reversed direction.

fdr: (reverse + FP) / (TP + FP) tpr: TP/(TP + FN) fpr: (reverse + FP) / (TN + FP) shd: undirected extra + undirected missing + reverse nnz: TP + FP precision: TP / (TP + FP) recall: TP / (TP + FN) F1: 2 * (recall * precision) / (recall + precision) gscore: max(0, (TP - FP)) / (TP + FN), A score ranges from 0 to 1

Parameters

est_graph: np.ndarray: [d, d] estimate, {0, 1}.
true_graph: np.ndarray: [d, d] ground truth graph, {0, 1}.

Examples

>>> import numpy as np
>>> from castle.metrics.evaluation import StructureMetrics
>>> est_graph = np.array([[0, 1, 0], [0, 0, 1], [1, 1, 0]])
>>> true_graph = np.array([[0, 1, 0], [0, 0, 1], [0, 0, 0]])
>>> metrics = StructureMetrics(est_graph, true_graph)
You can get all metrics from metrics.values, like the following example:
>>> metrics_value = metrics.values
>>> print(metrics_value)
{'fdr': 0.3333, 'tpr': 1.0, 'fpr': 1.0, 'shd': 1, 'nnz': 3, 'precision': 0.6667, 'recall': 1.0, 'F1': 0.8, 'gscore': 0.5}

Also, you can just get one metrics score if you need: >>> f1 = metrics.f1_score >>> print(f1) 0.8 >>> gscore = metrics.g_score >>> print(gscore) 0.5

static check_params(est_graph, true_graph)[source]

Check parameters and make them available

Parameters

est_graph: np.ndarray, Tensor: [d, d] estimate, {0, 1, -1}, -1 is undirected edge in CPDAG.
true_graph: np.ndarray, Tensor: [d, d] ground truth graph, {0, 1}.

Returns

est_graph true_graph

property f1_score

property fdr

property fpr

property g_score

property nnz

property precision

property recall

property shd

property tpr

property values