^i dZddlZddlmZmZmZddlmZm Z m Z m Z ddl Z e jeZdeeefdeeefdefdZdeeefdeeefdefd Zdeeefdeeefdefd Zdeeefdeeefdefd Zd eedefd Zy)z+Evaluation metrics for preference learning.N)ListDictAny)accuracy_scoreprecision_score recall_scoref1_scorey_truey_predreturnctt|jt|jz}|Dcgc]}|| }}|Dcgc]}|| }}t||Scc}wcc}w)zCompute accuracy. Args: y_true: Ground truth labels (dict mapping pattern_id to label) y_pred: Predicted labels (dict mapping pattern_id to label) Returns: Accuracy score )sortedsetkeysrr r pattern_idspid true_labels pred_labelss I/home/ubuntu/codebase/yexijia/保研/colocation_mvp/experiment/metrics.pyaccuracyr snV[[]+c&++-.@@AK*56+36#;+K6*56+36#;+K6 +{ 3376s  A- A2ctt|jt|jz}|Dcgc]}|| }}|Dcgc]}|| }}t||dScc}wcc}w)zCompute precision. Args: y_true: Ground truth labels y_pred: Predicted labels Returns: Precision score  zero_division)rrrrrs r precisionrspV[[]+c&++-.@@AK*56+36#;+K6*56+36#;+K6 ; 3 GG76  A/ A4ctt|jt|jz}|Dcgc]}|| }}|Dcgc]}|| }}t||dScc}wcc}w)zCompute recall. Args: y_true: Ground truth labels y_pred: Predicted labels Returns: Recall score rr)rrrrrs rrecallr.spV[[]+c&++-.@@AK*56+36#;+K6*56+36#;+K6  [ DD76rctt|jt|jz}|Dcgc]}|| }}|Dcgc]}|| }}t||dScc}wcc}w)zCompute F1 score. Args: y_true: Ground truth labels y_pred: Predicted labels Returns: F1 score rr)rrrr rs rf1r!?spV[[]+c&++-.@@AK*56+36#;+K6*56+36#;+K6 KC @@76rscoresc\|syd}tjd|ddt|d|S)zCompute dynamic threshold using median. IMPORTANT: Do NOT use fixed threshold. Args: scores: List of prediction scores Returns: Threshold value (median of scores) rgffffff?zComputed threshold: z.4fz (median of z scores))loggerdebuglen)r" thresholds rcompute_thresholdr(Ps9 I LL' #l3v;-xXY )__doc__numpynptypingrrrsklearn.metricsrrrr logging getLogger__name__r$intfloatrrrr!r(r)rr5s1""SS   8 $4T#s(^4T#s(^44$Hd38nHd38nHH"E4S>E4S>EeE"AtCH~AtCH~A%A"d5ker)