Li/=dZddlZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddl mZe jeZGddZy)XUser preference learner using average vector method with positive and negative feedback.N)ListOptionalDictAnyTuple) MemoryStore)PatternEmbedderc heZdZdZddedefdZdeee jee jffdZ dee jfdZ de dee jfd Zdde d edee jfd Z dd eee efdee dedeeefdZd eee efdeedee deefdZd eee efde jdeefdZde jde jdede jfdZdde dedee jfdZ dd eee efdee dedeeefdZy )PreferenceLearnerr decay_lambdause_preference_weightedc\t|_t|_||_||_y)uInitialize preference learner. Args: decay_lambda: Decay parameter for intent vector weight (α_t = e^(-λt)) use_preference_weighted: If True, multiply similarity by preference weight (adaptive fusion, 2025-style) N)r memoryr embedderr r)selfr rs G/home/ubuntu/codebase/yexijia/保研/colocation_mvp/learning/learner.py__init__zPreferenceLearner.__init__s'"m ') ('>$returnc|jj}g}g}|jdgD]}|jdi}|jdg}|D]M}t|tr|jdg}nt|t r|}n:|s=|j |O|jdg} | D]M}t|tr|jdg}nt|t r|}n:|s=|j |Od} t|dkDrd|jj|} | Gt| dkDr9tj| d} tjd t|d d} t|dkDrd|jj|} | Gt| dkDr9tj| d} tjd t|d | | y | | fS)a5Build user preference vectors from historical feedback. Returns: Tuple of (positive_vector, negative_vector), or (None, None) if no feedback - positive_vector: Average vector of liked patterns - negative_vector: Average vector of disliked patterns sessionsfeedbacklikepatterndislikeNraxiszBuilt positive vector from z liked patternszBuilt negative vector from z disliked patterns)NN)rloadget isinstancedictlistappendlenrencode_patternsnpmeanloggerdebug) rdataliked_patternsdisliked_patternssessionrliked pattern_datardislikedpositive_vectorvectorsnegative_vectors rbuild_user_vectorsz$PreferenceLearner.build_user_vectorss{{!xx B/G{{:r2HLL,E % lD1*..y"=G d3*G"))'2!& ||Ir2H ( lD1*..y"=G d3*G%,,W5!)%0< ~  "mm33NCG"s7|a'7"$'''": :3~;N:O_`  !A %mm334EFG"s7|a'7"$'''": :3?P;Q:RRdef  "'>//rc,|j\}}|S)zBuild user preference vector from historical feedback (backward compatibility). Returns: User preference vector (average of liked patterns), or None if no feedback )r5)r positive_vec_s rbuild_user_vectorz#PreferenceLearner.build_user_vector]s 113 aruser_idc|jj|}|sy|jdg}|jdg}tj|j j }|r8tj|Dcgc]}tj|c}d}tj|j j }|r8tj|Dcgc]}tj|c}d}||z }tjj|dk(ry|Scc}wcc}w)uCompute user preference vector from feedback (μ⁺ - μ⁻). Args: user_id: User identifier Returns: Feedback-based user vector (positive_mean - negative_mean), or None if no feedback Npositivenegativerr) rget_user_profiler r'zerosr get_sentence_embedding_dimensionr(arraylinalgnorm) rr: user_profilepositive_vectorsnegative_vectorsmean_posvmean_neg user_vectors rcompute_feedback_vectorz)PreferenceLearner.compute_feedback_vectorfs{{33G< '++J;'++J;88DMMJJLM ww5EF5E 5EFQOH88DMMJJLM ww5EF5E 5EFQOH) 99>>+ &! + G Gs D;,Etcl|jj|}d}|r)|jdrtj|d}|j |}||y||S||St j|j |z}||zd|z |zz}tjd|dd||S)uGet fused user preference vector combining LLM intent and feedback. Formula: u_t = α_t * u_llm + (1-α_t) * u_feedback where α_t = e^(-λt) Args: user_id: User identifier t: Interaction round number (0 for first interaction) Returns: Fused user preference vector, or None if neither intent nor feedback available Nu_llmzFused user vector: alpha_t=z.4fz, t=) r load_intentr r'rArKmathexpr r)r*)rr:rL intent_datarN u_feedbackalpha_tu_ts rget_user_vectorz!PreferenceLearner.get_user_vectorskk--g6  ;??73HH[12E11':  =Z/ =   L((D---12oW :: 273-tA3GH rNpatternsinteraction_roundcJ|r9|j||}|%|j||}|j|||S|j\}}||y|Dcgc]}|j dg} }| D cgc]} | s|  } } | s$t j ddgt|zS|jj| } | t jdyg} | D]}d}|itjj|}tjj|}|dkDr&|dkDr!tj||||zz }||z }|itjj|}tjj|}|dkDr&|dkDr!tj||||zz }||z}| j|g}d}| D]/} | r|j| ||dz }|jd1|j|||Scc}wcc} w)uScore patterns based on user preference vectors (fused LLM intent + feedback). If user_id is provided and Stage0 is enabled, uses fused vector: u_t = α_t * u_llm + (1-α_t) * u_feedback Otherwise, falls back to original method: score = similarity_to_positive - similarity_to_negative Args: patterns: List of pattern dictionaries, each containing 'pattern' field user_id: Optional user identifier for Stage0 fusion interaction_round: Current interaction round number (for decay calculation) Returns: List of similarity scores, or None if no user vectors available NrzNo valid patterns to score.z&Could not encode patterns for scoring.rrO)rW_score_with_vector_apply_preference_weightsr5r r)infor%rr&warningr'rBrCdotr$)rrXr:rY fused_vectorrawr7 negative_vecp pattern_listsp_listvalid_pattern_listsp_vecsscoresrHscore positive_normv_normpositive_similarity negative_normnegative_similarity final_scores valid_idxs rscore_patternsz PreferenceLearner.score_patternss>& //9JKL'--h E55hWMM&*%<%<%>" l  L$88@@x!y"-x @5BLM&VvML" KK5 653x=( (../BC > NNC DAE' " | < *A:-!"3*,&&L*AVmE[*\'00E' " | < *A:-!"3*,&&L*AVmE[*\'00E MM% +0  #F##F9$56Q ##C( $--h gNNgAMsH7H ?H ricP|jr|rt|t|k7r|S|jj|}|s|St |j dxsgt |j dxsgg}t |D]\}}|j dg}t|tr>|jdD cgc]#} | js| j%}} n(|r&|D cgc]} t| j}} tt|d} tfd|D} tfd|D} d| | z | z d zz} td td | } |j||| z |Scc} wcc} w) aApply preference-weighted fusion (adaptive fusion of similarity and feature-level preference). Aligns with 2025 work on adaptive fusion of multi-dimensional user preferences. weight = 1 + (n_liked - n_disliked) / |pattern| * 0.5, clamped to [0.5, 1.5]. rrr,rOc3,K|] }|vsd ywrON).0f liked_sets r z>PreferenceLearner._apply_preference_weights..s=Ua9n!U c3,K|] }|vsd ywrvrw)rxry disliked_sets rr{z>PreferenceLearner._apply_preference_weights..sC1l1BQr|g?g?g?)rr%rget_preference_featuressetr enumerater!strsplitstripmaxsumminr$)rrXrir:pfoutirdplistxnn_liked n_dislikedwr~rzs @@rr]z+PreferenceLearner._apply_preference_weightsso ++7c&kSQY]>ZM [[ 0 0 9Mv,"- 266),23 h'DAqEE)R(E%%,1KK,<J,53x=( (IINN;/ AYY^^A&Fzi!mVVA{3v 7IJ  j) c"   #F##F9$56Q ##C( $EAMsD9D>D>llm_vecfb_vecalphac||zd|z |zzS)uKFuse LLM intent vector and feedback vector. Formula: u_t = α * u_llm + (1-α) * u_feedback Args: llm_vec: LLM intent vector fb_vec: Feedback-based vector alpha: Fusion weight (0-1) Returns: Fused user preference vector rOrw)rrrrs r fuse_vectorzPreferenceLearner.fuse_vectorSsw!e)v!555rc|j|}|jj|}d}|r)|jdrt j |d}|||j |||S||S||Sy)aDUpdate user vector iteratively (for Stage4). Gets latest feedback and fuses with LLM intent vector. Args: user_id: User identifier alpha: Fusion weight for LLM intent vs feedback Returns: Updated fused user preference vector NrN)rKrrPr r'rAr)rr:rrTrSrNs rupdate_user_vector_iterativez.PreferenceLearner.update_user_vector_iterativebs11': kk--g6  ;??73HH[12E  !7##E:u= =  #   Lrc*|j|||S)aScore all patterns for evaluation. This is a wrapper around score_patterns for evaluation purposes. Args: patterns: List of pattern dictionaries user_id: Optional user identifier interaction_round: Current interaction round number Returns: List of similarity scores, or None if no user vectors available )r:rY)rr)rrXr:rYs rscore_all_patternsz$PreferenceLearner.score_all_patternss""8WPa"bbr)g?F)r)Nr)g333333?)__name__ __module__ __qualname____doc__floatboolrrrr'ndarrayr5r9rrKintrWrrrrrr]r\rrrrwrrr r sb ?U ?D ?>0E(2::*>@T*T$U>0@8BJJ#7 s x 7K D(s(s(8BJJ;O(TW[/0TOtDcN';TOhsmTO),TO5=d5k5JTOl$tCH~2FPTUZP[,4SM>B5k8-4S#X+?-bjj-]abg]h-^ 62:: 6rzz 6% 6TVT^T^ 6CPXY[YcYcPd>[_45c4S#X+?c(SV-c.1c:B4;:Ocrr )rnumpyr'rQtypingrrrrrlogging memory.storer learning.embedderr getLoggerrr)r rwrrrs=^ 33$-   8 $AcAcr