{"repo":"KomeijiForce/Active_Passive_Constraint_Koishiday_2024","free":true,"listed":false,"github":"https://github.com/KomeijiForce/Active_Passive_Constraint_Koishiday_2024","clone":"git clone https://github.com/KomeijiForce/Active_Passive_Constraint_Koishiday_2024.git","description":"Koishi's Day 2024 Paper (NeurIPS 2024): An advanced persona-driven role-playing system with global faithfulness quantification and optimization. In memory of the Koishi's Day of 2024.","language":"Python","stars":13,"topics":["ai-character","factuality-checking","faithfulness","global-optimization","metrics","quantification","role-playing","komeiji"],"license":"MIT","category":"analytics","readme_excerpt":"Koishi's Day 2024: Quantifying and Optimizing Global Faithfulness in Persona-driven Role-playing [English 中文 日本語] - Let there be fantasy [Update] APC is accepted to NeurIPS2024 ! 💚 Introduction [\\[Paper\\]](https://arxiv.org/abs/2405.07726) [\\[NeurIPS Version\\]](https://neurips.cc/virtual/2024/poster/94451) Persona-driven Role-playing (PRP) is so cool that it allows you to build AI characters with several short paragraphs to describe the persona (人设/設定)! However, how to keep the AI character faithfully to ALL persona statements is a hard problem. PRP agents always make bunches of mistakes or are always vague about the knowledge they should know. The main reason behind this limitation is the lack of a metric that can quantify the global PRP faithfulness. So I decide to do so following this intuition: Briefly speaking, whenever a query comes from a user, each persona statement will become an active (relevant to the query) or passive (irrelevant to the query) constraint. To satisfy the active constraint, the response needs to be entailed by the statement (containing the information in the statement) . Otherwise, for passive constraints, the response only needs to be not contradicted by them (not containing information that is incorrect according to the persona statement) . We traverse through all persona statements and see whether their constraints are satisfied or not. We count the number of satisfied constraints, which is used as the metric to evaluate the global PRP faithfuln","default_branch":null,"files":null,"tree":[],"storefront":"/r/KomeijiForce","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/KomeijiForce/Active_Passive_Constraint_Koishiday_2024/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}