Skip to content

251 expand preference model - #252

Open
dkkdark wants to merge 25 commits into
mainfrom
251-expand-preference-model
Open

251 expand preference model#252
dkkdark wants to merge 25 commits into
mainfrom
251-expand-preference-model

Conversation

@dkkdark

@dkkdark dkkdark commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

@dkkdark dkkdark linked an issue Jun 2, 2026 that may be closed by this pull request
Expand Preference model #251
Open
@github-actions

github-actions Bot commented Jun 2, 2026

Copy link
Copy Markdown
Current Branch Main Branch
Coverage Badge Coverage Badge

NoB0
NoB0 reviewed Jun 8, 2026
View reviewed changes
Comment thread usersimcrs/simulator/llm/prompt/stop_prompt.py Outdated
NoB0
NoB0 reviewed Jun 8, 2026
View reviewed changes
Comment thread usersimcrs/simulator/llm/prompt/utterance_generation_prompt.py Outdated
NoB0
NoB0 reviewed Jun 8, 2026
View reviewed changes
Comment thread usersimcrs/simulator/user_simulator.py Outdated
NoB0
NoB0 reviewed Jun 8, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/persona.py Outdated
NoB0
NoB0 reviewed Jun 8, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
dkkdark
dkkdark commented Jun 9, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py
@dkkdark
dkkdark requested a review from NoB0 June 9, 2026 17:03
NoB0
NoB0 reviewed Jun 16, 2026
View reviewed changes

@NoB0 NoB0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial comments

Comment thread usersimcrs/user_modeling/preference_model.py Outdated
Comment thread usersimcrs/user_modeling/preference_model.py Outdated
Comment thread usersimcrs/simulator/agenda_based/agenda_based_simulator.py Outdated
Comment thread usersimcrs/simulator/llm/prompt/prompt.py
Comment thread usersimcrs/simulator/llm/prompt/stop_prompt.py Outdated
Comment thread usersimcrs/simulator/llm/prompt/utterance_generation_prompt.py Outdated
Comment thread usersimcrs/user_modeling/pkg_preference_model.py
Comment thread usersimcrs/user_modeling/simple_preference_model.py
NoB0
NoB0 reviewed Jun 19, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference_model.py
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
@dkkdark
dkkdark requested a review from NoB0 July 14, 2026 15:46
NoB0
NoB0 reviewed Jul 21, 2026
View reviewed changes

@NoB0 NoB0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revise the initialization of the preferences based on the ratings and/or the historical dialogues (remove the end_session method that updates the preferences wihtout saving them for later)

Comment thread usersimcrs/simulator/llm/prompt/stop_prompt.py Outdated
Comment thread usersimcrs/simulator/llm/prompt/utterance_generation_prompt.py Outdated
Comment thread usersimcrs/simulator/llm/prompt/prompt.py Outdated
Comment thread usersimcrs/user_modeling/preference_model.py
Comment thread usersimcrs/user_modeling/simple_preference_model.py
Comment thread usersimcrs/user_modeling/structured_preference/signal_extractor.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference/signal_extractor.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
@dkkdark
dkkdark requested a review from NoB0 August 4, 2026 11:51
NoB0
NoB0 reviewed Aug 11, 2026
View reviewed changes

@NoB0 NoB0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few open questions to look at

Comment thread usersimcrs/user_modeling/persona.py Outdated
Comment thread usersimcrs/user_modeling/preference_model.py Outdated
Comment thread usersimcrs/user_modeling/simple_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference/preference_memory.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference/preference_memory.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference/llm_signal_extractor.py Outdated
Ksenia Blokhina added 2 commits August 18, 2026 10:17
@dkkdark
dkkdark requested a review from NoB0 August 18, 2026 08:23
NoB0
NoB0 reviewed Aug 18, 2026
View reviewed changes
Comment thread usersimcrs/user_modeling/structured_preference/preference_memory.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference/signal_extractor.py Outdated
Comment thread usersimcrs/user_modeling/structured_preference_model.py Outdated
def _slot_preference_signals(
self, signals: Iterable[PreferenceSignal]
) -> List[PreferenceSignal]:
"""Converts extracted signals to slot-value preference signals.

@NoB0 NoB0 Aug 18, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this method is unclear to me.

@dkkdark dkkdark Aug 18, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I move it to HeuristicPreferenceSignalsExtractor and called _item_metadata_preference_candidates. It's used to create signals

@dkkdark
dkkdark requested a review from NoB0 August 18, 2026 12:23
@dkkdark

dkkdark commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator Author

Existing problems:

  1. \end was treated as a normal user message. Problem: the user simulator says \end, but the agent receives it and keeps responding. Fixed in this branch
  2. Problem: after repeated “Sorry, I couldn’t find...” responses, the simulator keeps asking the same query with tiny wording changes.
    Fix: update stop_definition and/or prompt or relax constraints
  3. long-term preferences like future appear in the user’s opening request even though they are not in the information need: strengthen prompt wording:
  4. Weak heuristic rules. Strengthen them

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@NoB0 NoB0 Awaiting requested review from NoB0

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Expand Preference model

2 participants

@dkkdark @NoB0