remove is_chat argument
1 unresolved thread
1 unresolved thread
Merge request reports
Activity
Filter activity
requested review from @maximilian.kimmich
added 1 commit
- ac5b4c4a - remove need for cache key computation during runtime
added 1 commit
- 4ef26dcf - Add CLI option for maximum number of tokens being generated
added 1 commit
- a711a5e1 - remove max_tokens parameter from create_completion
added 5 commits
Toggle commit listadded 1 commit
- c97f5570 - fix error for openai model after refactoring
added 1 commit
- 65c45448 - fix error for openai model after refactoring
added 1 commit
- b47cdc45 - fix error for openai model after refactoring
- Resolved by Max Kimmich
- Resolved by Max Kimmich
- Resolved by Max Kimmich
- Resolved by Grießhaber Daniel
- Resolved by Grießhaber Daniel
- Resolved by Grießhaber Daniel
added 1 commit
- 7420ba00 - make OpenAI.build_model_input return all arguments as dict
added 1 commit
- 13b82620 - refactor _call_model_cached function implementation
added 1 commit
- 46b178e4 - replace path separators in model_name with underscores
added 1 commit
- b247f106 - refactor build_model_input to not modify its parameters
requested review from @maximilian.kimmich
added 1 commit
- 33f58a73 - fix build_model_input for OpenAI models to use same layout as the LLama models
83 82 messages = [self._get_user_message(prompt)] 84 85 if chat: 86 # a history is prepended to the messages, and we assume that it also includes a system message, i.e., we never add a system message in this case 87 # TODO is it better to check for a system message in the history? 88 if history is not None: 89 messages = history + messages 90 elif system_message: 91 messages.insert( 92 0, 93 self._get_system_message(system_message), 94 ) 95 model_input = {"messages": messages} 96 else: 97 model_input = {"prompt": prompt} 83 model_input, messages = self.build_model_input( changed this line in version 18 of the diff
in the old code the messages were only altered for the chat model. I refactored it now with a better abstraction factoring out the whole
create_completion
method since it didn't share any code between the chat and non-chat models anymore.One thing I'm not sure about now is if the history shouldn't be prepended to the non-chat messages as well, i.e. if it shouldn't be
return reponse, history + messages, usage
in
create_completion
of theLlama
modelEdited by Grießhaber Daniel
added 1 commit
- 331a4901 - simplify history handling in `ChatModel.create_completion`
mentioned in commit 7b9808b6
Please register or sign in to reply