Skip to content

Refactor models

Max Kimmich requested to merge refactor-models into master
  • There are now separate methods for creating messages for chat mode
  • Subclasses only take care of calling the model-specific completion methods
  • Model calls can be cached: _call_model will take care of it
    • Models are not taken into account for caching directly, but the options and kwargs used to initialize them are
    • Currently using the joblib.Memory module, not sure what the best option for caching (other options include ring, diskcache) is

Hasn't been tested using OpenAI API so far.

Edited by Max Kimmich

Merge request reports