Skip to content
Snippets Groups Projects
Commit 3faa07e9 authored by Max Kimmich's avatar Max Kimmich
Browse files

Offload all layers to GPU for llama_cpp

parent cbade5a0
No related branches found
No related tags found
1 merge request!7Refactor tasks and models and fix format for various models
Hide whitespace changes
Inline Side-by-side
Showing
with 1 addition and 1 deletion
evoprompt/models.py
+ 1
− 1
View file @ 3faa07e9
...@@ -209,7 +209,7 @@ class Llama(LLMModel): ...@@ -209,7 +209,7 @@ class Llama(LLMModel):
chat_handler: str | None = None, chat_handler: str | None = None,
verbose: int | bool = False, verbose: int | bool = False,
llama_verbose: bool = False, llama_verbose: bool = False,
n_gpu_layers: int = 60, n_gpu_layers: int = -1,
n_threads: int = 8, n_threads: int = 8,
n_ctx: int = 8192, n_ctx: int = 8192,
**kwargs, **kwargs,
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment