aditya/llama.cpp
1
0
Fork 0
mirror of https://git.adityakumar.xyz/llama.cpp.git synced 2024-11-09 23:29:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update README.md

Browse source
This commit is contained in:
Georgi Gerganov 2023-03-10 21:52:27 +02:00 committed by GitHub
parent 319cdb3e1f
commit 18ebda34d6
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -15,7 +15,7 @@ The main goal is to run the model using 4-bit quantization on a MacBook.
This was hacked in an evening - I have no idea if it works correctly. This was hacked in an evening - I have no idea if it works correctly.
So far, I've tested just the 7B model and the generated text starts coherently, but typically degrades significanlty after ~30-40 tokens. So far, I've tested just the 7B model and the generated text starts coherently, but typically degrades significanlty after ~30-40 tokens.
Here is a "typicaly" run: Here is a "typical" run:
```java ```java
make -j && ./main -m ./models/7B/ggml-model-q4_0.bin -t 8 -n 128 make -j && ./main -m ./models/7B/ggml-model-q4_0.bin -t 8 -n 128
@ -73,7 +73,7 @@ sampling parameters: temp = 0.800000, top_k = 40, top_p = 0.950000
If you are a fan of the original Star Wars trilogy, then you'll want to see this. If you are a fan of the original Star Wars trilogy, then you'll want to see this.
If you don't know your Star Wars lore, this will be a huge eye-opening and you will be a little confusing. If you don't know your Star Wars lore, this will be a huge eye-opening and you will be a little confusing.
Awesome movie.(end of text) Awesome movie. [end of text]
main: mem per token = 14434244 bytes main: mem per token = 14434244 bytes