Warning: Some posts on this platform may contain adult material intended for mature audiences only. Viewer discretion is advised. By clicking ‘Continue’, you confirm that you are 18 years or older and consent to viewing explicit content.
Would that actually be decent? Even 6b models feel way too rudimentary after experiencing 33+b models and/or chatgpt. I haven’t tried those really scaled down and optimized models, though!
They’re decent for text completion purposes, e.g. generating some corpspeak for an email, or generating some “wikipedia”-like text. You have to know how to write good prompts, don’t try to treat it like ChatGPT.
For example if i want to know about the history of Puerto Rico I would put:
“The history of puerto rico starts in about 480BC when”
lamacpp is kind of this. you give it a path to a model and a prompt. download a smallish model GGUF file by thebloke. lamacpp main exec has an interactive mode too which can function like a chatbot. i never built it on android though maybe you need android NDK
Compile llama.cpp, download a small GGML LLM model and you will have a quite intelligent assiatant running into your phone.
Would that actually be decent? Even 6b models feel way too rudimentary after experiencing 33+b models and/or chatgpt. I haven’t tried those really scaled down and optimized models, though!
Decent enough for a model 50 times smaller than ChatGPT. I use orca_mini_3b.
They’re decent for text completion purposes, e.g. generating some corpspeak for an email, or generating some “wikipedia”-like text. You have to know how to write good prompts, don’t try to treat it like ChatGPT.
For example if i want to know about the history of Puerto Rico I would put:
“The history of puerto rico starts in about 480BC when”
I got llama to compile but it crashes every time I tried running it.
Do we have any llama apk/deb/flatpack/appimage where you just choose the llm to use?
Idk if GPT4All is what you want.
Do we have apt install gtp4qll?
https://gpt4all.io/index.html
Says ubuntu, works fine on my non-apt distro.
lamacpp is kind of this. you give it a path to a model and a prompt. download a smallish model GGUF file by thebloke. lamacpp main exec has an interactive mode too which can function like a chatbot. i never built it on android though maybe you need android NDK