First Impressions of gpt4all
The main reason I rarely use LLMs (Large Language Models) like ChatGPT is privacy concerns. The data you share can often be used by the providers in a more or less arbitrary way.
I had looked into gpt4all some time ago. At that time, however, there didn’t seem to be any usable models. In my initial tests, the output was extremely slow (sometimes several seconds per word) and the content was also unusable.
My notebook isn’t particularly powerful. The specifications are 16GB RAM, Intel Core i5-10210U, no dedicated GPU.
Test
The installation process seems to have improved. I followed the Quickstart page and within a few minutes gpt4all was ready. However, downloading the models took a while.
My tests involved having the LLM generate a short story in German, then translating it into English, and providing recommendations for tourist attractions for a trip to Japan.
First, I tested Llama 3 8B Instruct. The content was quite usable. However, the model needed over a minute of “thinking time” each time and then produced 2.7 tokens per second.
Next, I tested GPT4All Falcon. While this was faster, needing only a few seconds of “thinking time,” it produced 3.2 tokens per second. However, the generated output was rubbish. It didn’t understand the tasks properly and often repeated itself.
Conclusion
gpt4all is almost usable. The output of the first models is good, and the speed is acceptable.
