GitHub - getumbrel/llama-gpt: A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

🌟 Introducing GitHub - getumbrel/llama-gpt: A self-hosted, offline chatbot powered by Llama 2 for 100% privacy! Now with Code Llama support, various model sizes, easy installation methods, and an OpenAI-compatible API. Join the community and contribute to this innovative AI tool today! 🤖✨ #AI #LLAMAGPT #GitHub

LlamaGPT is a self-hosted, offline chatbot powered by Llama, ensuring 100% privacy without data leaving the user's device.
It now supports Code Llama models and Nvidia GPUs, with various model sizes available for different memory requirements.
Installation methods include one-click deployment on umbrelOS, on M1/M2 Mac with Docker, on any x86 or arm64 system with Docker, and on Kubernetes.
An OpenAI-compatible API is accessible at http://localhost:3001, with documentation at http://localhost:3001/docs.
Benchmark results showing generation speeds for different hardware setups are available for various LlamaGPT models.
The roadmap includes plans for moving the model out of the Docker image, Metal support for Macs, CUDA support for Nvidia GPUs, custom model loading, and more.
Developers are encouraged to contribute to the project by addressing roadmap priorities and issues friendly to new contributors.
Acknowledgments are given to developers and teams involved in making LlamaGPT possible, including those responsible for building the chatbot UI, implementing llama.cpp, and fine-tuning the models.
The project is under the MIT license and actively maintained with a supportive community of contributors.