Open Source LLM AI tools

LLMs Work visually explains LLM internals—pre-trai...

LLMs Work visually explains LLM internals—pre-training, tokenization, transformer blocks, attention, inference, and RAG—plus dataset curation, training dynamics, and deployment considerations, with visualizations and live examples for debugging, evaluation, and model interpretation.

Mistral.rs is an efficient, versatile tool for hig...

Mistral.rs is an efficient, versatile tool for high-speed large language model (LLM) inference, offering multi-device support and extensive quantization options for seamless deployment on diverse hardware setups.

VLLM is a high-throughput, memory-efficient infere...

VLLM is a high-throughput, memory-efficient inference engine for Large Language Models, enabling faster responses and effective memory management. It supports multi-node configurations for scalability and offers robust documentation for seamless integration into workflows.

exllama is a memory-efficient tool for executing H...

exllama is a memory-efficient tool for executing Hugging Face transformers with the LLaMA models using quantized weights, enabling high-performance NLP tasks on modern GPUs while minimizing memory usage and supporting various hardware configurations.

The text-generation-webui is a Gradio-based web UI...

The text-generation-webui is a Gradio-based web UI for Large Language Models, supporting various backends and multiple interface modes. It allows quick model switching, extension integration, and dynamic LoRA loading for custom training.

LLM-answer-engine is an advanced answer engine lev...

LLM-answer-engine is an advanced answer engine leveraging Groq, Mixtral, Langchain.JS, Brave Search, Serper API, and OpenAI to provide sources, answers, images, videos, and follow-up questions efficiently. It offers an opensource Perplexity alternative.

Mistral AI offers developers a platform for buildi...

Mistral AI offers developers a platform for building cutting-edge generative AI models with a focus on performance and customization. Their models excel in reasoning tasks and benchmarks, providing flexible deployment options across infrastructures.