Open Source Infrastructure tools AI tools

Img2Prompt is an AI tool that generates text prompts from images and provides a public API for image captioning and prompt-based image generation. It can run on personal hardware or cloud platforms.

Lemonade is a self-hosted local AI platform offering GUI, CLI, REST API and SDKs to host and run multimodal models (text, image, code, speech), manage model lifecycle, benchmark inference, deploy on-prem agents, and keep data local.

Foundry Local runs AI models on-device using ONNX Runtime (CPU/GPU/NPU) to keep data local, offering an OpenAI-compatible API, Python/JS/C#/Rust SDKs, a model hub, and CLI tools for edge and enterprise deployments.

KoboldCpp is a versatile AI text-generation tool that supports various GGML and GGUF models with an intuitive UI, native image generation, and enhanced performance via CUDA and CLBlast acceleration.

VLLM is a high-throughput, memory-efficient inference engine for Large Language Models, enabling faster responses and effective memory management. It supports multi-node configurations for scalability and offers robust documentation for seamless integration into workflows.

Llama.cpp is an open-source tool for efficient inference of large language models. Run open source LLM models locally everywhere.

CloudCLI AI is a containerized remote development platform that provides persistent, cross-device coding sessions. It integrates AI coding agents, supports major IDEs, and offers team features for shared environments and configurations.

Unsloth Studio is a no-code web UI enabling local training, running, and exporting of open AI models like Qwen3.5 and NVIDIA Nemotron 3, simplifying experimentation for users without extensive technical expertise.

Open-source desktop app for running local LLMs on Windows/macOS/Linux, supporting text and multimodal inputs, file attachments, multiple model backends with hot-switching, chat/instruction modes, prompt-engineering tools, API/tool-calling, extensibility, and conversation branching.

Colibri is an open-source C inference engine that streams MoE model weights across a unified VRAM/RAM/storage hierarchy to minimize fast-memory usage, using per-layer prefetching, learned caching, batched expert unions, multi-backend execution, and placement planning.

SGLang is a high-performance serving framework for large language and multimodal models, delivering scalable, low-latency inference with high throughput across single-GPU to multi-node clusters, featuring runtime optimizations, OpenAI-compatible API endpoints, and deployment via pip or Docker.

Self Hosted AI Stack is an open-source Docker Compose platform for deploying local AI models with chat, document embeddings and retrieval, speech interfaces, and tool integrations. It bundles model serving, conversational UIs, embedding pipelines, and gateway components with CPU/GPU support, configuration presets, and security defaults for local, cloud, or hybrid deployment.