Open Source Infrastructure tools AI tools

Img2Prompt is an AI tool that generates text prompts from images and provides a public API for image captioning and prompt-based image generation. It can run on personal hardware or cloud platforms.

Lemonade is a self-hosted local AI platform offering GUI, CLI, REST API and SDKs to host and run multimodal models (text, image, code, speech), manage model lifecycle, benchmark inference, deploy on-prem agents, and keep data local.

Foundry Local runs AI models on-device using ONNX Runtime (CPU/GPU/NPU) to keep data local, offering an OpenAI-compatible API, Python/JS/C#/Rust SDKs, a model hub, and CLI tools for edge and enterprise deployments.

KoboldCpp is a versatile AI text-generation tool that supports various GGML and GGUF models with an intuitive UI, native image generation, and enhanced performance via CUDA and CLBlast acceleration.

VLLM is a high-throughput, memory-efficient inference engine for Large Language Models, enabling faster responses and effective memory management. It supports multi-node configurations for scalability and offers robust documentation for seamless integration into workflows.

Llama.cpp is an open-source tool for efficient inference of large language models. Run open source LLM models locally everywhere.

CloudCLI AI is a containerized remote development platform that provides persistent, cross-device coding sessions. It integrates AI coding agents, supports major IDEs, and offers team features for shared environments and configurations.

Unsloth Studio is a no-code web UI enabling local training, running, and exporting of open AI models like Qwen3.5 and NVIDIA Nemotron 3, simplifying experimentation for users without extensive technical expertise.

Open-source desktop app for running local LLMs on Windows/macOS/Linux, supporting text and multimodal inputs, file attachments, multiple model backends with hot-switching, chat/instruction modes, prompt-engineering tools, API/tool-calling, extensibility, and conversation branching.

Colibri is an open-source C inference engine that streams MoE model weights across a unified VRAM/RAM/storage hierarchy to minimize fast-memory usage, using per-layer prefetching, learned caching, batched expert unions, multi-backend execution, and placement planning.

SGLang is a high-performance serving framework for large language and multimodal models, delivering scalable, low-latency inference with high throughput across single-GPU to multi-node clusters, featuring runtime optimizations, OpenAI-compatible API endpoints, and deployment via pip or Docker.