Local Cpu Inference

local.ai runs language models locally without GPUs. Its Rust backend keeps the binary under 10 MB and performs CPU inference with GGML quantization. A single‑click interface streams responses to a UI, while a model manager tracks, verifies, and resumes downloads.

Foundry Local runs AI models on-device using ONNX Runtime (CPU/GPU/NPU) to keep data local, offering an OpenAI-compatible API, Python/JS/C#/Rust SDKs, a model hub, and CLI tools for edge and enterprise deployments.

canirun.ai is a searchable database mapping AI models to compatible hardware, listing CPUs/GPUs (including Apple M-series and NVIDIA cards), model requirements, VRAM/memory needs, filters and comparisons to plan local inference, fine-tuning, or deployment.

General Compute is an OpenAI-compatible inference API using custom ASIC accelerators to deliver high throughput (e.g., 950 tokens/sec) and dramatically lower power consumption (≈17 kW vs. 120 kW per rack), enabling developers to switch providers by simply changing the base URL and API key. It suppor

Groq is an inference platform that uses custom LPU silicon for low‑latency, high‑throughput AI workloads. It supports large language and multimodal models via an OpenAI‑compatible API, with modular deployment and predictable performance for NLP, vision, and recommendation tasks.

LM Studio runs open‑source large language models locally on Mac (M‑series), Windows, and Linux, enabling private, offline inference. It offers command‑line and headless deployment, server‑side API, SDKs, a model hub, and LM Link for remote model access.

LLMWare AI installs a lightweight client on PCs, providing instant access to 100+ AI models optimized for Intel and Qualcomm hardware. It supports RAG, auto‑tunes weights, runs locally without Wi‑Fi, and offers an admin console for monitoring, scaling, and audit logs.

Inferless is a serverless platform for deploying machine learning models seamlessly. It offers automatic load balancing, custom runtime environments, and automated CI/CD workflows, minimizing infrastructure management while scaling efficiently from single to millions of requests.

Atomic Chat is a fully offline, on-device AI chat app for macOS, Windows, Linux, iOS, and Android that runs 1,000+ LLMs locally with built-in agent support, persistent memory, and privacy-first design. It features TurboQuant optimizations for up to 8x faster attention and lower memory use, with one-

Runpod supplies on‑demand GPUs in 31 regions, offering single‑node pods, multi‑node clusters, and serverless workloads. It delivers low‑latency inference, efficient fine‑tuning, instant scaling, S3‑compatible storage, real‑time logs, and sub‑200 ms cold starts.

Compact edge platform featuring the Hailo‑8 accelerator for up to 83 TOPs. Supports USB, PCIe, Ethernet, and GPIO; runs Linux ≥ 6.18 with drivers, enabling rapid AI deployment for real‑time inference in automotive, security, and industrial inspection.

Cerebras provides a wafer-scale AI accelerator and software stack that enables single-node training of very large LLMs, high-throughput low-latency inference (GLM-4.6 at 1,000 TPS), PyTorch SDK, deployment options, and MLOps tooling.

Massed Compute delivers on‑demand GPU/CPU resources via API and desktop interface, supporting NVIDIA A100/H100/L40/A6000 GPUs and custom clusters. Bare‑metal servers provide direct physical access, while an Inventory API streamlines instance management in a Tier III data‑center with expert support.

Wafer AI is a serverless inference platform that lets you run open-source LLMs in production with OpenAI-compatible APIs. It offers dedicated endpoints with optimized performance, long-context support, and caching to reduce costs for coding, reasoning, and agent workloads.

Trooper.AI provides private EU-hosted bare-metal GPU servers for model training, fine-tuning, and inference, with one-click AI environment templates, full root SSH and NVMe storage, tested CUDA on Ubuntu 22.04, scalable hardware and pause/upgrade controls.

fal.ai offers a unified API for generating images, videos, audio, and 3D models from a library of over 1,000 production‑ready assets. It provides serverless GPU inference, private deployment options, NVIDIA‑cluster fine‑tuning, SOC 2 compliance, and enterprise‑grade support.

SiliconFlow is an AI infrastructure platform enabling high-speed inference for LLMs and multimodal applications, supporting serverless, reserved, and private-cloud deployments. It offers low-latency processing, elastic compute, and built-in monitoring for scalable, cost-efficient AI workloads.

Pioneer automates retraining and deployment of open-source models, using live inference data for fine-tuning and one-shot adaptation. It manages adaptive inference, routing, RAG pipelines, agent workflows, synthetic data generation, monitoring, and automated checkpoint promotion.

GPUX is a serverless inference platform that delivers 1‑second cold starts and GPU‑accelerated execution for models like Stable Diffusion XL, ESRGAN, and Whisper. It supports P2P and read‑write volume access for rapid, scalable deployment on NVIDIA RTX 4090 GPUs.

Cirrascale offers a private AI cloud that supports training and inference on AMD, Cerebras, NVIDIA, and Qualcomm accelerators. It provides zero DevOps, no data‑transfer fees, high‑bandwidth networking, and configurable multi‑GPU servers, streamlining workflows and accelerating deployment.

Inception Labs' diffusion-based large language models (dLLMs) offer faster, more efficient, and cost-effective text generation than traditional autoregressive models. With built-in error correction, multimodal support, and structured output control, they excel in function calling and complex data ge

EmpirioLabs AI is a platform for hosting, deploying, and scaling open-source and proprietary AI models via API or web playground. It supports multimodal, long-context models with optimized endpoints, creative templates, and high-throughput rate limits for production workloads.

Cloud GPU rental platform offering on-demand VMs and bare-metal servers with A100/H100/RTX4090 and other GPUs, configurable vRAM/vCPU, persistent volumes, spot instances, and API-driven provisioning for training, inference, rendering, and HPC workloads.

Lightning AI is a PyTorch Lightning‑based cloud platform for training, deploying, and serving models at scale. It offers GPU workspaces, managed clusters, fractional pay‑as‑you‑go GPU capacity, inference APIs, serverless deployment, security, and integration with LitServe, LitGPT, and LLMs.

MimicPC is a cloud-based AI tool for image generation and AI application deployment in the cloud, offering over 20 pre-deployment applications, including Stable Diffusion.

LearnFast AI offers a 24/7 instant solver for physics and math problems, providing step‑by‑step solutions using GPT‑4o. It handles calculations, text, and image inputs, supporting students, tutors, and lifelong learners with flexible submission options.

DeepSense.ai provides end‑to‑end AI solutions for enterprises, integrating large language models, retrieval‑augmented generation, MLOps, advanced computer‑vision, edge inference, and predictive analytics to deliver scalable, real‑time AI agents, co‑pilots, and maintenance optimization.

ZETIC deploys TorchScript, TensorFlow, and ONNX models to mobile and embedded devices, quantizing for CPU, GPU, or NPU to reach up to 60× speed and 50% size reduction. It supplies benchmarks and a 3‑line offline code snippet for privacy‑preserving AI.

gpt-oss playground provides open-weight demos of gpt-oss-120b and 20b for infrastructure testing, distributed and on-device inference, benchmarking, API integration, and reproducible research, with adjustable reasoning levels and visible-reasoning for diagnostics. Demo-only; validate outputs.

Open-claw.org is a premium, subscription-based AI deployment platform that lets you launch powerful AI agents in one click

HumanLayer is an open-source IDE and orchestration layer for AI coding agents, managing parallel Claude Code sessions, multiclaude workflows, worktrees and remote workers, with context-engineering tools, session replay, workflow templates and GitHub-integrated code-review automation.

Liner.ai is a cross‑platform no‑code ML app that trains models locally in minutes on images, text, audio, or video. It auto‑selects algorithms, offers ready‑to‑use templates, and exports models for web, mobile, or edge deployment.

Union.ai is a cloud‑native AI orchestration platform that lets data scientists and ML engineers build, test, and deploy high‑velocity, pure Python workflows. It supports dynamic branching, real‑time inference, automatic failure recovery, caching, versioning, and observability dashboards.

The Full Stack offers a complete AI lifecycle curriculum, covering prompt engineering, LLMOps, deep learning, GPU selection, model monitoring, ethics, and MLOps. It trains developers, product managers, and researchers to design, build, and deploy AI applications.

MindSpore is a comprehensive AI framework designed for algorithm engineers and data scientists, facilitating the development, deployment, and management of AI models across various platforms. Its key features include built-in support for distributed training and hardware optimization, ensuring scala

Jan is an offline ChatGPT alternative for Mac, Windows, and Linux. Enjoy customizable AI assistants, productivity boosts, and secure, exportable data. Integrate with OpenAI equivalent API server and soon-to-come mobile app.

ClearML AI Infrastructure Platform unifies GPU management, model development, and generative‑AI deployment across on‑prem, cloud, and hybrid setups, offering secure multi‑tenant provisioning, priority scheduling, fractional GPU allocation, integrated IDE, CI/CD, and streamlined workflows for data sc

Unsloth Studio is a no-code web UI enabling local training, running, and exporting of open AI models like Qwen3.5 and NVIDIA Nemotron 3, simplifying experimentation for users without extensive technical expertise.

Enclave AI runs large language models on Mac and iPhone, keeping all text, voice, and document processing offline. It supports on‑device speech recognition, synthesis, and custom assistants, with local, encrypted conversation history and PDF summarization, ensuring privacy.

TAHO is an AI/HPC infrastructure layer that accelerates model serving, training and batch workloads, cutting inference latency and cold starts, improving GPU utilization and parallelism via PeerMesh to boost throughput and reduce compute costs across cloud and on‑prem.

UbiOps offers a unified interface to deploy AI models on local, hybrid, or multi‑cloud environments. It provides version control, API management, resource prioritization, automated scaling, GPU provisioning, and Kubernetes orchestration, aiding cost, security, and compliance for production workloads

Llama is a local AI tool that enables users to create customizable and efficient language models without relying on cloud-based platforms, available for download on MacOS, Windows, and Linux.

Cognitora is a cloud platform for running autonomous AI agents with managed GPUs, storage, persistent virtual machines, and vector DBs. It offers SDKs and integrations, sub-second sandboxed microVMs, auto-scaling, multi-agent coordination, live code execution and terminals.

On‑Page analyzes pages with Google ranking signals, scoring title relevance, intent, freshness, authority, and visual impact. AI Optimizer suggests entity‑based keyword tweaks; Auto‑Optimizer adds related entities; link‑relevancy tools flag irrelevant backlinks, predictive guest‑post evaluation occu

little-coder is a Pi-based coding agent for running 5–25 GB local LLMs via llama.cpp or Ollama, offering Python/Node CLIs and TypeScript extensions, reproducible benchmarks, build/serve guides, and tools for local code generation, on-device development, and evaluation.

UBIAI fine‑tunes LLMs with classifiers, retrievers, and reasoning. It automates PDF/DOCX labeling, synthetic data, and quality filtering; offers 15‑minute prompt‑level tuning or 2‑4 hour weight training; exports to GGUF, safetensors, or Hugging Face for API or custom deployment.

OpenHuman is an open-source personal AI framework for private, on‑premises deployments and local model execution, providing an agent framework, prompt management, local speech (Whisper/Piper), integrations, Docker/one‑click deployment, and developer tooling.

Cortex is a blockchain platform that integrates AI into decentralized applications, enabling on-chain AI inference with GPU resources. It features smart contracts with machine learning, supports Solidity, and offers a collaborative ecosystem for AI model sharing.

Langbase offers a serverless platform for building, deploying, and scaling AI agents. It unifies access to 600+ LLMs, provides built‑in memory, vector, and file storage, and supports durable multi‑step workflows with monitoring and custom actions.

AdaL is a coding agent that keeps code private, learns team patterns, supports terminal and web interfaces, offers model switching (Gemini‑Pro‑3.1, Claude‑Opus‑4.6, Opus‑4.6), and integrates with 1,000+ tools via the Model Context Protocol to automate documentation, design, and deployment.