Model Regression Monitoring

Monitaur is an AI governance platform that automates drift, bias, and stress testing for all models. It centralizes policy, risk, and compliance, providing continuous monitoring, vendor controls, and audit‑ready reporting across the entire model lifecycle.

LangWatch enables real‑time testing of LLM agents, offering simulation, prompt management, audit trails, and batch testing across models. It integrates with OpenTelemetry, LangChain, LangGraph, and supports self‑hosted, cloud, and role‑based access.

MLflow is an open‑source AI engineering platform that tracks LLM and agent execution, monitors performance, cost, and safety, manages prompts, and supports experiment tracking, tuning, and deployment across multiple clouds or on‑premises.

Visualping monitors website changes—visual, textual, or code—in real time, sending alerts via email, Teams, Slack, webhooks, or API. It provides before‑and‑after screenshots, AI‑highlighted significant changes, and easy browser integration for individuals and teams.

worldmonitor.app is a real-time global intelligence dashboard that overlays live signals from 500+ feeds and 65+ data providers onto an interactive map. It correlates geopolitical events, infrastructure outages, and sensor alerts with market movements for analysts, traders, and risk managers.

LLM Arena enables users to compare multiple large language models side-by-side, analyzing features like accuracy and capabilities. It supports up to 10 models, facilitating informed decision-making for researchers and developers in selecting the right LLM for their needs.

Pioneer automates retraining and deployment of open-source models, using live inference data for fine-tuning and one-shot adaptation. It manages adaptive inference, routing, RAG pipelines, agent workflows, synthetic data generation, monitoring, and automated checkpoint promotion.

Latitude offers end‑to‑end observability for LLM deployments, recording inputs, outputs, and context. It enables manual annotations, automated error grouping, continuous evaluation, and prompt optimization with GEPA. OTEL telemetry and SDK integrations support major model providers.

BenchLLM evaluates language‑model applications via API or CLI, running JSON/YAML test suites with automated, interactive, or custom strategies. It supports OpenAI, LangChain, and any API, detecting regressions, generating reports, and visualizing results for continuous QA.

Real‑time employee monitoring for Windows, macOS, and Linux. Tracks screens, keystrokes, and apps, offering dashboards, analytics, and reports. Supports office, remote, hybrid, and offline modes with time‑tracking, alerts, SSO, API, and compliance‑ready data retention.

Roark - Voice AI Evals provides monitoring and evaluation tools for voice AI, tracking over 40 call metrics, facilitating multi-speaker analysis, and ensuring compliance with regulations while optimizing voice agent performance through customizable dashboards and automated alerts.

LM Studio runs open‑source large language models locally on Mac (M‑series), Windows, and Linux, enabling private, offline inference. It offers command‑line and headless deployment, server‑side API, SDKs, a model hub, and LM Link for remote model access.

llmarena.ai offers side-by-side LLM comparisons across major providers, showing specs like context window, output capacity, modality and routing options. Filters and role-based categories help developers, ML engineers, product managers and researchers select suitable models.

Open‑source AI code‑review platform that plugs into GitHub, GitLab, Bitbucket, and Azure DevOps at the pull‑request level. Model‑agnostic, it runs custom rule sets, tracks technical debt, and delivers real‑time metrics without storing source code.

Simulation-driven platform that evaluates and monitors AI agents across modalities with realistic multi-turn scenarios, CI/CD-integrated automated tests, configurable safety/policy guardrails, and analytics for failures, hallucinations, and performance to ensure production readiness.

Klu accelerates LLM app development by enabling collaborative prompt design, version control, and automated evaluation across multiple providers. It offers unified observability, cost and drift tracking, private infrastructure, continuous monitoring, and integration with 50+ tools for scalable AI de

ModelOp is a centralized AI governance platform designed to manage enterprise AI initiatives, including generative AI and large language models. It offers automated compliance, real-time reporting, and risk mitigation tools, with over 50 integrations and customizable governance templates for streaml

Runway offers Gen‑4.5 generative video and GWM‑1 world models for real‑time simulation, robotics, and interactive environments. Its Characters API creates autonomous video agents from a single image. Ideal for filmmakers, architects, game developers, and educators.

Parea AI tracks LLM calls, logs cost, latency, and quality, and lets teams create evaluation sets and annotate data in one UI. It offers SDKs and connectors for OpenAI, Anthropic, LangChain, and LiteLLM, enabling continuous observability and prompt testing.

Anomalo automates data quality across structured, semi‑structured, and unstructured data in cloud lakes and warehouses. Using unsupervised ML, it detects anomalies, validates completeness, enforces governance without code, and offers lineage mapping and KPI tracking.

Statstream is an AI-driven IoT platform for monitoring energy usage, production parameters, and utilities in mid-scale enterprises. It provides real-time data access, customizable reporting, and alerts to optimize energy efficiency and minimize downtime.

Testbook AI is a Chrome extension that records and replays user interactions to create regression and UI comparison tests without coding. It supports cross‑browser validation, integrates with Sauce Labs, BrowserStack, LambdaTest, exports code, and auto‑heals scripts.

ModelsLab offers API‑based generative AI for image, video, audio, and language tasks, including editing, generation, and voice synthesis. It supports GPU server deployment, custom workflows, fine‑tuning, and LoRA adaptation for creators and developers.

Typo offers real‑time visibility into development lifecycles, tracking DORA metrics, cycle time, sprint predictability, and productivity. AI code reviews reduce review time and bugs. Integrated natively with CI/CD and version control, it supports secure, enterprise‑scale, data‑driven insights.

42Signals AI delivers real‑time e‑commerce intelligence, tracking product listings, pricing, and search performance across major marketplaces. It monitors unauthorized sellers, provides price alerts, and analyzes customer reviews to inform inventory and marketing decisions.

B2Metric consolidates event, transactional, and behavioral data into a single source, enabling AI‑driven segmentation, churn prediction, and LTV modeling. Real‑time funnel analytics and multichannel campaign tools optimize conversions without manual data prep.

Graphite Note is a user-friendly, no-code predictive analytics tool for cross-industry teams. It delivers accurate predictions (outcomes, lead conversions), analyzes customer behavior, creates personalized marketing strategies, optimizes campaigns, and forecasts demand, simplifying complex data ana

Rainforest QA is a no‑code AI test automation platform for SaaS that instantly generates regression plans and executable tests. It auto‑heals UI changes, offers a visual editor, and integrates with CI/CD for fast, parallel runs and clear debugging.

Lebesgue centralizes eCommerce data from Shopify, WooCommerce, Meta, Google, TikTok, Klaviyo, Amazon, and GA4 into a unified dashboard. It offers first‑party attribution, C‑LTV modeling, product performance, competitive benchmarking, and AI‑guided budget recommendations.

Respan offers AI observability by tracing prompts, tool calls, and responses, enabling end‑to‑end debugging, evaluation with human, code, and LLM reviews, and real‑time monitoring for quality, cost, and compliance, and deployment orchestration across multiple cloud providers.

Trendspid is a comprehensive trade platform offering technical analysis tools, price alerts, market scanning, backtesting strategies, raindrop charts, unusual option flow tracking, and a 7-day free trial with market data from multiple sources.

Weights & Biases is an AI developer platform that simplifies machine learning experiments with tools for tracking, visualizing, and optimizing models. It enhances workflow efficiency through interactive visualizations and collaboration features.

LLM SEO Monitor tracks keyword rankings and AI-generated SERP results across ChatGPT, Claude and Gemini, highlights content gaps and ranking opportunities, provides competitor analysis, automated alerts, exportable reports and API integrations for workflow automation.

Stat Sniper is an AI-driven player prop tracker and bet-tracking app that auto-grades prop bets across sportsbooks into a single ledger. It calculates hit rate, ROI, and provides AI model picks to analyze edges and inform staking decisions.

EvalsOne is an evaluation platform for developers and researchers to assess LLM prompts, RAG, and agents using rule‑based or LLM‑based methods, human judgment, and customizable evaluators. It supports multiple APIs and integrates with major AI frameworks.

VModel provides a unified REST API that lets developers deploy and run custom or community‑built models with a single line of code. It supports Node.js, Python, and cURL for image, text, and video tasks, automatically scaling for production workloads.

Coderbuds automates code‑review workflows, nudging reviewers, suggesting PR splits, and diagnosing deployment failures. It balances workloads, flags stale or oversized changes, shares knowledge, and records DORA and SPACE metrics without storing code, boosting lead time and quality for small teams.

Retalon: AI tool for retail intelligence, providing predictive analytics from planning to markdowns. Enhance profitability through trend prediction, inventory optimization, and informed decision-making for retailers. Based in Toronto, Canada.

Rerun visualizes robotics logs, converting them into training‑ready datasets. Supporting C++, Python, Rust SDKs, it offers browser and desktop viewers for zooming, filtering, and annotating recordings, built‑in dataframe queries, and a shared catalog for collaborative debugging and secure enterprise

Confident AI is an evaluation platform for assessing large language models, enabling benchmarking, unit testing, and A/B testing. It streamlines dataset management and monitoring, ensuring optimal performance and alignment with benchmarks for LLM applications.

Oversight centralizes financial and compliance data, delivering continuous monitoring, fraud detection, and audit automation. Machine‑learning models achieve 95% risk‑detection and 99% duplicate‑payment prevention, integrating natively with ERP and spend‑management systems.

LLM Pricing MCP Server exposes real-time model metrics — token rates, benchmarks, latency, and endpoint availability — inside MCP-enabled assistants, with tools to filter, compare, and rank models for cost- and performance-aware selection and provider compatibility checks.

Codexbar integrates with Cursor to centralize session-based authentication and API usage reporting, surfacing per-model and billing-window metrics, provider cost estimates, account selection, storage footprint, and CLI/manual login options for developer, DevOps, and security teams.

Augmeta Causal Intelligence continuously monitors funnel KPIs, identifies causal drivers, and automatically triggers corrective actions to close revenue leaks. It keeps funnel health optimal, reduces manual work, and ensures steady revenue flow with secure, scalable infrastructure.

WorkMagic automates incremental lift testing with geo‑based holdouts, integrating Shopify and other data to deliver real‑time media mix projections and budget allocation recommendations for paid channels while identifying halo effects across sales channels.

Metaview automates candidate sourcing with 24/7 AI agents, generates interview notes and scorecards, and integrates outreach sequencing. It links to ATS, CRM, and scheduling tools, offers real‑time compliance checks, analytics, and DEI insights for secure, compliant talent acquisition.

Breadcrumbs offers enterprise‑grade, code‑free lead scoring that pulls GTM data via OAuth, visualizes predictive insights, supports multivariate testing, and routes leads in real time to improve lead quality and conversion rates.

Moontower is an options analytics tool for tracking volatility metrics across multiple asset classes. It features proprietary charts, a position visualizer, a VRP scanner, and educational resources, facilitating informed decision-making for traders.

Inception Labs' diffusion-based large language models (dLLMs) offer faster, more efficient, and cost-effective text generation than traditional autoregressive models. With built-in error correction, multimodal support, and structured output control, they excel in function calling and complex data ge

Lingotrack automates language‑learning progress tracking, visualizes study data, and offers a crowdsourced database of foreign media sorted by difficulty. Users log consumption with one‑click or GPT‑powered input, view dashboards, and join community challenges and reviews.