Model Regression Monitoring

Monitaur is an AI governance platform that automates drift, bias, and stress testing for all models. It centralizes policy, risk, and compliance, providing continuous monitoring, vendor controls, and audit‑ready reporting across the entire model lifecycle.

LangWatch enables real‑time testing of LLM agents, offering simulation, prompt management, audit trails, and batch testing across models. It integrates with OpenTelemetry, LangChain, LangGraph, and supports self‑hosted, cloud, and role‑based access.

BenchLLM evaluates language‑model applications via API or CLI, running JSON/YAML test suites with automated, interactive, or custom strategies. It supports OpenAI, LangChain, and any API, detecting regressions, generating reports, and visualizing results for continuous QA.

Latitude offers end‑to‑end observability for LLM deployments, recording inputs, outputs, and context. It enables manual annotations, automated error grouping, continuous evaluation, and prompt optimization with GEPA. OTEL telemetry and SDK integrations support major model providers.

Real‑time employee monitoring for Windows, macOS, and Linux. Tracks screens, keystrokes, and apps, offering dashboards, analytics, and reports. Supports office, remote, hybrid, and offline modes with time‑tracking, alerts, SSO, API, and compliance‑ready data retention.

llmarena.ai offers side-by-side LLM comparisons across major providers, showing specs like context window, output capacity, modality and routing options. Filters and role-based categories help developers, ML engineers, product managers and researchers select suitable models.

Roark - Voice AI Evals provides monitoring and evaluation tools for voice AI, tracking over 40 call metrics, facilitating multi-speaker analysis, and ensuring compliance with regulations while optimizing voice agent performance through customizable dashboards and automated alerts.

LM Studio runs open‑source large language models locally on Mac (M‑series), Windows, and Linux, enabling private, offline inference. It offers command‑line and headless deployment, server‑side API, SDKs, a model hub, and LM Link for remote model access.

Typo offers real‑time visibility into development lifecycles, tracking DORA metrics, cycle time, sprint predictability, and productivity. AI code reviews reduce review time and bugs. Integrated natively with CI/CD and version control, it supports secure, enterprise‑scale, data‑driven insights.

Klu accelerates LLM app development by enabling collaborative prompt design, version control, and automated evaluation across multiple providers. It offers unified observability, cost and drift tracking, private infrastructure, continuous monitoring, and integration with 50+ tools for scalable AI de

Open‑source AI code‑review platform that plugs into GitHub, GitLab, Bitbucket, and Azure DevOps at the pull‑request level. Model‑agnostic, it runs custom rule sets, tracks technical debt, and delivers real‑time metrics without storing source code.

Runway offers Gen‑4.5 generative video and GWM‑1 world models for real‑time simulation, robotics, and interactive environments. Its Characters API creates autonomous video agents from a single image. Ideal for filmmakers, architects, game developers, and educators.

VModel provides a unified REST API that lets developers deploy and run custom or community‑built models with a single line of code. It supports Node.js, Python, and cURL for image, text, and video tasks, automatically scaling for production workloads.

Anomalo automates data quality across structured, semi‑structured, and unstructured data in cloud lakes and warehouses. Using unsupervised ML, it detects anomalies, validates completeness, enforces governance without code, and offers lineage mapping and KPI tracking.

Parea AI tracks LLM calls, logs cost, latency, and quality, and lets teams create evaluation sets and annotate data in one UI. It offers SDKs and connectors for OpenAI, Anthropic, LangChain, and LiteLLM, enabling continuous observability and prompt testing.

Rolemodel.ai is an AI tool that creates custom avatars and conversational AI assistants to enhance personal growth and productivity. It uses GPT-4 technology and provides expert guidance and resources for its users.

Testbook AI is a Chrome extension that records and replays user interactions to create regression and UI comparison tests without coding. It supports cross‑browser validation, integrates with Sauce Labs, BrowserStack, LambdaTest, exports code, and auto‑heals scripts.

Unsloth Studio is a no-code web UI enabling local training, running, and exporting of open AI models like Qwen3.5 and NVIDIA Nemotron 3, simplifying experimentation for users without extensive technical expertise.

ModelsLab offers API‑based generative AI for image, video, audio, and language tasks, including editing, generation, and voice synthesis. It supports GPU server deployment, custom workflows, fine‑tuning, and LoRA adaptation for creators and developers.

Spark Beta by Mixpanel is an AI tool that uses natural language processing to provide insights on product, marketing, and revenue questions. It offers efficient report generation and CEO insights, while simplifying data management for better decision-making.

42Signals AI delivers real‑time e‑commerce intelligence, tracking product listings, pricing, and search performance across major marketplaces. It monitors unauthorized sellers, provides price alerts, and analyzes customer reviews to inform inventory and marketing decisions.

Respan offers AI observability by tracing prompts, tool calls, and responses, enabling end‑to‑end debugging, evaluation with human, code, and LLM reviews, and real‑time monitoring for quality, cost, and compliance, and deployment orchestration across multiple cloud providers.

Graphite Note is a user-friendly, no-code predictive analytics tool for cross-industry teams. It delivers accurate predictions (outcomes, lead conversions), analyzes customer behavior, creates personalized marketing strategies, optimizes campaigns, and forecasts demand, simplifying complex data ana

B2Metric consolidates event, transactional, and behavioral data into a single source, enabling AI‑driven segmentation, churn prediction, and LTV modeling. Real‑time funnel analytics and multichannel campaign tools optimize conversions without manual data prep.

Rargus turns app reviews, support tickets, social media, and other customer feedback into structured insights. It consolidates multi‑channel data to help product managers, UX teams, and marketers prioritize features, refine experiences, and monitor brand loyalty.

Lebesgue centralizes eCommerce data from Shopify, WooCommerce, Meta, Google, TikTok, Klaviyo, Amazon, and GA4 into a unified dashboard. It offers first‑party attribution, C‑LTV modeling, product performance, competitive benchmarking, and AI‑guided budget recommendations.

Parea AI tracks LLM calls via Python/TypeScript SDKs, letting teams evaluate models on custom data, spot regressions, iterate prompts in a playground, monitor cost, latency and quality, and collect human annotations for fine‑tuning.

Fluxguard automatically crawls complex sites, monitors HTML, PDF, and visual changes, and evaluates them against user rules. It delivers real‑time alerts via APIs or webhooks, summarizes results, and reduces manual review and risk‑monitoring workload.

Trendspid is a comprehensive trade platform offering technical analysis tools, price alerts, market scanning, backtesting strategies, raindrop charts, unusual option flow tracking, and a 7-day free trial with market data from multiple sources.

Augmeta Causal Intelligence continuously monitors funnel KPIs, identifies causal drivers, and automatically triggers corrective actions to close revenue leaks. It keeps funnel health optimal, reduces manual work, and ensures steady revenue flow with secure, scalable infrastructure.

EvalsOne is an evaluation platform for developers and researchers to assess LLM prompts, RAG, and agents using rule‑based or LLM‑based methods, human judgment, and customizable evaluators. It supports multiple APIs and integrates with major AI frameworks.

Weights & Biases is an AI developer platform that simplifies machine learning experiments with tools for tracking, visualizing, and optimizing models. It enhances workflow efficiency through interactive visualizations and collaboration features.

VWO Testing is an experimentation platform for A/B, multivariate, split‑URL, and feature tests across web, mobile, and server code. It delivers real‑time analytics, Bayesian SmartStats, AI‑generated variations, and seamless integration with major analytics tools.

LLM Arena enables users to compare multiple large language models side-by-side, analyzing features like accuracy and capabilities. It supports up to 10 models, facilitating informed decision-making for researchers and developers in selecting the right LLM for their needs.

Retalon: AI tool for retail intelligence, providing predictive analytics from planning to markdowns. Enhance profitability through trend prediction, inventory optimization, and informed decision-making for retailers. Based in Toronto, Canada.

Rerun visualizes robotics logs, converting them into training‑ready datasets. Supporting C++, Python, Rust SDKs, it offers browser and desktop viewers for zooming, filtering, and annotating recordings, built‑in dataframe queries, and a shared catalog for collaborative debugging and secure enterprise

LLM SEO Monitor tracks keyword rankings and AI-generated SERP results across ChatGPT, Claude and Gemini, highlights content gaps and ranking opportunities, provides competitor analysis, automated alerts, exportable reports and API integrations for workflow automation.

Markopolo tracks anonymous visitors, analyzes 384 signals, and predicts purchase intent. It delivers real‑time 1:1 outreach via email, SMS, WhatsApp, voice, and push, plus instant cart recovery. Supports 50+ languages, cultural intelligence, and integrates with existing systems for unified journeys.

Confident AI is an evaluation platform for assessing large language models, enabling benchmarking, unit testing, and A/B testing. It streamlines dataset management and monitoring, ensuring optimal performance and alignment with benchmarks for LLM applications.

UserWatch uses AI to auto‑generate A/B tests, feature flags, funnels, cohorts, and dashboards from defined metrics. It scans session replays to spot drop‑off points, delivers actionable insights, and can create Jira tickets, enabling rapid, data‑driven UX and revenue improvements.

Oversight centralizes financial and compliance data, delivering continuous monitoring, fraud detection, and audit automation. Machine‑learning models achieve 95% risk‑detection and 99% duplicate‑payment prevention, integrating natively with ERP and spend‑management systems.

WorkMagic automates incremental lift testing with geo‑based holdouts, integrating Shopify and other data to deliver real‑time media mix projections and budget allocation recommendations for paid channels while identifying halo effects across sales channels.

ManageBetter uses AI to automate performance reviews, offering one‑click generation, analytics, 360° feedback, milestone tracking, coaching tools, and real‑time 1:1 scheduling, cutting review time by up to 80% while centralizing data for actionable insights.

Reflection 70B is an open‑source 70 B Llama 3.1‑based model that uses real‑time reflection tuning for self‑correction. It outperforms GPT‑4o on MMLU, HumanEval, MATH, IFEval, GSM8K, supporting accurate coding, debugging, and reasoning tasks via API, with a no‑registration web interface.

Reggie - AI Language Tutor enhances speaking skills through live pronunciation correction, personalized learning, progress tracking, and fluency optimization. Engage in immersive conversations for real progress tailored to your skill level.

Breadcrumbs offers enterprise‑grade, code‑free lead scoring that pulls GTM data via OAuth, visualizes predictive insights, supports multivariate testing, and routes leads in real time to improve lead quality and conversion rates.

Plat.AI is a real‑time decision‑making engine that auto‑builds, deploys, and updates ML models without code. It offers automated preprocessing, one‑click deployment, API integration, and dashboards for performance monitoring and regulatory compliance across finance, insurance, marketing and more.

Metaview automates candidate sourcing with 24/7 AI agents, generates interview notes and scorecards, and integrates outreach sequencing. It links to ATS, CRM, and scheduling tools, offers real‑time compliance checks, analytics, and DEI insights for secure, compliant talent acquisition.

MacroMicro supplies real‑time global macro data, interactive charts, and cycle‑analysis tools for cross‑country comparison (US, China, EU, etc.). It offers recession probability, Hawk‑Dove, and Optimism indices, central‑bank filings, ETF screening, and integration with D&B GlobalView.

CEBRA compresses high‑dimensional behavioral and neural time series into low‑dimensional, interpretable embeddings, supporting supervised and self‑supervised workflows. It preserves consistency across sessions and modalities, enabling accurate cross‑species trajectory decoding and multimodal integra