Skip to content

Catalogdev-infra

Promptfoo

radarSteady momentum

Open-source CLI and library for evaluating, benchmarking, and red-teaming LLM apps.

Visit promptfoo.dev →Add to kitOpen in Noemium

ledger entry · observed by @catalog-radar

Open-source CLI and library for evaluating, benchmarking, and red-teaming LLM apps.

Radar · no verdict. Confirmed from primary sources; we have not issued ship, situational, or skip.

Price trailchangelog →

No price move in this repo yet.

Known limitations

  • Radar: not field-run in this catalog wave.
  • Community red teaming is capped at 10,000 probes per month and omits team sharing, continuous monitoring, and managed cloud.

Facts

pricing
freemium1
price note
Community plan is free forever with all core evaluation features and 10,000 red-team probes per month; Enterprise and On-Premise are custom.
free tier
yes
open source
yes
api
no
self-host
yes
category
dev-infra
evidence
radar

Sources

  1. 1promptfoo.dev[snapshot]
  2. 2promptfoo.dev/docs/[snapshot]
  3. 3promptfoo.dev/pricing/[snapshot]

No source, no number.

Edit this page on GitHub

Similar joball alternatives →

toolverdictpriceossself-host
Promptfooradarfreemiumyesyes
Modalshipfreemiumnono
OpenRoutershipfreemiumnono
Replicateshippaidnono

Facts from the catalog files — not scores you can buy. Quality and speed stay in the briefing, not in a fake 1–5 grid.

Modal
dev-infraship

Modal

Serverless GPU and compute platform for running ML and AI workloads.

APIFREE TIER

freemiumfree tier ($30/mo compute); pay-as-you-go beyond

Blueshift — gaining momentum
OpenRouter
dev-infraship

OpenRouteranchor

One API for every major model — unified billing, fallbacks and routing.

APIFREE TIER

freemiumpay-per-token pass-through plus a small fee

Blueshift — gaining momentum
Replicate
dev-infraship

Replicate

Run any community model via API — pay per second, no GPU wrangling.

API

paidpay per run; image models from ~$0.003/image

Steady momentum