Today we’re rolling out Pipelines, PromptHub’s CI/CD layer that automatically runs your evaluations on the events you care about (commits, merge requests, API calls, and more). Think GitHub Actions-style guardrails for prompts.

Key Points

  • Event-Driven & No-Code: Select any evaluators when creating a pipeline, pick the trigger events, and you’re live in minutes.
  • Rule-Based Guardrails: Set pass-rate thresholds (e.g., “≥ 80 %”) so unsafe, low-performing, or secret-leaking prompts never reach prod.
  • Dataset Benchmarking: Attach a dataset to keep every new prompt version measured against the same golden standard.
  • Real-Time Visibility: Pass/fail status lands right in the merge request, with a full run history and trend lines in the Pipelines dashboard.
  • Team-Friendly: Anyone on your team — developer, PM, or analyst — can spin up a pipeline without YAML or CLI gymnastics.
  • Available on Team & Enterprise plans.

Benefits

  • Move faster with confidence – every prompt change clears the same automated tests that caught bugs last week.
  • Automate the boring stuff – pipelines run 24/7
  • Stop costly regressions – secret leaks, profanity, or performance drops trigger an instant block instead of rollbacks.

How It Works

  • Build a pipeline: Pick the trigger events (merge requests, commits, API runs, etc) and evaluators, optionally attach a dataset, and set a sampling rate.
  • Set pass/fail rules: Define a minimum pass rate or custom logic such as “block if any secret-leak check fails.” (Enterprise plan only)
  • Let it rip: On every trigger, PromptHub runs the pipeline automatically

    • Demos

    Here is a short (~3min) demo that goes from start to finish.

    Notes

    • Available on Team and Enterprise plans
    • Slack alerts coming in the future
    Headshot of PromptHub co-founder Dan Cleary
    Dan Cleary
    Founder