feat(promptfoo): add crab pf serve Slack polling daemon - #37
Merged
Merged
Conversation
Adds a local daemon that polls for Slack DMs and runs the promptfoo target discovery agent on behalf of team members. Non-technical users can DM the Crab bot with "pf: <target spec>" and receive generated promptfooconfig.yaml files back in-thread. - New slack.ts: Slack Web API helpers using native fetch (zero deps) - New serve.ts: daemon loop with setup flow, DM polling, and job mgmt - Updated cli.ts: routes `serve` subcommand, adds --reasoning flag - Updated providers.ts: reasoningEffort support for GPT-5/o-series Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MrFlounder
force-pushed
the
feat/pf-serve-slack-daemon
branch
from
February 22, 2026 07:25
a9feb29 to
61c1703
Compare
This was referenced Mar 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Team members — including non-technical people — need a way to run the promptfoo target discovery agent without setting up a local dev environment. A Slack-based interface lets anyone DM the Crab bot with a target specification and get back a ready-to-use
promptfooconfig.yaml.What
Adds
crab pf serve, a local Slack polling daemon that:pf:trigger prefixFiles changed
plugins/promptfoo/src/slack.tsfetch(), zero new dependencies. Token resolution, user lookup, DM channel management, message posting, reactions, file download/upload (3-step flow).plugins/promptfoo/src/serve.ts--setupflow (prompts for Slack username and provider preferences), poll loop, job directory management. Defaults toopenai:gpt-5withreasoning: low.plugins/promptfoo/src/cli.tsservesubcommand, adds--reasoningflag support, updates help text.plugins/promptfoo/src/agent/providers.tsreasoningEffortsupport for GPT-5/o-series models (max_completion_tokensinstead ofmax_tokens, skipstemperaturefor reasoning models).plugins/promptfoo/package-lock.jsonHow to Test
CRAB_SLACK_BOT_TOKENenv var (or addslack.bot_tokento~/.crabcode/config.yaml)OPENAI_API_KEYenv varcrab pf serve --setupand follow prompts to configure your Slack usernamecrab pf serve -vto start the daemon in verbose modepf: My API is at http://localhost:8080/chatpromptfooconfig.yaml🤖 Generated with Claude Code