Skip to content

Commit 296edf4

Browse files
authored
Replace safeinputs testing with Tavily MCP web search in smoke tests (#9750)
1 parent 7512eb4 commit 296edf4

6 files changed

Lines changed: 58 additions & 10 deletions

File tree

‎.github/workflows/smoke-claude.lock.yml‎

Lines changed: 15 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.github/workflows/smoke-claude.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ strict: true
2020
imports:
2121
- shared/mcp-pagination.md
2222
- shared/gh.md
23+
- shared/mcp/tavily.md
2324
network:
2425
allowed:
2526
- defaults
@@ -64,7 +65,7 @@ timeout-minutes: 10
6465
1. **GitHub MCP Testing**: Review the last 2 merged pull requests in ${{ github.repository }}
6566
2. **Serena Go Testing**: Use the `serena-go` tool to run a basic go command like "go version" to verify the tool is available
6667
3. **Playwright Testing**: Use playwright to navigate to https://github.com and verify the page title contains "GitHub"
67-
4. **Safe Input gh Tool Testing**: Use the `safeinputs-gh` tool to run "gh issues list --limit 3" to verify the tool can access GitHub issues
68+
4. **Tavily Web Search Testing**: Use the Tavily MCP server to perform a web search for "GitHub Agentic Workflows" and verify that results are returned with at least one item
6869
5. **File Writing Testing**: Create a test file `/tmp/gh-aw/agent/smoke-test-claude-${{ github.run_id }}.txt` with content "Smoke test passed for Claude at $(date)" (create the directory if it doesn't exist)
6970
6. **Bash Tool Testing**: Execute bash commands to verify file creation was successful (use `cat` to read the file back)
7071

‎.github/workflows/smoke-codex.lock.yml‎

Lines changed: 17 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎.github/workflows/smoke-codex.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ engine: codex
1616
strict: true
1717
imports:
1818
- shared/gh.md
19+
- shared/mcp/tavily.md
1920
network:
2021
allowed:
2122
- defaults
@@ -60,7 +61,7 @@ timeout-minutes: 10
6061
1. **GitHub MCP Testing**: Review the last 2 merged pull requests in ${{ github.repository }}
6162
2. **Serena Go Testing**: Use the `serena-go` tool to run a basic go command like "go version" to verify the tool is available
6263
3. **Playwright Testing**: Use playwright to navigate to https://github.com and verify the page title contains "GitHub"
63-
4. **Safe Input gh Tool Testing**: Use the `safeinputs-gh` tool to run "gh issues list --limit 3" to verify the tool can access GitHub issues
64+
4. **Tavily Web Search Testing**: Use the Tavily MCP server to perform a web search for "GitHub Agentic Workflows" and verify that results are returned with at least one item
6465
5. **File Writing Testing**: Create a test file `/tmp/gh-aw/agent/smoke-test-codex-${{ github.run_id }}.txt` with content "Smoke test passed for Codex at $(date)" (create the directory if it doesn't exist)
6566
6. **Bash Tool Testing**: Execute bash commands to verify file creation was successful (use `cat` to read the file back)
6667

0 commit comments

Comments
 (0)