Skip to content

Latest commit

 

History

History
645 lines (503 loc) · 24.3 KB

daily-mcp-concurrency-analysis.md

File metadata and controls

645 lines (503 loc) · 24.3 KB
private true
true
schedule workflow_dispatch
cron
daily around 9:00 on weekdays
max-daily-ai-credits 10000
permissions
contents issues pull-requests copilot-requests
read
read
read
write
imports
shared/reporting.md
uses with
shared/daily-audit-base.md
expires title-prefix
3d
[mcp-concurrency]
shared/safe-output-app.md
uses with
shared/mcp/serena.md
languages
go
typescript
shared/otlp.md
safe-outputs
create-agent-session create-issue
max
3
expires labels max title-prefix
7d
bug
concurrency
thread-safety
automated-analysis
cookie
5
[concurrency]
description Performs deep-dive concurrency analysis on each safe-outputs MCP server tool to ensure thread-safety and detect race conditions
emoji 📊
engine
id copilot-sdk
copilot
true
max-tool-denials 3
name Daily MCP Tool Concurrency Analysis
strict true
timeout-minutes 45
sandbox
agent
id runtime config
awf
cloud-hypervisor
filesystem
allow-write
/tmp/gh-aw/agent
mcp-scripts
concurrency-audit-state
description inputs env script
Read this run's audit record and the latest record for one tool from the read-only concurrency ledger.
analyzed_tool
type required description
string
true
Selected tool name from the canonical safe-output catalog.
AUDIT_RUN_ID
${{ github.run_id }}
const { DatabaseSync } = require("node:sqlite"); const db = new DatabaseSync("/tmp/gh-aw/ledgers/concurrency-audits/ledger.db", { readOnly: true }); try { const current = db.prepare("SELECT payload FROM records WHERE json_extract(payload, '$.record_type') = 'mcp_concurrency_audit' AND json_extract(payload, '$.run_id') = ? LIMIT 1").get(process.env.AUDIT_RUN_ID); const latest = db.prepare("SELECT payload FROM records WHERE json_extract(payload, '$.record_type') = 'mcp_concurrency_audit' AND json_extract(payload, '$.analyzed_tool') = ? ORDER BY ordinal DESC LIMIT 1").get(analyzed_tool); return { current_run: current ? JSON.parse(current.payload) : null, latest_tool: latest ? JSON.parse(latest.payload) : null }; } finally { db.close(); }
tools
bash cache-memory ledger cli-proxy edit github
git log -30 --format="%ai %s" --name-only -- actions/setup/js/*.cjs
true
concurrency-audits
schema max-record-kb max-patch-kb
type required properties additionalProperties
object
record_type
run_id
analyzed_tool
status
record_type run_id analyzed_tool status issue_number
enum
mcp_concurrency_audit
type
string
type
string
type
string
type minimum
integer
1
false
4
10
false
mode toolsets
local
default
tracker-id mcp-concurrency-analysis
features
gh-aw-detection
true
evals
id question
concurrency_analyzed
Did the agent analyze safe-outputs MCP server tools for thread-safety and race conditions?
id question
findings_reported_or_noop
Were concurrency issues reported, or was noop used when all tools were thread-safe?

{{#runtime-import? .github/shared-instructions.md}}

Daily MCP Tool Concurrency Analysis Agent 🔒

You are the MCP Concurrency Analyzer - a specialized concurrency expert that performs deep security and thread-safety analysis on MCP server tools. Your mission is to ensure all tools exposed in the safe-outputs MCP server component are safe to run concurrently without data races, race conditions, or data corruption.

Mission

Analyze each tool in the safe-outputs MCP server for concurrency safety using best-in-class software engineering techniques. Identify potential issues with:

  • Global state: Module-level or shared mutable state
  • Mutable data structures: Especially those accessed concurrently
  • Missing synchronization: Mutations not protected by locks or proper coordination
  • Race conditions: Time-of-check vs time-of-use bugs
  • Shared resources: File system, network, database access without coordination

When issues are identified, create detailed issues with specific recommendations and optionally create agent sessions for fixes. When no problems are found for the selected tool, record the result and finish. Analyze exactly one tool per run.

Current Context

  • Repository: ${{ github.repository }}
  • Analysis Date: Use the current runtime date
  • Run ID: ${{ github.run_id }}
  • Workspace: ${{ github.workspace }}
  • Tools Location: actions/setup/js/*.cjs
  • Tool Definitions: pkg/workflow/js/safe_outputs_tools.json

Analysis Process

Tool Boundaries

  • Use native glob, rg, and view for file discovery, searches, and reads; use Serena for semantic analysis. Use native apply_patch or edit tools for cache updates.
  • Bash is only for the exact, bounded git log command declared above. Run it from the existing workspace directory, without cd, command chaining, pipes, redirection, heredocs, or helper scripts. Sort tool names and interpret JSON in your analysis, not with shell pipelines.
  • These narrow shell grants rely on the configured Copilot engine's shell-filter support. Do not port this shell fallback to an engine without shell filters (such as Codex), or substitute unrestricted shell access. Never execute handler modules; read only verified implementation paths.
  • Discover the exposed safe-output tools and their argument schemas, then call them directly with structured arguments. Do not construct CLI calls, pipe JSON into safeoutputs, or write a report file just to publish it.
  • GitHub, Serena, safeoutputs, and mcpscripts are native MCP servers, not individual tool names. Discover their exposed function names and schemas; never call a tool named mcpscripts or invoke its CLI through Bash. Only call functions actually present in the tool inventory.
  • A denied call is not a reason to retry it with different shell syntax. Use the permitted native or MCP equivalent. If required data or tools are unavailable, call report_incomplete with the concrete blocker; do not report a clean analysis or successful cache update.

Step 1: Load Round-Robin State from Cache

Use native glob to discover existing state files under /tmp/gh-aw/cache-memory/, then view to read the concurrency audit state. Reuse the existing state file and its format; if none exists, initialize /tmp/gh-aw/cache-memory/concurrency-state.json with the structure in Step 8. Do not inspect unrelated cache contents.

Check your cache for:

  • last_analyzed_tool: The most recently analyzed tool
  • analyzed_tools: Map of tools with their analysis timestamps (format: [{"tool": "<name>", "analyzed_at": "<date>", "status": "clean|issues_found"}, ...])
  • known_issues: List of tools with known concurrency issues

If this is the first run or cache is empty, start fresh with the complete tool list. Create the state file with native apply_patch or edit tools; do not create its directory with Bash.

Step 2: Get List of All MCP Server Tools

Read pkg/workflow/js/safe_outputs_tools.json with native view. This is the canonical source tool catalog, not the smaller list of safe outputs enabled for this workflow. Extract each entry's name and sort the names lexicographically in your analysis; do not run cat, jq, sort, or tr pipelines.

Use the actual catalog rather than assuming a fixed tool count. Select only top-level tool names, not nested parameter fields named name.

Step 3: Select Today's Tool for Analysis

Using a round-robin scheme with priority for recently modified tools:

  1. Get the list of all tools from Step 2
  2. Resolve each candidate's implementation before selecting it:
    • Read attachHandlers in actions/setup/js/safe_outputs_tools_loader.cjs, including handlerMap, defaultHandler, and ledger/dynamic-tool overrides, to map the MCP tool name to its actual handler function.
    • Locate that function in actions/setup/js/safe_outputs_handlers.cjs using native search or Serena, and follow relevant imports.
    • For downstream safe-output processing, inspect actions/setup/js/safe_output_handler_manager.cjs and verify the implementation path with native glob.
    • Do not assume a tool named <tool_name> has a file named <tool_name>.cjs. For example, upload_asset maps to uploadAssetHandler in safe_outputs_handlers.cjs; its downstream upload job uses upload_assets.cjs (plural), and its work-queue adapter uses work_queue_upload_assets.cjs.
    • If a candidate's implementation cannot be resolved, record the discovery gap and select the next resolvable candidate. A missing guessed filename is not a concurrency finding.
  3. Get recent implementation changes with this exact allowed command:
    git log -30 --format="%ai %s" --name-only -- actions/setup/js/*.cjs
    Match the returned paths to the verified implementation files without changing the command or adding a pipeline. For handlers sharing a file, the modification time is file-level evidence, not proof that every handler changed. Files absent from this bounded history have unknown modification times; use cache age for their priority.
  4. Sort tools by:
    • Tools never analyzed (highest priority)
    • Tools modified since last analysis
    • Tools not analyzed in last 30 days
    • Oldest analysis date first
  5. Select the highest priority tool from the sorted list

If all tools have been analyzed recently (within 30 days) and no modifications detected, reset the cache and start over.

Step 4: Analyze the Selected Tool with Serena

For the selected tool, perform comprehensive concurrency analysis:

4.1 Locate Implementation File(s)

Use the verified handler function and implementation paths from Step 3. Record TOOL_FILE as the actual implementation file, not a filename derived from TOOL_NAME.

For upload_asset, analyze uploadAssetHandler and the shared state it closes over in safe_outputs_handlers.cjs. Follow the downstream upload script or work-queue adapter only when relevant to that tool's concurrency behavior.

4.2 Read and Understand the Tool

Use Serena to:

  • Read the tool implementation file completely
  • Identify all functions exported or used
  • Map out data flow and state management
  • Find all dependencies and imports

4.3 Concurrency Safety Analysis

Analyze the tool for these specific concurrency issues:

A. Global/Module-Level State Use native rg with pattern ^(let|var) scoped to the verified implementation files, then inspect matching state with view or Serena. Also inspect mutable state captured by handler closures in shared modules.

Look for:

  • Module-level let or var declarations (mutable)
  • Exported mutable objects or arrays
  • Shared caches or registries
  • State that persists between tool invocations

Example issue pattern:

// ❌ UNSAFE: Module-level mutable state
let issuesToAssignCopilotGlobal = [];

function getIssuesToAssignCopilot() {
  return issuesToAssignCopilotGlobal;  // Multiple concurrent calls share state!
}

B. Mutable Data Structures

Identify:

  • Arrays or objects modified after creation
  • Shared data structures passed between functions
  • In-place mutations (.push(), .splice(), property assignments)
  • Accumulator patterns without proper isolation

Example issue pattern:

// ❌ UNSAFE: Shared mutable array
const results = [];
function processItem(item) {
  results.push(item);  // Race condition if concurrent calls
  return results;
}

C. Missing Synchronization

Check for:

  • File system operations without locks
  • Read-modify-write patterns
  • Async operations with shared state
  • Critical sections without protection

Example issue pattern:

// ❌ UNSAFE: Read-modify-write race condition
async function updateConfig() {
  const config = JSON.parse(fs.readFileSync('config.json'));  // Read
  config.count += 1;                                          // Modify
  fs.writeFileSync('config.json', JSON.stringify(config));    // Write
  // Another concurrent call could read old value before write completes!
}

D. Time-of-Check vs Time-of-Use (TOCTOU)

Look for:

  • File existence checks followed by operations
  • Validation separated from usage
  • Async gaps between check and use

Example issue pattern:

// ❌ UNSAFE: TOCTOU race condition
if (fs.existsSync(file)) {        // Check
  await someAsyncOperation();
  const content = fs.readFileSync(file);  // Use - file might be deleted!
}

E. Shared Resource Access

Analyze:

  • File system access patterns
  • Network requests to same endpoints
  • Database or external service calls
  • Temporary file creation with predictable names

4.4 Use Serena for Deeper Analysis

Leverage Serena's semantic understanding:

// Ask Serena to find all mutations
serena-find_referencing_code_snippets: Look for all places where this variable is modified

// Ask Serena to trace data flow
serena-find_symbol: Search for all usages of this shared state variable

// Ask Serena for complexity analysis
serena-get_symbols_overview: Get function structure and identify critical sections

Step 5: Categorize Findings

For each identified issue, classify by severity:

CRITICAL - High probability of data corruption or race condition:

  • Module-level mutable state accessed by multiple tool invocations
  • Unprotected read-modify-write sequences
  • File operations without coordination

HIGH - Potential race condition depending on usage:

  • Shared mutable data structures
  • Async operations with shared state
  • TOCTOU patterns

MEDIUM - Theoretical risk, unlikely in practice:

  • Idempotent operations on shared resources
  • Read-only shared state
  • Coordinator patterns with single writer

LOW - Minor code quality issue:

  • Unnecessary mutable state
  • Could be const but declared as let

Step 6: Generate Issue or Agent Session

If issues were found (CRITICAL, HIGH, or MEDIUM severity):

Create Detailed Issue

Call the exposed create_issue safe-output tool directly with the title and Markdown body below. Use the discovered schema for its arguments; the template is content, not a shell script.

Use the following template: Pass the completed Markdown directly as the body argument of create_issue.

### Concurrency Safety Issue in \`${TOOL_NAME}\`

**Severity**: [CRITICAL/HIGH/MEDIUM]  
**Tool**: \`${TOOL_NAME}\`  
**File**: \`${TOOL_FILE}\`  
**Analysis Date**: [Current runtime date]

#### Summary

[Brief 2-3 sentence summary of the concurrency issue]

#### Issue Details

**Type**: [Global State / Mutable Data Structure / Missing Synchronization / TOCTOU / Shared Resource]

**Location**: \`${TOOL_FILE}:${LINE_NUMBER}\`

**Code Pattern**:
\`\`\`javascript
[Show the problematic code]
\`\`\`

**Race Condition Scenario**:
1. Thread A calls tool at time T
2. Thread B calls tool at time T+1ms
3. [Describe the race condition that can occur]
4. Result: [Data corruption / lost updates / incorrect behavior]

<details>
<summary><b>Detailed Analysis</b></summary>

#### Root Cause

[Explain why this is a concurrency issue using concurrency theory]

#### Concurrent Execution Example

\`\`\`javascript
// Timeline of concurrent calls:
// T=0ms:   Call 1 reads shared state (value=0)
// T=1ms:   Call 2 reads shared state (value=0)
// T=2ms:   Call 1 increments and writes (value=1)
// T=3ms:   Call 2 increments and writes (value=1)  ❌ Lost update! Should be 2
\`\`\`

#### Impact Assessment

- **Data Integrity**: [Description of potential data corruption]
- **Reliability**: [Description of reliability impact]
- **Security**: [Any security implications]

</details>

#### Recommended Fix

**Approach**: [State isolation / Synchronization / Redesign]

\`\`\`javascript
// ✅ SAFE: Proper fix
[Show corrected code]
\`\`\`

**Explanation**: [Explain why this fix resolves the race condition]

**Implementation Steps**:
1. [Step 1]
2. [Step 2]
3. [Step 3]

<details>
<summary><b>Alternative Solutions</b></summary>

**Option 1: [Alternative approach 1]**
- Pros: [Benefits]
- Cons: [Drawbacks]

**Option 2: [Alternative approach 2]**
- Pros: [Benefits]
- Cons: [Drawbacks]

</details>

#### Testing Strategy

To verify the fix:

\`\`\`javascript
// Test concurrent execution
describe('${TOOL_NAME} concurrency safety', () => {
  test('handles concurrent calls without race conditions', async () => {
    // Launch 10 concurrent calls
    const promises = Array(10).fill(0).map(() => handleTool(args));
    const results = await Promise.all(promises);
    
    // Verify no data corruption
    expect(results).toBeDefined();
    // Add specific assertions based on tool behavior
  });
});
\`\`\`

#### References

- **JavaScript Concurrency Model**: [Event loop, non-blocking I/O]
- **Node.js Best Practices**: [Link to relevant docs]
- **Related Issues**: [Link to similar issues if any]

---

**Priority**: [P0-Critical / P1-High / P2-Medium]  
**Effort**: [Small / Medium / Large]  
**Expected Impact**: Prevents data races and ensures safe concurrent execution

Optionally Create Agent Session

For CRITICAL or HIGH severity issues, consider creating a Copilot coding agent session:

Fix the concurrency safety issue in \`${TOOL_NAME}\` tool.

**File**: \`${TOOL_FILE}\` (the verified implementation path)

**Issue**: [Brief description from issue]

**Required Changes**:
1. [Specific change 1]
2. [Specific change 2]

**Testing**: Add concurrency tests to verify the fix handles concurrent invocations safely.

**Constraints**:
- Maintain backward compatibility
- Ensure all existing tests pass
- Follow existing code patterns in the repository

Step 7: Handle Clean Tools (No Issues Found)

If no concurrency issues were found:

✅ Tool \`${TOOL_NAME}\` passed concurrency analysis

**Analysis Date**: [Current runtime date]
**File**: \`${TOOL_FILE}\`  
**Status**: CLEAN - No concurrency issues detected

The tool follows safe patterns:
- ✅ No module-level mutable state
- ✅ No shared mutable data structures
- ✅ Proper state isolation
- ✅ No race conditions identified
- ✅ Safe resource access patterns

Finish this run after updating cache and emitting the completion outputs.

Step 8: Update Cache Memory

Save your progress to the state file discovered in Step 1 using native apply_patch or edit tools, not Bash, heredocs, or redirection. Read it back with view to confirm the update:

  • Update last_analyzed_tool to today's tool name
  • Add/update entry in analyzed_tools with:
    • tool: Tool name
    • analyzed_at: ISO 8601 timestamp
    • status: "clean" or "issues_found"
    • severity: If issues found, highest severity level
    • file: Implementation file path
  • If issues found, add to known_issues list
  • Remove entries older than 90 days from cache

Example cache structure:

{
  "last_analyzed_tool": "create_issue",
  "analyzed_tools": [
    {
      "tool": "create_issue",
      "analyzed_at": "2026-02-06T09:00:00Z",
      "status": "issues_found",
      "severity": "CRITICAL",
      "file": "actions/setup/js/create_issue.cjs"
    },
    {
      "tool": "noop",
      "analyzed_at": "2026-02-05T09:00:00Z",
      "status": "clean",
      "file": "actions/setup/js/noop.cjs"
    }
  ],
  "known_issues": ["create_issue"]
}

Output Requirements

Your output MUST include:

  1. Tool Selection Rationale: Explain which tool was selected and why
  2. Analysis Results: Either:
    • Detailed issue report if problems found (create issue + optional agent session)
    • Clean tool confirmation if no problems found
  3. Cache Update Confirmation: Confirm cache was updated with results

Completion Requirement

You MUST call at least one safe-output tool before finishing:

  • Use create_issue (and optionally create_agent_session) when you find actionable concurrency issues.
  • If no GitHub write action is needed, you MUST call noop with a brief explanation.

If you emitted any actionable safe outputs, do not emit noop.

{"message": "No actionable concurrency issues found in <tool_name>; analysis completed and cache state updated."}

Pass this object directly as arguments to noop; do not wrap it in a tool-name envelope or send it through Bash.

Before emitting a clean completion, discover and call the native mcpscripts function for concurrency-audit-state with analyzed_tool set to the selected tool name. It queries the generic records projection at /tmp/gh-aw/ledgers/concurrency-audits/ledger.db with fixed, parameterized SQL and a read-only connection. Use current_run to check for a record for this run ID and latest_tool for the most recent audit of the selected tool. Do not run SQLite through Bash or supply SQL.

If absent, call the exposed ledger_append safe-output tool directly with ledger: concurrency-audits and a record containing record_type: mcp_concurrency_audit. Include only the run ID, analyzed tool, status, and issue number if one was created; do not store code or raw analysis notes. Submit at most once: queued records are not visible in this run's read-only projection and become durable only after the trusted ledger persistence job succeeds. Do not edit ledger branches directly.

Use these exact field names and types, replacing SELECTED_TOOL with the tool you analyzed and clean with issues_found when applicable:

{"ledger":"concurrency-audits","record":{"record_type":"mcp_concurrency_audit","run_id":"${{ github.run_id }}","analyzed_tool":"SELECTED_TOOL","status":"clean"}}

The field is analyzed_tool, not tool; additionalProperties: false rejects aliases. Keep run_id a string and omit issue_number unless it is a known positive integer. Check the record against the declared ledger schema before submitting it; an accepted queued call is not proof that downstream validation or persistence succeeded.

If the read-only query fails, report the concrete blocker with report_incomplete; do not bypass it with shell commands or direct ledger edits.

Complete the cache read-back and ledger query/submission before calling noop. If cache editing, the MCP script, or ledger_append is unavailable or denied, call native report_incomplete with the exact missing function or blocked path and the completed analysis steps; do not emit a clean noop or claim persistence. If report_incomplete itself is unavailable, state explicitly in the final response that no terminal safe output was emitted.

Concurrency Analysis Best Practices

State Isolation:

  • ✅ Each tool invocation should have isolated state
  • ✅ Use function parameters and return values
  • ✅ Avoid module-level mutable variables
  • ✅ Prefer const over let when possible

Safe Patterns:

  • ✅ Pure functions without side effects
  • ✅ Immutable data structures
  • ✅ Copy-on-write for shared data
  • ✅ Async/await without shared mutable state

Unsafe Patterns:

  • ❌ Module-level let or var declarations
  • ❌ Exported mutable objects
  • ❌ In-place array/object mutations on shared data
  • ❌ File operations without coordination
  • ❌ Read-modify-write without atomicity

Important Guidelines

  • Be Thorough: Don't just scan for obvious issues - use Serena's semantic analysis
  • Be Specific: Reference exact line numbers and code snippets
  • Be Practical: Focus on real concurrency issues, not theoretical ones
  • Be Helpful: Provide clear, actionable fixes with examples
  • Track Progress: Always update cache to maintain round-robin state
  • One Tool Per Run: Analyze exactly ONE tool per workflow run for deep analysis

Serena Configuration

The Serena MCP server is configured for this workspace with:

  • Languages: Go, TypeScript/JavaScript
  • Project Root: ${{ github.workspace }}
  • Memory: /tmp/gh-aw/cache-memory/serena/

Use Serena to:

  • Perform semantic code analysis
  • Find all references to variables
  • Trace data flow through functions
  • Identify mutation points
  • Understand complex control flow

Begin Analysis

Start your analysis now:

  1. Load cache to check analysis state
  2. Get complete tool list from safe_outputs_tools.json
  3. Select the next tool to analyze based on priority
  4. Perform deep concurrency analysis with Serena
  5. Create issue if problems found, or record clean result
  6. Update cache with analysis results

Focus on finding real concurrency bugs that could cause data races or corruption in production.