Skip to content

Add actions job log buffer and profiler #866

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mattdholloway merged 46 commits into from
Aug 19, 2025
Merged

Add actions job log buffer and profiler #866

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
8a5efb9
add sliding window for actions logs
mattdholloway Aug 12, 2025
e6ef962
refactor: fix sliding
mattdholloway Aug 12, 2025
271c7c2
remove trim content
mattdholloway Aug 12, 2025
e65c0f0
only use up to 1mb of memory for logs
mattdholloway Aug 12, 2025
ade3852
update to tail lines in second pass
mattdholloway Aug 13, 2025
5a76cbd
add better memory usage calculation
mattdholloway Aug 13, 2025
8e60fb2
increase window size to 5MB
mattdholloway Aug 13, 2025
e1c3143
update test
mattdholloway Aug 13, 2025
75dc8e7
Merge branch 'main' into actions-job-log-buffer
mattdholloway Aug 13, 2025
b128e44
Merge branch 'actions-job-log-buffer' of https://github.com/github/gi…
mattdholloway Aug 13, 2025
88d16d2
Merge branch 'main' into actions-job-log-buffer
mattdholloway Aug 14, 2025
4bf84b2
Merge branch 'main' into actions-job-log-buffer
mattdholloway Aug 15, 2025
6b8f2ba
update vers
mattdholloway Aug 15, 2025
8002fbd
undo vers change
mattdholloway Aug 15, 2025
52e531e
add incremental memory tracking
mattdholloway Aug 15, 2025
8f85398
use ring buffer
mattdholloway Aug 15, 2025
0d19480
remove unused ctx param
mattdholloway Aug 15, 2025
f104e67
remove manual GC clear
mattdholloway Aug 15, 2025
9d273b9
fix cca feedback
mattdholloway Aug 15, 2025
4e43327
extract ring buffer logic to new package
mattdholloway Aug 15, 2025
2ff2d4f
handle log content processing errors and use correct param for maxjob…
mattdholloway Aug 15, 2025
c6f5f7f
fix tailing
mattdholloway Aug 15, 2025
1c1061c
account for if tailLines exceeds window size
mattdholloway Aug 15, 2025
75b8c94
add profiling thats reusable
mattdholloway Aug 15, 2025
71bfac8
remove profiler testing
mattdholloway Aug 15, 2025
a43b03c
refactor profiler: introduce safeMemoryDelta for accurate memory delt…
mattdholloway Aug 15, 2025
d9c8825
linter fixes
mattdholloway Aug 15, 2025
ec070ee
Update pkg/buffer/buffer.go
mattdholloway Aug 15, 2025
0434b7e
use flag for maxJobLogLines
mattdholloway Aug 18, 2025
fb301c6
add param passing for context window size
mattdholloway Aug 18, 2025
106d802
refactor: rename contextWindowSize to contentWindowSize for consistency
mattdholloway Aug 18, 2025
516c0f7
fix: use tailLines if bigger but only if <= 5000
mattdholloway Aug 18, 2025
6c3c31a
fix: limit tailLines to a maximum of 500 for log content download
mattdholloway Aug 18, 2025
82d4ce2
Update cmd/github-mcp-server/main.go
mattdholloway Aug 18, 2025
e40e289
Update cmd/github-mcp-server/main.go
mattdholloway Aug 18, 2025
4310db8
move profiler to internal/
mattdholloway Aug 18, 2025
e0767fe
update actions test with new profiler location
mattdholloway Aug 18, 2025
9677c07
fix: adjust buffer size limits
mattdholloway Aug 18, 2025
ed6bc2d
make line buffer 1028kb
mattdholloway Aug 18, 2025
696e5fd
fix mod path
mattdholloway Aug 18, 2025
10e1995
change test to use same buffer size as normal use
mattdholloway Aug 18, 2025
2eb2e16
improve test for non-sliding window implementation to not count empty…
mattdholloway Aug 18, 2025
47aaa01
make test memory measurement more accurate
mattdholloway Aug 18, 2025
6b910ff
Merge branch 'main' into actions-job-log-buffer
mattdholloway Aug 18, 2025
4a673c9
Merge branch 'main' into actions-job-log-buffer
mattdholloway Aug 18, 2025
556a41c
remove impossible conditional
mattdholloway Aug 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
make line buffer 1028kb
  • Loading branch information
@mattdholloway
mattdholloway committed Aug 18, 2025
commit ed6bc2dde391b2e48f2df6ccf2de7fb4460bc17c
2 changes: 1 addition & 1 deletion pkg/buffer/buffer.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func ProcessResponseAsRingBufferToEnd(httpResp *http.Response, maxJobLogLines in
writeIndex := 0

scanner := bufio.NewScanner(httpResp.Body)
scanner.Buffer(make([]byte, 0, 1024), 1024*1024)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
Comment thread
mattdholloway marked this conversation as resolved.
Comment thread
mattdholloway marked this conversation as resolved.

for scanner.Scan() {
line := scanner.Text()
Expand Down
8 changes: 4 additions & 4 deletions pkg/github/actions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ import (
"strconv"
"strings"

"github.com/github/github-mcp-server/internal/profiler"
buffer "github.com/github/github-mcp-server/pkg/buffer"
ghErrors "github.com/github/github-mcp-server/pkg/errors"
"github.com/github/github-mcp-server/pkg/profiler"
"github.com/github/github-mcp-server/pkg/translations"
"github.com/google/go-github/v74/github"
"github.com/mark3labs/mcp-go/mcp"
Expand Down Expand Up @@ -761,9 +761,9 @@ func downloadLogContent(ctx context.Context, logURL string, tailLines int, maxLi
tailLines = 1000

Copilot AI Aug 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The magic number 1000 should be defined as a named constant to improve maintainability and make the default value explicit throughout the codebase.

Suggested change
tailLines = 1000
tailLines = DefaultTailLines

Copilot uses AI. Check for mistakes.
}

bufferSize := maxLines
if tailLines > maxLines && tailLines <= 5000 {
bufferSize = tailLines
bufferSize := tailLines
if bufferSize > maxLines {
bufferSize = maxLines
}

processedInput, totalLines, httpResp, err := buffer.ProcessResponseAsRingBufferToEnd(httpResp, bufferSize)
Expand Down