Repository navigation
Delete unused CodeQL bundles from the toolcache before downloading #4124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
ceb85f2
762a5ed
4a0b22e
a7a90f3
2681b03
1331773
e13c3dc
31da345
034db72
deece8f
40a6b38
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
`deleteToolcacheBundles` removes `$RUNNER_TOOL_CACHE/CodeQL` and reports which versions were there. It refuses to follow a symlinked CodeQL directory so that it can only ever delete paths that are really inside the toolcache, and reports failures rather than throwing. Not called yet. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Loading branch information
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -197,16 +197,95 @@ async function downloadAndExtractZstdWithStreaming( | |
| await tar.extractTarZst(response, dest, tarVersion, logger); | ||
| } | ||
|
|
||
| /** Gets the path to the toolcache directory for the specified version of the CodeQL tools. */ | ||
| export function getToolcacheDirectory(version: string): string { | ||
| /** Gets the path to the toolcache directory that holds all versions of the CodeQL tools. */ | ||
| function getToolcacheToolDirectory(): string { | ||
| return path.join( | ||
| getRequiredEnvParam("RUNNER_TOOL_CACHE"), | ||
| TOOLCACHE_TOOL_NAME, | ||
| semver.clean(version) || version, | ||
| ); | ||
| } | ||
|
|
||
| /** Gets the name of the toolcache directory that holds the given version of the CodeQL tools. */ | ||
| function getToolcacheVersionDirectoryName(version: string): string { | ||
| return semver.clean(version) || version; | ||
| } | ||
|
|
||
| /** Gets the path to the toolcache directory for the specified version of the CodeQL tools. */ | ||
| export function getToolcacheDirectory(version: string): string { | ||
| return path.join( | ||
| getToolcacheToolDirectory(), | ||
| getToolcacheVersionDirectoryName(version), | ||
| os.arch() || "", | ||
| ); | ||
| } | ||
|
|
||
| /** The outcome of trying to reclaim disk space by deleting the CodeQL tools from the toolcache. */ | ||
| export interface ToolcacheCleanupResult { | ||
| /** The versions of the CodeQL tools that were deleted. */ | ||
| deletedVersions: string[]; | ||
| /** | ||
| * Whether we hit an error while trying to delete the tools. Distinguishes a toolcache that had | ||
| * nothing to reclaim from one we failed to clean up. | ||
| */ | ||
| failed: boolean; | ||
| } | ||
|
|
||
| /** | ||
| * Deletes the CodeQL tools from the toolcache. | ||
| * | ||
| * Only safe to call when we are about to download the tools, since that means we did not resolve | ||
| * them from the toolcache and so nothing in there is in use by this job. | ||
| * | ||
| * This only ever touches the CodeQL directory of the toolcache, and is best-effort: any failure is | ||
| * logged rather than propagated, since the caller can proceed without the disk space. | ||
| * | ||
| * @returns the versions that were deleted, and whether we hit an error while trying. | ||
| */ | ||
| export async function deleteToolcacheBundles( | ||
| logger: Logger, | ||
| ): Promise<ToolcacheCleanupResult> { | ||
| const toolDirectory = getToolcacheToolDirectory(); | ||
|
|
||
| try { | ||
| // Refuse to follow a symlinked CodeQL directory, so that we can only ever delete paths that are | ||
| // really inside the toolcache. | ||
| if ((await fs.promises.lstat(toolDirectory)).isSymbolicLink()) { | ||
| logger.info( | ||
| `Not deleting the CodeQL tools from the toolcache since ${toolDirectory} is a symlink.`, | ||
| ); | ||
| return { deletedVersions: [], failed: true }; | ||
| } | ||
| } catch { | ||
| logger.debug( | ||
| `There are no CodeQL tools at ${toolDirectory} to delete from the toolcache.`, | ||
| ); | ||
| return { deletedVersions: [], failed: false }; | ||
| } | ||
|
|
||
| try { | ||
| const versions = ( | ||
| await fs.promises.readdir(toolDirectory, { withFileTypes: true }) | ||
| ) | ||
| .filter((entry) => entry.isDirectory()) | ||
| .map((entry) => entry.name) | ||
| .sort(); | ||
|
|
||
| await fs.promises.rm(toolDirectory, { force: true, recursive: true }); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To be a bit extra safe, do you think it would make sense to perform the symlink check for each of the specific version directories as well before deleting them individually to make sure that the specific version directories aren't symlinks?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This would be odd on a runner that claims to be GitHub-hosted, but can do. In most cases this shouldn't slow things down much as we expect exactly one CodeQL entry in the toolcache. |
||
|
|
||
| logger.info( | ||
| `Deleted the CodeQL tools at ${toolDirectory} from the toolcache to free up disk space. ` + | ||
| `Versions deleted: ${versions.join(", ") || "none"}.`, | ||
| ); | ||
|
|
||
| return { deletedVersions: versions, failed: false }; | ||
| } catch (e) { | ||
| logger.info( | ||
| `Failed to delete the CodeQL tools at ${toolDirectory} from the toolcache: ${getErrorMessage(e)}`, | ||
|
mbg marked this conversation as resolved.
Outdated
|
||
| ); | ||
| return { deletedVersions: [], failed: true }; | ||
| } | ||
| } | ||
|
|
||
| export function writeToolcacheMarkerFile( | ||
| extractedPath: string, | ||
| logger: Logger, | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.