Running out of disk space on Windows 2019

Describe the bug
I am trying to build a project with CMake (links below) and one of the workers is running out of disk space in the process.

Area for Triage:
Servers

Question, Bug, or Feature?:
Bug

Virtual environments affected

Expected behavior
Not running out of disk space

Actual behavior
Running out of disk space

Here is the respective Actions log:
https://github.com/grische/blender/runs/932880160?check_suite_focus=true

and the workflow file:
https://github.com/grische/blender/blob/47d8f2e535d7892f282ae95ca64a9641078801d7/.github/workflows/blender_master_ci.yaml

Hello, could you please add the following step at the begin of your pipeline and share results?

steps:
- run: Get-PSDrive

@maxim-lobanov

@maxim-lobanov https://github.com/grische/blender/runs/933668150?check_suite_focus=true

Thank you for providing build logs.
So runner has

88.64 GB on C drive
12 GB on D drive

D drive is limited (mostly, intended for checkout step)

Based on runner documentation, we provide only 14 GB of free space on machines.

As for your particular issue, I think currently, your build uses D drive only (12 GB) and don't use C drive. I suggest reconfiguring build to build on C drive if it is possible

@maxim-lobanov

I see. Interesting that the other environments (windows-2016, Ubuntu, macOS) do not fail.

@maxim-lobanov how would you change the above workflow file to use C instead of D?

Hello, thank you for your response.
This issue is not reproduced on MacOS and Linux because they use the single drive (/).
Moving your build to C drive is tricky (it is rare case when D drive doesn't have enough free space for repository but it looks like your submodules take ~10GB):

Replace actions/checkout with git clone <repourl> C:\Repo (actions/checkout can't clone repository to C drive)
set working directory property for all further steps

@dmitry-shibanov

@dmitry-shibanov How would change the above workflow file?

The workflow file uses a matrix of Ubuntu/macOS/Windows, so I would need to create two checkout tasks, one with actions/checkout for ubuntu/macOS and a manual one for Windows?
It also already uses working-directory: ${{ github.workspace }}/build, so I would need to create a separate step to introduce a new environment variable and then hardcode it for windows to C:\Repo and use it for ubuntu/macOS like the above?

Is there really no easier way to handle this Windows build? If I could override github.workspace, this might make the steps above redundant.

@grische

Hello, @grische , unfortunately github.workspace variable can’t be overridden.
Also, as far as I know there no easier way to use C drive without significant modification of pipeline
(D drive is default for builds and officially we don’t provide more than 14 GB of free space; moving repository to drive C is just possible workaround)

Define separate job for windows (outside of test matrix)
Use git clone and specify place on disk C:/

- name: Checkout Blender
  run: |
      mkdir C:/blender.git
      git clone https://github.com/grische/blender.git C:/blender.git --depth 1

Change working directory for all next steps

- name: submodules and libraries update
  working-directory: C:/blender.git
  run: python C:/blender.git/build_files/buildbot/worker_update.py

Hello, We close the issue. But feel free to reopen it, if you have any concerns.

How do I reopen this, it doesn't work at all??!!!! I copied everything, and it constantly whines about an invalid directory, what the fuck? Why the fuck does D have so little space, why don't you transfer some from C? Is there actually a fully complete working example for this, why is it so fucking difficult to change a simple working directory? Why the fuck can't github.workspace be configured? I've been for 3 hours on this and it should've taken 5 minutes, what are you doing over there? Why hasn't this been fixed yet?

How do I reopen this, it doesn't work at all??!!!! I copied everything, and it constantly whines about an invalid directory, what the fuck? Why the fuck does D have so little space, why don't you transfer some from C? Is there actually a fully complete working example for this, why is it so fucking difficult to change a simple working directory? Why the fuck can't github.workspace be configured? I've been for 3 hours on this and it should've taken 5 minutes, what are you doing over there? Why hasn't this been fixed yet?

The invalid directory is caused by using a working-directory value when the directory does not exist yet; you need to make sure the step that performs the cloning does not set the destination directory as the working directory. Also, the cloning will not work correctly as written because it will not checkout the right branch.

Workaround Windows CI running out of disk space during tests

@ddobrev

@ddobrev you have here a fully working example (in this case for NodeJS)

# This workflow will do a clean install of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions

name: Test API server

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    strategy:
      matrix:
        platform: [ubuntu-latest, macos-latest]
        node-version: [lts/*]
        
    runs-on: ${{ matrix.platform }}

    steps:
    - uses: actions/checkout@v3
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v2
      with:
        node-version: ${{ matrix.node-version }}
        cache: 'npm'
    - name: Upgrade NPM
      run: npm install -g npm
    - run: npm ci
    - run: npm test
  
  windows:      
    runs-on: windows-latest
    steps:
      - name: Checkout
        run: |
          mkdir C:/geoapipt.git
          git clone https://github.com/jfoclpf/geoapi.pt.git C:/geoapipt.git --depth 1
      - name: Upgrade NPM
        working-directory: C:/geoapipt.git
        run: npm install -g npm
      - name: Install dependencies
        working-directory: C:/geoapipt.git
        run: npm ci
      - name: Test
        working-directory: C:/geoapipt.git
        run: npm test

Workaround Windows CI running out of disk space during tests

Build under drive C

Navigation Menu

Repository navigation

Running out of disk space on Windows 2019 #1341

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Running out of disk space on Windows 2019 #1341

Description

Activity

maxim-lobanov commented on Jul 31, 2020

grische commented on Jul 31, 2020

maxim-lobanov commented on Aug 1, 2020

grische commented on Aug 1, 2020

dmitry-shibanov commented on Aug 4, 2020

grische commented on Aug 4, 2020

dmitry-shibanov commented on Aug 5, 2020

dmitry-shibanov commented on Aug 10, 2020

ddobrev commented on Feb 18, 2021

jdm commented on Feb 25, 2021

jfoclpf commented on Nov 30, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions