Skip to content

Commit 049118f

Browse files
kszucsjonkeane
kszucs
authored and
jonkeane
committed
ARROW-15133: [CI] Remove util_checkout.sh and util_cleanup.sh scripts
- `ci/scripts/util_checkout.sh` was used to checkout submodules because `actions/checkout@v2` has removed support for that, but they have restored it since. - `ci/scripts/util_cleanup.sh` was used to free up disk space on github actions runners, because at that time it was limited to 7GB, from a recent run it looks like the linux runners now have 32GB free space so we can try to disable the cleanup step sparing almost a minute of build time Closes apache#11974 from kszucs/checkout Authored-by: Krisztián Szűcs <szucs.krisztian@gmail.com> Signed-off-by: Jonathan Keane <jkeane@gmail.com>
1 parent 81c8a0e commit 049118f

31 files changed

Lines changed: 89 additions & 431 deletions

‎.github/workflows/archery.yml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,8 +50,6 @@ jobs:
5050
- name: Git Fixup
5151
shell: bash
5252
run: git branch master origin/master || true
53-
- name: Free Up Disk Space
54-
run: ci/scripts/util_cleanup.sh
5553
- name: Setup Python
5654
uses: actions/setup-python@v1
5755
with:

‎.github/workflows/cpp.yml‎

Lines changed: 7 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -70,10 +70,7 @@ jobs:
7070
uses: actions/checkout@v2
7171
with:
7272
fetch-depth: 0
73-
- name: Fetch Submodules and Tags
74-
run: ci/scripts/util_checkout.sh
75-
- name: Free Up Disk Space
76-
run: ci/scripts/util_cleanup.sh
73+
submodules: recursive
7774
- name: Cache Docker Volumes
7875
uses: actions/cache@v2
7976
with:
@@ -108,6 +105,7 @@ jobs:
108105
uses: actions/checkout@v2
109106
with:
110107
fetch-depth: 0
108+
submodules: recursive
111109
- name: Check CMake presets
112110
run: |
113111
cd cpp
@@ -148,9 +146,7 @@ jobs:
148146
uses: actions/checkout@v2
149147
with:
150148
fetch-depth: 0
151-
- name: Fetch Submodules and Tags
152-
shell: bash
153-
run: ci/scripts/util_checkout.sh
149+
submodules: recursive
154150
- name: Install Dependencies
155151
shell: bash
156152
run: |
@@ -241,13 +237,10 @@ jobs:
241237
uses: actions/checkout@v2
242238
with:
243239
fetch-depth: 0
244-
- name: Fetch Submodules and Tags
245-
shell: bash
246-
run: ci/scripts/util_checkout.sh
240+
submodules: recursive
247241
- name: Build
248242
shell: bash
249-
run: |
250-
ci/scripts/cpp_build.sh $(pwd) $(pwd)/build
243+
run: ci/scripts/cpp_build.sh $(pwd) $(pwd)/build
251244
- name: Test
252245
shell: bash
253246
run: ci/scripts/cpp_test.sh $(pwd) $(pwd)/build
@@ -305,17 +298,14 @@ jobs:
305298
uses: actions/checkout@v2
306299
with:
307300
fetch-depth: 0
308-
- name: Fetch Submodules and Tags
309-
shell: bash
310-
run: ci/scripts/util_checkout.sh
301+
submodules: recursive
311302
- uses: msys2/setup-msys2@v2
312303
with:
313304
msystem: MINGW${{ matrix.mingw-n-bits }}
314305
update: true
315306
- name: Setup MSYS2
316307
shell: msys2 {0}
317-
run: |
318-
ci/scripts/msys2_setup.sh cpp
308+
run: ci/scripts/msys2_setup.sh cpp
319309
- name: Cache ccache
320310
uses: actions/cache@v2
321311
with:

‎.github/workflows/csharp.yml‎

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -53,9 +53,6 @@ jobs:
5353
uses: actions/checkout@v2
5454
with:
5555
fetch-depth: 0
56-
- name: Fetch Submodules and Tags
57-
shell: bash
58-
run: ci/scripts/util_checkout.sh
5956
- name: Install Source Link
6057
shell: bash
6158
run: dotnet tool install --global sourcelink
@@ -84,9 +81,6 @@ jobs:
8481
uses: actions/checkout@v2
8582
with:
8683
fetch-depth: 0
87-
- name: Fetch Submodules and Tags
88-
shell: bash
89-
run: ci/scripts/util_checkout.sh
9084
- name: Install Source Link
9185
run: dotnet tool install --global sourcelink
9286
- name: Build
@@ -117,9 +111,6 @@ jobs:
117111
- name: Install Source Link
118112
shell: bash
119113
run: dotnet tool install --global sourcelink
120-
- name: Fetch Submodules and Tags
121-
shell: bash
122-
run: ci/scripts/util_checkout.sh
123114
- name: Build
124115
shell: bash
125116
run: ci/scripts/csharp_build.sh $(pwd)

‎.github/workflows/dev.yml‎

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -41,10 +41,6 @@ jobs:
4141
uses: actions/checkout@v2
4242
with:
4343
fetch-depth: 0
44-
- name: Fetch Submodules and Tags
45-
run: ci/scripts/util_checkout.sh
46-
- name: Free Up Disk Space
47-
run: ci/scripts/util_cleanup.sh
4844
- name: Setup Python
4945
uses: actions/setup-python@v1
5046
with:
@@ -75,9 +71,6 @@ jobs:
7571
uses: actions/checkout@v2
7672
with:
7773
fetch-depth: 0
78-
- name: Fetch Submodules and Tags
79-
shell: bash
80-
run: ci/scripts/util_checkout.sh
8174
- name: Install Python
8275
uses: actions/setup-python@v1
8376
with:
@@ -96,5 +89,4 @@ jobs:
9689
ci/scripts/release_test.sh $(pwd)
9790
- name: Run Merge Script Test
9891
shell: bash
99-
run: |
100-
pytest -v dev/test_merge_arrow_pr.py
92+
run: pytest -v dev/test_merge_arrow_pr.py

‎.github/workflows/docs.yml‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,6 @@ jobs:
5353
uses: actions/checkout@v2
5454
with:
5555
fetch-depth: 0
56-
- name: Fetch Submodules and Tags
57-
run: ci/scripts/util_checkout.sh
58-
- name: Free Up Disk Space
59-
run: ci/scripts/util_cleanup.sh
6056
- name: Cache Docker Volumes
6157
uses: actions/cache@v2
6258
with:

‎.github/workflows/go.yml‎

Lines changed: 1 addition & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -58,10 +58,6 @@ jobs:
5858
uses: actions/checkout@v2
5959
with:
6060
fetch-depth: 0
61-
- name: Fetch Submodules and Tags
62-
run: ci/scripts/util_checkout.sh
63-
- name: Free Up Disk Space
64-
run: ci/scripts/util_cleanup.sh
6561
- name: Setup Python
6662
uses: actions/setup-python@v1
6763
with:
@@ -91,10 +87,6 @@ jobs:
9187
uses: actions/checkout@v2
9288
with:
9389
fetch-depth: 0
94-
- name: Fetch Submodules and Tags
95-
run: ci/scripts/util_checkout.sh
96-
- name: Free Up Disk Space
97-
run: ci/scripts/util_cleanup.sh
9890
- name: Setup Python
9991
uses: actions/setup-python@v1
10092
with:
@@ -125,10 +117,6 @@ jobs:
125117
uses: actions/checkout@v2
126118
with:
127119
fetch-depth: 0
128-
- name: Fetch Submodules and Tags
129-
run: ci/scripts/util_checkout.sh
130-
- name: Free Up Disk Space
131-
run: ci/scripts/util_cleanup.sh
132120
- name: Setup Python
133121
uses: actions/setup-python@v1
134122
with:
@@ -160,9 +148,6 @@ jobs:
160148
uses: actions/checkout@v2
161149
with:
162150
fetch-depth: 0
163-
- name: Fetch Submodules and Tags
164-
shell: bash
165-
run: ci/scripts/util_checkout.sh
166151
- name: Build
167152
shell: bash
168153
run: ci/scripts/go_build.sh .
@@ -188,9 +173,6 @@ jobs:
188173
uses: actions/checkout@v2
189174
with:
190175
fetch-depth: 0
191-
- name: Fetch Submodules and Tags
192-
shell: bash
193-
run: ci/scripts/util_checkout.sh
194176
- name: Build
195177
shell: bash
196178
run: ci/scripts/go_build.sh .
@@ -218,9 +200,6 @@ jobs:
218200
uses: actions/checkout@v2
219201
with:
220202
fetch-depth: 0
221-
- name: Fetch Submodules and Tags
222-
shell: bash
223-
run: ci/scripts/util_checkout.sh
224203
- name: Brew Install Arrow
225204
shell: bash
226205
run: brew install apache-arrow
@@ -257,9 +236,6 @@ jobs:
257236
uses: actions/checkout@v2
258237
with:
259238
fetch-depth: 0
260-
- name: Fetch Submodules and Tags
261-
shell: bash
262-
run: ci/scripts/util_checkout.sh
263239
- uses: msys2/setup-msys2@v2
264240
with:
265241
msystem: MINGW${{ matrix.mingw-n-bits }}
@@ -273,7 +249,7 @@ jobs:
273249
run: |
274250
echo "CGO_CPPFLAGS=-I$(cygpath --windows ${MINGW_PREFIX}/include)" >> $GITHUB_ENV
275251
echo "CGO_LDFLAGS=-g -O2 -L$(cygpath --windows ${MINGW_PREFIX}/lib) -L$(cygpath --windows ${MINGW_PREFIX}/bin)" >> $GITHUB_ENV
276-
echo "GOROOT=$(cygpath --windows ${MINGW_PREFIX}/lib/go)" >> $GITHUB_ENV
252+
echo "GOROOT=$(cygpath --windows ${MINGW_PREFIX}/lib/go)" >> $GITHUB_ENV
277253
echo "GOPATH=$(cygpath --windows ${HOME}/gopath)" >> $GITHUB_ENV
278254
mkdir -p $(cygpath --windows ${HOME}/gopath)
279255
echo "MINGW_PREFIX=$(cygpath --windows ${MINGW_PREFIX})" >> $GITHUB_ENV

‎.github/workflows/integration.yml‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -62,15 +62,12 @@ jobs:
6262
uses: actions/checkout@v2
6363
with:
6464
fetch-depth: 0
65-
- name: Fetch Submodules and Tags
66-
run: ci/scripts/util_checkout.sh
65+
submodules: recursive
6766
- name: Checkout Arrow Rust
6867
uses: actions/checkout@v2
6968
with:
7069
repository: apache/arrow-rs
7170
path: rust
72-
- name: Free Up Disk Space
73-
run: ci/scripts/util_cleanup.sh
7471
- name: Cache Docker Volumes
7572
uses: actions/cache@v2
7673
with:

‎.github/workflows/java.yml‎

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -64,12 +64,7 @@ jobs:
6464
uses: actions/checkout@v2
6565
with:
6666
fetch-depth: 0
67-
- name: Fetch Submodules and Tags
68-
shell: bash
69-
run: ci/scripts/util_checkout.sh
70-
- name: Free Up Disk Space
71-
shell: bash
72-
run: ci/scripts/util_cleanup.sh
67+
submodules: recursive
7368
- name: Cache Docker Volumes
7469
uses: actions/cache@v2
7570
with:
@@ -107,9 +102,7 @@ jobs:
107102
uses: actions/checkout@v2
108103
with:
109104
fetch-depth: 0
110-
- name: Fetch Submodules and Tags
111-
shell: bash
112-
run: ci/scripts/util_checkout.sh
105+
submodules: recursive
113106
- name: Build
114107
shell: bash
115108
run: ci/scripts/java_build.sh $(pwd) $(pwd)/build

‎.github/workflows/java_jni.yml‎

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -56,10 +56,7 @@ jobs:
5656
uses: actions/checkout@v2
5757
with:
5858
fetch-depth: 0
59-
- name: Fetch Submodules and Tags
60-
run: ci/scripts/util_checkout.sh
61-
- name: Free Up Disk Space
62-
run: ci/scripts/util_cleanup.sh
59+
submodules: recursive
6360
- name: Cache Docker Volumes
6461
uses: actions/cache@v2
6562
with:
@@ -89,10 +86,7 @@ jobs:
8986
uses: actions/checkout@v2
9087
with:
9188
fetch-depth: 0
92-
- name: Fetch Submodules and Tags
93-
run: ci/scripts/util_checkout.sh
94-
- name: Free Up Disk Space
95-
run: ci/scripts/util_cleanup.sh
89+
submodules: recursive
9690
- name: Cache Docker Volumes
9791
uses: actions/cache@v2
9892
with:

‎.github/workflows/js.yml‎

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -51,10 +51,6 @@ jobs:
5151
uses: actions/checkout@v2
5252
with:
5353
fetch-depth: 0
54-
- name: Fetch Submodules and Tags
55-
run: ci/scripts/util_checkout.sh
56-
- name: Free Up Disk Space
57-
run: ci/scripts/util_cleanup.sh
5854
- name: Setup Python
5955
uses: actions/setup-python@v1
6056
with:
@@ -85,9 +81,6 @@ jobs:
8581
uses: actions/checkout@v2
8682
with:
8783
fetch-depth: 0
88-
- name: Fetch Submodules and Tags
89-
shell: bash
90-
run: ci/scripts/util_checkout.sh
9184
- name: Install NodeJS
9285
uses: actions/setup-node@v1
9386
with:
@@ -111,7 +104,7 @@ jobs:
111104
- name: Checkout Arrow
112105
uses: actions/checkout@v1
113106
with:
114-
submodules: true
107+
fetch-depth: 0
115108
- name: Install NodeJS
116109
uses: actions/setup-node@v1
117110
with:

0 commit comments

Comments
 (0)