Skip to content
This repository was archived by the owner on Sep 9, 2025. It is now read-only.

Storage testing #303

Merged
hendrikvanantwerpen merged 13 commits into from
Jul 28, 2023
Merged

Storage testing #303

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
a9ab7d2
Configure benchmarks LFS
Jul 28, 2023
bcfa9ed
Use LFS for performance benchmark
Jul 28, 2023
3114ca7
End-to-end CLI test
Jul 28, 2023
5f12b91
Fix workflow
Jul 28, 2023
98d46ea
Cancel workflow runs on new push
Jul 28, 2023
3890d7a
Fix typo
Jul 28, 2023
f9907a1
Do not serialize jobs for quicker results
Jul 28, 2023
b6a059b
Fix path
Jul 28, 2023
a7e8b22
Keep using checkout for base branch
Jul 28, 2023
06f8c28
Use curated data sets for CI
Jul 28, 2023
46bca31
Fix CLI test
Jul 28, 2023
8d37445
Cleanup perf workflow
Jul 28, 2023
342d987
force-perf-test
Jul 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Use curated data sets for CI
  • Loading branch information
Hendrik van Antwerpen
Hendrik van Antwerpen committed Jul 28, 2023
commit 06f8c28037563278958a83cd99fc0788fed915ef
4 changes: 3 additions & 1 deletion .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
benchmarks/** filter=lfs diff=lfs merge=lfs -text
data/*.tar.gz filter=lfs diff=lfs merge=lfs -text
data/*.tgz filter=lfs diff=lfs merge=lfs -text
data/*.zip filter=lfs diff=lfs merge=lfs -text

@rewinfrey rewinfrey Jul 28, 2023

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still not clear on when LFS is required vs not required in local dev. Does this require interacting with LFS for local dev in anyway?

@hendrikvanantwerpen hendrikvanantwerpen Jul 28, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question 🤔 I think when you change things matching the patterns, you'll have to. But they're currently only used in CI, so in most cases I don't think so. But I'll try this out and otherwise open a follow-up PR updating docs around this. (An external contributor wants to address the serialization bug, so merging this will help set them up easier.)

35 changes: 18 additions & 17 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ jobs:
matrix:
rust: [stable]
env:
TEST_NAME: typeorm-0.3.17
TEST_NAME: typescript_minimal_project
TSSG_TS: tree-sitter-stack-graphs-typescript

steps:
Expand All @@ -147,26 +147,27 @@ jobs:
- name: Build
run: cargo build --release -p ${{ env.TSSG_TS }} --features cli
- name: Checkout benchmark code
run: unzip benchmarks/${{ env.TEST_NAME }}.zip
run: unzip data/${{ env.TEST_NAME }}.zip
- name: Index
run: |
target/release/${{ env.TSSG_TS }} index -D ${{ env.TEST_NAME }}.sqlite --hide-error-details \
${{ env.TEST_NAME }}/package.json \
${{ env.TEST_NAME }}/tsconfig.json \
${{ env.TEST_NAME }}/src/data-source/ \
${{ env.TEST_NAME }}/src/entity-manager/
target/release/${{ env.TSSG_TS }} index \
-D ${{ env.TEST_NAME }}.sqlite \
--hide-error-details \
${{ env.TEST_NAME }}
- name: Index (forced)
run: |
target/release/${{ env.TSSG_TS }} index -D ${{ env.TEST_NAME }}.sqlite --hide-error-details \
${{ env.TEST_NAME }}/package.json \
${{ env.TEST_NAME }}/tsconfig.json \
${{ env.TEST_NAME }}/src/data-source/ \
${{ env.TEST_NAME }}/src/entity-manager/
target/release/${{ env.TSSG_TS }} index \
-D ${{ env.TEST_NAME }}.sqlite \
--hide-error-details \
${{ env.TEST_NAME }}
- name: Query
run: |
target/release/${{ env.TSSG_TS }} query -D ${{ env.TEST_NAME }}.sqlite definition \
${{ env.TEST_NAME }}/src/data-source/DataSource.ts:6:10 \
${{ env.TEST_NAME }}/src/data-source/DataSource.ts:84:23
- name: Query
target/release/${{ env.TSSG_TS }} query \
-D ${{ env.TEST_NAME }}.sqlite \
definition \
${{ env.TEST_NAME }}/index.ts:3:10
- name: Status
run: |
target/release/${{ env.TSSG_TS }} status -D ${{ env.TEST_NAME }}.sqlite --all
target/release/${{ env.TSSG_TS }} status \
-D ${{ env.TEST_NAME }}.sqlite \
--all
8 changes: 4 additions & 4 deletions .github/workflows/perf.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ env:
HEAD_DIR: head
HEAD_DB: head.sqlite
HEAD_ARTIFACT: head-perf-results
TEST_NAME: TypeScript-4.9.5
TEST_NAME: typescript_benchmark
TEST_REPO: microsoft/TypeScript
TEST_REF: v4.9.5
TEST_SRC: src/compiler
TEST_SRC: src/compiler/utilities.ts
MASSIF_OUT: perf.out
MASSIF_REPORT: perf.txt
TSSG_TS: tree-sitter-stack-graphs-typescript
Expand Down Expand Up @@ -223,15 +223,15 @@ jobs:
- name: Checkout benchmark code
if: steps.cache-head-result.outputs.cache-hit != 'true'
run: |
unzip ${{ env.HEAD_DIR }}/benchmarks/${{ env.TEST_NAME }}.zip
unzip ${{ env.HEAD_DIR }}/data/${{ env.TEST_NAME }}.zip
- name: Profile head memory
if: steps.cache-head-result.outputs.cache-hit != 'true'
run: |
valgrind \
--tool=massif \
--massif-out-file=${{ env.MASSIF_OUT }} \
${{ env.HEAD_DIR }}/target/release/${{ env.TSSG_TS }} \
index -D ${{ env.HEAD_DB }} --max-file-time=30 --hide-error-details -- ${{ env.TEST_NAME }}/${{ env.TEST_SRC }}
index -D ${{ env.HEAD_DB }} --max-file-time=30 --hide-error-details -- ${{ env.TEST_NAME }}
ms_print ${{ env.MASSIF_OUT }} > ${{ env.MASSIF_REPORT }}
#
# Upload results
Expand Down
3 changes: 0 additions & 3 deletions benchmarks/TypeScript-4.9.5.zip
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions benchmarks/typeorm-0.3.17.zip
View file
Open in desktop

This file was deleted.

32 changes: 32 additions & 0 deletions data/typescript_benchmark.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Performance Benchmark

A set of files, taken from the TypeScript compiler v4.9.5, of different sizes:

```
15 text files.
15 unique files.
0 files ignored.

github.com/AlDanial/cloc v 1.96 T=0.12 s (121.5 files/s, 250225.1 lines/s)
---------------------------------------------------------------------------------------------
File blank comment code
---------------------------------------------------------------------------------------------
typescript_benchmark/types.ts 1085 1548 6642
typescript_benchmark/nodeFactory.ts 598 758 5544
typescript_benchmark/es2015.ts 477 1082 2861
typescript_benchmark/generators.ts 301 903 1985
typescript_benchmark/module.ts 173 394 1437
typescript_benchmark/system.ts 204 546 1195
typescript_benchmark/resolutionCache.ts 112 101 920
typescript_benchmark/sourcemap.ts 99 47 614
typescript_benchmark/utilities.ts 64 92 371
typescript_benchmark/emitNode.ts 35 87 172
typescript_benchmark/es5.ts 10 40 72
typescript_benchmark/node.ts 14 4 66
typescript_benchmark/builderPublic.ts 8 117 56
typescript_benchmark/es2019.ts 4 1 33
typescript_benchmark/builderStatePublic.ts 1 0 13
---------------------------------------------------------------------------------------------
SUM: 3185 5720 21981
---------------------------------------------------------------------------------------------
```
3 changes: 3 additions & 0 deletions data/typescript_benchmark.zip
View file
Open in desktop
Git LFS file not shown
3 changes: 3 additions & 0 deletions data/typescript_minimal_project.zip
View file
Open in desktop
Git LFS file not shown