Skip to content

[Bug]: ZipFS doesn't implement throwIfNoEntry which is slowing down tsc #4283

Closed
#4475
Closed
[Bug]: ZipFS doesn't implement throwIfNoEntry which is slowing down tsc#4283
#4475
Assignees
merceyz
Labels
bugSomething isn't workingupholdedReal issues without formal reproduction

Description

@vire
vire
opened on Mar 28, 2022

Self-service

  • I'd be willing to implement a fix

Describe the bug

When running tsc or eslint bins with typescript aware rules, this issue causes typescript program to resolve files for significant amount of time.

I did profile the process and flamegraph boils down to makeError function in pnp.cjs

image

see below on screenshot (the command ran for 25 seconds, but the reporting of non-existing files took most of the time)

image

when I console.log in makeError fn I get

rg ENOTDIR out | wc -l 
   230

and

rg ENOENT out | wc -l 
   59851

can be related to #2120

To reproduce

I could try to recreate a repo to reproduce but it would take some time

Environment

System:
    OS: macOS 12.1
    CPU: (8) arm64 Apple M1 Pro
    Memory: 86.58 MB / 16.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.14.0 - /private/var/folders/gr/8m03w_x95hxct7mpn9_gx8f40000gn/T/xfs-c7a7be7d/node
    Yarn: 3.2.0 - /private/var/folders/gr/8m03w_x95hxct7mpn9_gx8f40000gn/T/xfs-c7a7be7d/yarn
    npm: 8.3.1 - /var/folders/gr/8m03w_x95hxct7mpn9_gx8f40000gn/T/fnm_multishells/47019_1648458043892/bin/npm
  Managers:
    Homebrew: 3.4.3 - /opt/homebrew/bin/brew
    pip3: 22.0.2 - /opt/homebrew/bin/pip3
    RubyGems: 3.0.3.1 - /usr/bin/gem
  Utilities:
    Make: 3.81 - /usr/bin/make
    GCC: 4.2.1 - /usr/bin/gcc
    Git: 2.32.0 - /usr/bin/git
    Clang: 13.0.0 - /usr/bin/clang

Additional context

No response

Activity

  1. vire
    added
    bugSomething isn't working
    on Mar 28, 2022

  2. arcanis commented on Mar 28, 2022

    @arcanis
    arcanis
    on Mar 28, 2022
    Member

    That doesn't seem something about Yarn - TS just does a lot of "trial and error" searches in its resolution.

  3. arcanis
    added
    waiting for feedbackWill autoclose in a while unless more data are provided
    on Mar 28, 2022

  4. merceyz commented on Mar 28, 2022

    @merceyz
    merceyz
    on Mar 28, 2022
    Member

    We can fix this by implementing throwIfNoEntry which TypeScript uses to avoid the overhead of creating Errors, it's something I intend to implement but I haven't gotten around to it yet.

  5. merceyz
    added
    upholdedReal issues without formal reproduction
    and removed
    waiting for feedbackWill autoclose in a while unless more data are provided
    on Mar 28, 2022
  6. merceyz
    changed the title [-][performance]: pnp ENOTDIR and ENOTDIR making typescript process slow[/-] [+][Bug]: PnP doesn't implement `throwIfNoEntry` which is slowing down `tsc`[/+] on Mar 28, 2022

  7. vire commented on Mar 28, 2022

    @vire
    vire
    on Mar 28, 2022
    Author

    @arcanis @merceyz I did observe a really interesting behavior

    having a workspaces monorepo and a single folder with own package.json and tsconfig.json, the time required for lint/compile is 1/2 compared when the root tsconfig.json is used + the amount of ENOTDIR and ENOENT is lower, therefore less calls to makeError

    rg ENOTDIR out2 | wc -l
     114
rg ENOENT out2 | wc -l
   44116

    I was thinking this issue relates to pnp feature when the TS calls to locate types in node_modules takes too long and eats a lot of CPU

  8. vire commented on Mar 28, 2022

    @vire
    vire
    on Mar 28, 2022
    Author

    an excerpt of the errors

    ENOENT stat '/node_modules/@types/node
ENOENT stat '/node_modules/@types/react.d.ts'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT stat '/node_modules/lodash/isEqual'
ENOENT stat '/node_modules/lodash/isEqual.ts'
ENOENT stat '/node_modules/lodash/isEqual.tsx'
ENOENT stat '/node_modules/lodash/isEqual.d.ts'
ENOENT stat '/node_modules/@types/lodash/isEqual'
ENOENT stat '/node_modules/@types/lodash/isEqual.ts'
ENOENT stat '/node_modules/@types/lodash/isEqual.tsx'
ENOENT stat '/node_modules/eslint-import-resolver-typescript/lib/cjs'
ENOENT open '/node_modules/chalk/source/package.json'
    • other 40k+ lines
  9. merceyz
    self-assigned this
    on May 20, 2022
  10. merceyz
    changed the title [-][Bug]: PnP doesn't implement `throwIfNoEntry` which is slowing down `tsc`[/-] [+][Bug]: `ZipFS` doesn't implement `throwIfNoEntry` which is slowing down `tsc`[/+] on May 20, 2022
  11. merceyz
    closed this as completedin #4475on Jul 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingupholdedReal issues without formal reproduction

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions