Skip to content

Use the "withFileTypes" options with fs.readdir(). #14994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Eric Snow (ericsnowcurrently) merged 36 commits into from
Jan 11, 2021
Merged

Use the "withFileTypes" options with fs.readdir(). #14994

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7a68cd4
Use the "withFileTypes" options with fs.readdir().
ericsnowcurrently Dec 17, 2020
d443136
Filter only files, not directories.
ericsnowcurrently Jan 4, 2021
34be222
All "dedented" lines to be empty.
ericsnowcurrently Jan 4, 2021
f2e1bf9
Add functional tests for findInterpretersInDir().
ericsnowcurrently Jan 5, 2021
501f4d4
lint
ericsnowcurrently Jan 5, 2021
27ac913
Drop an outdated TODO.
ericsnowcurrently Jan 5, 2021
fec3e81
Factor out an "opts" arg for findInterpretersInDir().
ericsnowcurrently Jan 6, 2021
935bde5
Add the "ignoreErrors" option to findInterpretersInDir().
ericsnowcurrently Jan 6, 2021
dcd4c11
Factor out the "cfg" arg to iterExecutables().
ericsnowcurrently Jan 6, 2021
70f93bc
Do not export iterExecutables().
ericsnowcurrently Jan 6, 2021
33497f5
Add some notes to iterExecutables().
ericsnowcurrently Jan 6, 2021
ec5acb1
Fold back in the findInterpretersInDir "opts" as args..
ericsnowcurrently Jan 7, 2021
68eb0ed
Factor out matchFile().
ericsnowcurrently Jan 7, 2021
745c038
Change checkBin from an arg to a constant.
ericsnowcurrently Jan 7, 2021
45b7dfd
Add a note about recursion depth.
ericsnowcurrently Jan 7, 2021
7430fab
Clarify about "depth".
ericsnowcurrently Jan 7, 2021
8f60121
Filter the subdirs rather than the files.
ericsnowcurrently Jan 7, 2021
6bc20c7
Drop ensureFSTreeIfWindows() and ensureFSTreeIfNotWindows().
ericsnowcurrently Jan 7, 2021
91af899
Make checkBin a local variable for the sake of unit testing.
ericsnowcurrently Jan 11, 2021
2d93e99
Add a NEWS entry.
ericsnowcurrently Jan 11, 2021
e9c873c
Skip currently bogus Python executables on Windows during tests.
ericsnowcurrently Jan 11, 2021
26a4703
Fix executable order in Windows test.
ericsnowcurrently Jan 11, 2021
3b45233
Fix getInterpreterPathFromDir().
ericsnowcurrently Jan 11, 2021
d067097
Drop the file match in getInterpreterPathFromDir().
ericsnowcurrently Jan 11, 2021
348363d
Add some debugging.
ericsnowcurrently Jan 11, 2021
8f4f85b
Add more debugging.
ericsnowcurrently Jan 11, 2021
95c6dc1
Add more debugging.
ericsnowcurrently Jan 11, 2021
43c6f5b
Add more debugging.
ericsnowcurrently Jan 11, 2021
2120d98
Add more debugging.
ericsnowcurrently Jan 11, 2021
7ff9b95
Add more debugging.
ericsnowcurrently Jan 11, 2021
f9934db
Add more debugging.
ericsnowcurrently Jan 11, 2021
6eb5fde
Yield symlinks from findInterpretersInDir().
ericsnowcurrently Jan 11, 2021
88ec229
Add symlink support to ensureFSTree().
ericsnowcurrently Jan 11, 2021
a654357
Test for symlinks.
ericsnowcurrently Jan 11, 2021
c6b7ec1
Drop debugging code.
ericsnowcurrently Jan 11, 2021
246329a
lint
ericsnowcurrently Jan 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Filter the subdirs rather than the files.
  • Loading branch information
@ericsnowcurrently
ericsnowcurrently committed Jan 7, 2021
commit 8f60121826e7e946287f745db783230849186dee
19 changes: 8 additions & 11 deletions src/client/pythonEnvironments/common/commonUtils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ type FileFilterFunc = (filename: string) => boolean;
export function findInterpretersInDir(
root: string,
recurseLevel?: number,
filterFile?: FileFilterFunc,
filterSubDir?: FileFilterFunc,
ignoreErrors?: boolean,
): AsyncIterableIterator<string> {
const cfg = {
filterFile,
filterSubDir,
maxDepth: recurseLevel,
ignoreErrors: ignoreErrors || false,
};
Expand All @@ -45,7 +45,7 @@ async function* iterExecutables(
// "currentDepth" is the depth of the current level of recursion.
currentDepth: number,
cfg: {
filterFile: FileFilterFunc | undefined;
filterSubDir: FileFilterFunc | undefined;
maxDepth: number | undefined;
ignoreErrors: boolean;
},
Expand Down Expand Up @@ -78,16 +78,13 @@ async function* iterExecutables(
// to incur the extra cost of `fs.lstat()`.
if (entry.isDirectory()) {
if (cfg.maxDepth && currentDepth <= cfg.maxDepth) {
yield* iterExecutables(filename, currentDepth + 1, cfg);
if (matchFile(filename, cfg.filterSubDir, cfg.ignoreErrors)) {
yield* iterExecutables(filename, currentDepth + 1, cfg);
}
}
Comment thread
karrtikr marked this conversation as resolved.
Resolved
} else if (entry.isFile()) {
// TODO: We only need to do filtering on subdirs. Files
// can be filtered by the caller. This will be fixed in
// an upcoming change.
if (matchFile(filename, cfg.filterFile, cfg.ignoreErrors || false)) {
if (checkBin(filename)) {
yield filename;
}
if (checkBin(filename)) {
yield filename;
}
}
// We ignore all other file types, including symlinks.
Comment thread
ericsnowcurrently marked this conversation as resolved.
Resolved
Expand Down
86 changes: 35 additions & 51 deletions src/test/pythonEnvironments/common/commonUtils.functional.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ async function ensureFSTreeIfNotWindows(tree: string): Promise<void> {
}
}

suite('pyenvs common utils - findInterpretersInDir()', () => {
suite('pyenvs common utils - finding Python executables', () => {
const datadir = path.join(__dirname, '.data');

function resolveDataFiles(rootName: string, relnames: string[]): string[] {
Expand All @@ -30,13 +30,13 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {

async function find(
rootName: string,
// These are passed through to findInterpretersInDir().
maxDepth?: number,
filterFile?: (x: string) => boolean,
filterDir?: (x: string) => boolean,
// Errors are helpful when testing, so we don't bother ignoring them.
): Promise<string[]> {
const results: string[] = [];
const root = path.join(datadir, rootName);
const executables = findInterpretersInDir(root, maxDepth, filterFile);
const executables = findInterpretersInDir(root, maxDepth, filterDir);
for await (const filename of executables) {
results.push(filename);
}
Expand All @@ -60,6 +60,8 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
sub2.2/
<spam.exe>
python3.exe
sub3/
python.exe
<spam.exe>
spam.txt
<python.exe>
Expand All @@ -80,6 +82,8 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
sub2.2/
<spam>
python3
sub3/
python
<spam>
spam.txt
<python>
Expand Down Expand Up @@ -112,30 +116,6 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {

assert.deepEqual(found, expected);
});

test('filtered', async () => {
const expected = resolveDataFiles(
rootName,
IS_WINDOWS
? [
// These will match.
'python2.exe',
'python3.8.exe',
]
: [
// These will match.
'python2',
'python3.8',
],
);
function filter(filename: string): boolean {
return filename.match(/python\d/) != null;
}

const found = await find(rootName, undefined, filter);

assert.deepEqual(found, expected);
});
});

suite('recursive', () => {
Expand All @@ -150,6 +130,7 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
'python3.8.exe',
'sub2/sub2.1/sub2.1.1/python.exe',
'sub2/sub2.2/python3.exe',
'sub3/python.exe',
]
: [
// These will match.
Expand All @@ -158,6 +139,7 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
'python3.8',
'sub2/sub2.1/sub2.1.1/python',
'sub2/sub2.2/python3',
'sub3/python',
],
);

Expand All @@ -172,22 +154,24 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
IS_WINDOWS
? [
// These will match.
'python.exe',
'python2.exe',
'python3.8.exe',
'sub2/sub2.2/python3.exe',
'sub3/python.exe',
]
: [
// These will match.
'python',
'python2',
'python3.8',
'sub2/sub2.2/python3',
'sub3/python',
],
);
function filter(filename: string): boolean {
return filename.match(/python\d/) != null;
function filterDir(dirname: string): boolean {
return dirname.match(/sub\d$/) !== null;
}

const found = await find(rootName, 3, filter);
const found = await find(rootName, 3, filterDir);

assert.deepEqual(found, expected);
});
Expand Down Expand Up @@ -248,18 +232,18 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
rootName,
IS_WINDOWS
? [
// These will match.
// These order here matters.
'python.exe',
'python2.exe',
'python2.7.exe',
'python2.exe',
'python27.exe',
'python3.exe',
'python3.8.exe',
'python3.exe',
'python38.exe',
'python381.exe',
]
: [
// These will match.
// These order here matters.
'python',
'python2',
'python2.7',
Expand Down Expand Up @@ -390,35 +374,35 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
rootName,
IS_WINDOWS
? [
// These will match.
// These order here matters.
'3.8/bin/python.exe',
'3.8/bin/python3.exe',
'3.8/bin/python3.8.exe',
'3.8/bin/python3.exe',
'my-python/python3.exe',
'py/2.7/bin/python.exe',
'py/2.7/bin/python2.exe',
'py/2.7/bin/python2.7.exe',
'py/2.7/bin/python2.exe',
'py/3.8/bin/python.exe',
'py/3.8/bin/python3.exe',
'py/3.8/bin/python3.8.exe',
'py/3.8/bin/python3.exe',
'python/3.8/bin/python.exe',
'python/3.8/bin/python3.exe',
'python/3.8/bin/python3.8.exe',
'python/3.8/bin/python3.exe',
'python/bin/python.exe',
'python/bin/python3.exe',
'python/bin/python3.8.exe',
'python-3.8/bin/python3.exe',
'python/bin/python3.exe',
'python-3.8/bin/python3.8.exe',
'python.3.8/bin/python3.exe',
'python-3.8/bin/python3.exe',
'python.3.8/bin/python3.8.exe',
'python.3.8/bin/python3.exe',
'python2/python.exe',
'python3/python3.exe',
'python3.8/bin/python3.exe',
'python3.8/bin/python3.8.exe',
'python3.8/bin/python3.exe',
'python38/bin/python3.exe',
]
: [
// These will match.
// These order here matters.
'3.8/bin/python',
'3.8/bin/python3',
'3.8/bin/python3.8',
Expand Down Expand Up @@ -517,19 +501,19 @@ suite('pyenvs common utils - findInterpretersInDir()', () => {
rootName,
IS_WINDOWS
? [
// These will match.
// These order here matters.
'pythons/python.exe',
'pythons/python2.exe',
'pythons/python2.7.exe',
'pythons/python3.exe',
'pythons/python2.exe',
'pythons/python3.7.exe',
'pythons/python3.8.exe',
'pythons/python3.exe',
'python3.8.exe/python.exe',
'Python3.exe',
'PYTHON.EXE',
]
: [
// These will match.
// These order here matters.
'pythons/python',
'pythons/python2',
'pythons/python2.7',
Expand Down