Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
b260c2a
Add conda environments
rchiodo Mar 4, 2020
885f972
Reenable multiple interpreters
rchiodo Mar 4, 2020
d3f04f6
Change path for conda environments
rchiodo Mar 4, 2020
11c6097
Add pip requirements
rchiodo Mar 4, 2020
db60ded
Get multiple interpreters test to work
rchiodo Mar 4, 2020
fca098e
Partially working conda search
rchiodo Mar 5, 2020
ba0fc56
Get conda working locally
rchiodo Mar 5, 2020
d4f3fee
Make windows supported environments
rchiodo Mar 5, 2020
fb16261
Fix hygiene
rchiodo Mar 5, 2020
bfb4cec
Fix memory leak for daemons
rchiodo Mar 6, 2020
43cf4a1
Fix hang on conda
rchiodo Mar 6, 2020
8e11f39
Actually retry
rchiodo Mar 6, 2020
2573ea0
Two missing setting change updates
rchiodo Mar 6, 2020
c418e11
Rework the failure capture
rchiodo Mar 6, 2020
318b82c
Get rid of memory leak when session dies
rchiodo Mar 6, 2020
cd9112a
Make tests leak less memory
rchiodo Mar 6, 2020
fd76344
Hygiene problem
rchiodo Mar 6, 2020
a83acca
More potential cleanup
rchiodo Mar 6, 2020
be983b2
More changes to clean up memory as soon as possible
rchiodo Mar 9, 2020
39f5e4b
Dont auto start jupyter
rchiodo Mar 9, 2020
dcd6c0f
More leaks found and fixes for breaking tests
rchiodo Mar 9, 2020
21eab4b
Merge remote-tracking branch 'origin/master' into rchiodo/test_with_a…
rchiodo Mar 9, 2020
87fbdbb
Fix missing args when test is shutting down
rchiodo Mar 10, 2020
96b4395
Put back some of the functional requirements
rchiodo Mar 10, 2020
587bafc
Update tests to not use static
rchiodo Mar 10, 2020
94b75fd
Fix linter tests
rchiodo Mar 10, 2020
7712c47
More output for linter tests
rchiodo Mar 10, 2020
b9fc139
Linter only fails with other tests
rchiodo Mar 10, 2020
6f85ffb
Install linters in conda too
rchiodo Mar 10, 2020
130fc50
Make base also have dependencies as it's being picked up anyway
rchiodo Mar 10, 2020
48cc263
Add news entry
rchiodo Mar 11, 2020
bcdfdaf
Review feedback
rchiodo Mar 11, 2020
04d6caa
Fix unit tests and one of the functional tests
rchiodo Mar 11, 2020
1f03b7c
Functional tests and workspace tests at the same time fail
rchiodo Mar 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Review feedback
  • Loading branch information
rchiodo committed Mar 11, 2020
commit bcdfdaf16e3ad8609ac217a2dfaa79e0a996665c
8 changes: 6 additions & 2 deletions src/client/datascience/jupyter/jupyterSessionManager.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,12 @@ export class JupyterSessionManager implements IJupyterSessionManager {

// tslint:disable-next-line: no-any
const sessionManager = this.sessionManager as any;
this.sessionManager.dispose();
this.sessionManager = undefined;
try {
await this.sessionManager.shutdownAll();
} finally {
this.sessionManager.dispose();
this.sessionManager = undefined;
}

// The session manager can actually be stuck in the context of a timer. Clear out the specs inside of
// it so the memory for the session is minimized. Otherwise functional tests can run out of memory
Expand Down
15 changes: 13 additions & 2 deletions src/client/interpreter/activation/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ const condaRetryMessages = [
'The directory is not empty'
];

/**
* This class exists so that the environment variable fetching can be cached in between tests. Normally
* this cache resides in memory for the duration of the EnvironmentActivationService's lifetime, but in the case
* of our functional tests, we want the cached data to exist outside of each test (where each test will destroy the EnvironmentActivationService)
* This gives each test a 3 or 4 second speedup.
*/
export class EnvironmentActivationServiceCache {
private static useStatic = false;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some comments explaining the purpose of this class would be useful.
Specially for core extension devs.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay


In reply to: 391124953 [](ancestors = 391124953)

private static staticMap = new Map<string, InMemoryCache<NodeJS.ProcessEnv | undefined>>();
Expand Down Expand Up @@ -180,11 +186,15 @@ export class EnvironmentActivationService implements IEnvironmentActivationServi
const command = `${activationCommand} && echo '${getEnvironmentPrefix}' && python ${printEnvPyFile.fileToCommandArgument()}`;
traceVerbose(`Activating Environment to capture Environment variables, ${command}`);

// Conda activate can hang on certain systems. Fail after 30 seconds.
// Do some wrapping of the call. For two reasons:
// 1) Conda activate can hang on certain systems. Fail after 30 seconds.
// See the discussion from hidesoon in this issue: https://github.com/Microsoft/vscode-python/issues/4424
// His issue is conda never finishing during activate. This is a conda issue, but we
// should at least tell the user.
// 2) Retry because of this issue here: https://github.com/microsoft/vscode-python/issues/9244
// This happens on AzDo machines a bunch when using Conda (and we can't dictate the conda version in order to get the fix)
let result: ExecutionResult<string> | undefined;
let tryCount = 1;
while (!result) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels dangerous to keep trying indefinitely. I'd gate this to 10-20 or other retries. I.e. max out at 5-10-20 seconds.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sure. I suppose it's possible it could get stuck indefinitely.


In reply to: 391128679 [](ancestors = 391128679)

try {
result = await processService.shellExec(command, {
Expand All @@ -201,9 +211,10 @@ export class EnvironmentActivationService implements IEnvironmentActivationServi
// Special case. Conda for some versions will state a file is in use. If
// that's the case, wait and try again. This happens especially on AzDo
const excString = exc.toString();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to add comments about the issue this is trying to solve (probably link to issue number). Right now its not obvious that we're trying to work around some conda issue.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I.e. its not obvious as to why we even need a retry.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll link to the issue and explain here that conda sucks.


In reply to: 391126817 [](ancestors = 391126817)

if (condaRetryMessages.find(m => excString.includes(m))) {
if (condaRetryMessages.find(m => excString.includes(m)) && tryCount < 10) {
traceInfo(`Conda is busy, attempting to retry ...`);
result = undefined;
tryCount += 1;
await sleep(500);
} else {
throw exc;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,12 @@ import { sendTelemetryEvent } from '../../../telemetry';
import { EventName } from '../../../telemetry/constants';
import { IInterpreterLocatorService, IInterpreterWatcher, PythonInterpreter } from '../../contracts';

/**
* This class exists so that the interpreter fetching can be cached in between tests. Normally
* this cache resides in memory for the duration of the CacheableLocatorService's lifetime, but in the case
* of our functional tests, we want the cached data to exist outside of each test (where each test will destroy the CacheableLocatorService)
* This gives each test a 20 second speedup.
*/
export class CacheableLocatorPromiseCache {
private static useStatic = false;
private static staticMap = new Map<string, Deferred<PythonInterpreter[]>>();
Expand Down