Tail Recursive Package Resolution by cloudRoutine · Pull Request #2066 · fsprojects/Paket

cloudRoutine · 2016-12-03T10:15:10Z

I checked the IL to make sure that it's tail recursive
I tested it against the repos that were causing stackoverflows in #2030, it successfully resolved for both

Although the output for

source https://nuget.org/api/v2

github logary/logary src/Logary.Facade/Facade.fs 4b21d888e83d72d1238ab0f36110e2a3946240e7

nuget Argu
nuget FsCheck
nuget FSharp.Core ~> 3
nuget FSharpx.Core
nuget BenchmarkDotNet
nuget NuGet.CommandLine
nuget Microsoft.NET.Sdk alpha 
nuget FSharp.NET.Sdk 1.0.0-alpha-000007 alpha framework: netstandard1.6
nuget NETStandard.Library 1.6.1 framework: netstandard1.6
nuget System.Diagnostics.TraceSource

group netcore 
    source https://www.nuget.org/api/v2

    nuget FSharp.Core 4.0.1.7-alpha alpha framework: netstandard1.6

group build
    source https://www.nuget.org/api/v2

    nuget FAKE

show that something is still clearly wrong

cloudRoutine · 2016-12-03T14:50:48Z

@matthid @forki I ran the integration tests again in debug mode to see what other errors might pop up without tail recursive optimization and the #1174 ninject test failed

Any chance either of you remember what part of the resolution function is failing when this kind of output occurs?

cloudRoutine · 2016-12-04T17:41:30Z

The verbose output for the failing test is

Test Name:	#1579 update allows unpinned
Test Outcome:	Failed
Result Message:	
System.Exception : Paket failed with:
	There was a version conflict during package resolution.
  Resolved packages:
   - MvvmLight 4.2.32.7
   - Paket.Test.A 1.0.0-prerelease
  Could not resolve package Mvvmlight >= 4.0 < 5.0:
   - Paket.Test.A 1.0.0-prerelease requested package Mvvmlight: 4.2.30

  Please try to relax some conditions.
StackTrace:
     at Paket.PackageResolver.ResolutionModule.getModelOrFail@212.Invoke(String message) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\PackageResolver.fs:line 212
   at Paket.UpdateProcess.groups@141-5.Invoke(GroupName groupName, DependenciesGroup dependenciesGroup) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 149
   at Microsoft.FSharp.Collections.MapTreeModule.mapiOpt[a,b,c](FSharpFunc`3 f, MapTree`2 m)
   at Microsoft.FSharp.Collections.FSharpMap`2.Map[b](FSharpFunc`2 f)
   at Paket.UpdateProcess.selectiveUpdate(Boolean force, FSharpFunc`2 getSha1, FSharpFunc`2 getSortedVersionsF, FSharpFunc`2 getPackageDetailsF, LockFile lockFile, DependenciesFile dependenciesFile, UpdateMode updateMode, SemVerUpdateMode semVerUpdateMode) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 139
   at Paket.UpdateProcess.SelectiveUpdate(DependenciesFile dependenciesFile, UpdateMode updateMode, SemVerUpdateMode semVerUpdateMode, Boolean force) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 200
   at Paket.UpdateProcess.SmartInstall(DependenciesFile dependenciesFile, UpdateMode updateMode, UpdaterOptions options) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 215
   at <StartupCode$Paket-Core>.$PublicAPI.Update@206.Invoke(Unit unitVar0) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\PublicAPI.fs:line 206
   at Paket.Utils.RunInLockedAccessMode[a](String rootFolder, FSharpFunc`2 action) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\Utils.fs:line 733
   at Paket.Program.main@432-15.Invoke(ParseResults`1 results) in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 432
   at Paket.Program.processWithValidation[T](Boolean silent, FSharpFunc`2 validateF, FSharpFunc`2 commandF, ParseResults`1 result) in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 36
   at Paket.Program.main$cont@414(ParseResults`1 results, Boolean silent, Boolean fromBootstrapper, Unit unitVar) in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 432
   at Paket.Program.main() in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 414
Result StandardOutput:	
C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp already exists.
C:\Users\jared\Github\Forks\Paket\bin\paket.exe pack templatefile paket.A.template version 1.0.0-prerelease output bin
C:\Users\jared\Github\Forks\Paket\bin\paket.exe update -v
Paket version 4.0.0-alpha030
found: C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies
Parsing C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies
Resolving packages for group Main:
   0 packages in resolution.
   2 requirements left
     - Mvvmlight, >= 4.0 < 5.0
     - Paket.Test.A, 

  Trying to resolve Mvvmlight >= 4.0 < 5.0 (from C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies)
  - fetching versions for Mvvmlight
 - Mvvmlight 4.2.32.7
   1 packages in resolution.
     - MvvmLight, 4.2.32.7
   2 requirements left
     - Paket.Test.A, 
     - MvvmLightLibs, >= 4.4.32.7

  Trying to resolve Paket.Test.A  (from C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies)
  - fetching versions for Paket.Test.A
 - Paket.Test.A 1.0.0-prerelease
   2 packages in resolution.
     - MvvmLight, 4.2.32.7
     - Paket.Test.A, 1.0.0-prerelease
   2 requirements left
     - Mvvmlight, 4.2.30
     - MvvmLightLibs, >= 4.4.32.7

  Trying to resolve Mvvmlight 4.2.30 (from Paket.Test.A 1.0.0-prerelease)
  Could not resolve package Mvvmlight >= 4.0 < 5.0:
   - Paket.Test.A 1.0.0-prerelease requested package Mvvmlight: 4.2.30

    ==> Trying different resolution.
 - Mvvmlight 4.2.30.0
     unlisted
 - Mvvmlight 4.1.27.1
     unlisted
 - Mvvmlight 4.1.27.0
     unlisted
 - Mvvmlight 4.1.26.1
     unlisted
 - Mvvmlight 4.1.26.0
     unlisted
 - Mvvmlight 4.1.23.0
     unlisted

I spent all day yesterday working on this, but I have no idea what the issue is 😞

forki · 2016-12-04T18:51:44Z

I assume one of the tree cuts is gone. Am 04.12.2016 6:41 nachm. schrieb "Jared Hester" <notifications@github.com>:

…

The verbose output for the failing test is Test Name: #1579 update allows unpinned Test Outcome: Failed Result Message: System.Exception : Paket failed with: There was a version conflict during package resolution. Resolved packages: - MvvmLight 4.2.32.7 - Paket.Test.A 1.0.0-prerelease Could not resolve package Mvvmlight >= 4.0 < 5.0: - Paket.Test.A 1.0.0-prerelease requested package Mvvmlight: 4.2.30 Please try to relax some conditions. StackTrace: at ***@***.***(String message) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\PackageResolver.fs:line 212 at ***@***.***(GroupName groupName, DependenciesGroup dependenciesGroup) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 149 at Microsoft.FSharp.Collections.MapTreeModule.mapiOpt[a,b,c](FSharpFunc`3 f, MapTree`2 m) at Microsoft.FSharp.Collections.FSharpMap`2.Map[b](FSharpFunc`2 f) at Paket.UpdateProcess.selectiveUpdate(Boolean force, FSharpFunc`2 getSha1, FSharpFunc`2 getSortedVersionsF, FSharpFunc`2 getPackageDetailsF, LockFile lockFile, DependenciesFile dependenciesFile, UpdateMode updateMode, SemVerUpdateMode semVerUpdateMode) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 139 at Paket.UpdateProcess.SelectiveUpdate(DependenciesFile dependenciesFile, UpdateMode updateMode, SemVerUpdateMode semVerUpdateMode, Boolean force) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 200 at Paket.UpdateProcess.SmartInstall(DependenciesFile dependenciesFile, UpdateMode updateMode, UpdaterOptions options) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\UpdateProcess.fs:line 215 at ***@***.***(Unit unitVar0) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\PublicAPI.fs:line 206 at Paket.Utils.RunInLockedAccessMode[a](String rootFolder, FSharpFunc`2 action) in C:\Users\jared\Github\Forks\Paket\src\Paket.Core\Utils.fs:line 733 at ***@***.***(ParseResults`1 results) in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 432 at Paket.Program.processWithValidation[T](Boolean silent, FSharpFunc`2 validateF, FSharpFunc`2 commandF, ParseResults`1 result) in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 36 at ***@***.***(ParseResults`1 results, Boolean silent, Boolean fromBootstrapper, Unit unitVar) in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 432 at Paket.Program.main() in C:\Users\jared\Github\Forks\Paket\src\Paket\Program.fs:line 414 Result StandardOutput: C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp already exists. C:\Users\jared\Github\Forks\Paket\bin\paket.exe pack templatefile paket.A.template version 1.0.0-prerelease output bin C:\Users\jared\Github\Forks\Paket\bin\paket.exe update -v Paket version 4.0.0-alpha030 found: C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies Parsing C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies Resolving packages for group Main: 0 packages in resolution. 2 requirements left - Mvvmlight, >= 4.0 < 5.0 - Paket.Test.A, Trying to resolve Mvvmlight >= 4.0 < 5.0 (from C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies) - fetching versions for Mvvmlight - Mvvmlight 4.2.32.7 1 packages in resolution. - MvvmLight, 4.2.32.7 2 requirements left - Paket.Test.A, - MvvmLightLibs, >= 4.4.32.7 Trying to resolve Paket.Test.A (from C:\Users\jared\Github\Forks\Paket\integrationtests\scenarios\i001579-unlisted\temp\paket.dependencies) - fetching versions for Paket.Test.A - Paket.Test.A 1.0.0-prerelease 2 packages in resolution. - MvvmLight, 4.2.32.7 - Paket.Test.A, 1.0.0-prerelease 2 requirements left - Mvvmlight, 4.2.30 - MvvmLightLibs, >= 4.4.32.7 Trying to resolve Mvvmlight 4.2.30 (from Paket.Test.A 1.0.0-prerelease) Could not resolve package Mvvmlight >= 4.0 < 5.0: - Paket.Test.A 1.0.0-prerelease requested package Mvvmlight: 4.2.30 ==> Trying different resolution. - Mvvmlight 4.2.30.0 unlisted - Mvvmlight 4.1.27.1 unlisted - Mvvmlight 4.1.27.0 unlisted - Mvvmlight 4.1.26.1 unlisted - Mvvmlight 4.1.26.0 unlisted - Mvvmlight 4.1.23.0 unlisted I spent all day yesterday working on this, but I have no idea what the issue is 😞 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2066 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNMOnytsKb3vwMg6hU2WZph3ZGUoKks5rEvtKgaJpZM4LDQYw> .

forki · 2016-12-12T08:43:34Z

you disabled magic mode?
now both CIs are red?

cloudRoutine · 2016-12-12T19:57:43Z

I can't remember where I first saw it, but magic mode was causing travis builds to fail so I disabled it here.
I have no idea why dotnet pack is failing now

forki · 2016-12-12T20:03:52Z

Please bring back magic mode. Pack did never work. I think @en is working on it Am 12.12.2016 8:57 nachm. schrieb "Jared Hester" <notifications@github.com>:

…

I can't remember where I first saw it, but magic mode was causing travis builds to fail so I disabled it here. I have no idea why dotnet pack is failing now — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2066 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNEnzgEaAuLb7BrYhvVgL81tn2pJjks5rHac4gaJpZM4LDQYw> .

cloudRoutine · 2016-12-13T00:32:02Z

@forki it's back to appveyor passing

forki · 2016-12-13T08:12:05Z

thanks so much for this!

cloudRoutine · 2016-12-13T08:16:40Z

I'm still a bit concerned about how the packages are resolving, even though it works properly

It's going through a lot more packages than it should be. Even after I got the unlisted package traversal stage working it's still going through a larger package list than it should. I think something is wrong with the list of compatible packages being provided for a specific version constraint.

forki · 2016-12-13T08:24:54Z

it's still going through a larger package list than it should

more than before?

cloudRoutine · 2016-12-13T08:29:48Z

I think so, but it's hard to tell since the tests aren't giving full output when they ran on appveyor in previous PRs. This is another reason #2070 would be useful.

cloudRoutine added 5 commits December 1, 2016 17:43

stackpack and debug & structured displays

74f8596

step and steploop can stand independently

e1baab5

reorganize step function

5a64291

maintain lists of floating conflicts and steps

14034ef

tail recursive package resolution

65b29e7

cloudRoutine force-pushed the overflow branch from 6314fc8 to 65b29e7 Compare December 3, 2016 10:54

update mono check

cd6d161

cloudRoutine force-pushed the overflow branch from 2625e15 to 0560eab Compare December 4, 2016 04:44

fix flag in outer loop stage

abaebbf

cloudRoutine force-pushed the overflow branch from 0560eab to abaebbf Compare December 4, 2016 12:19

more detailed verbose output

fd1108f

additional flag to track an unlisted package search inner step loop

677b862

cloudRoutine force-pushed the overflow branch 2 times, most recently from eb4c1f2 to 41a2d2f Compare December 5, 2016 22:25

cloudRoutine changed the title ~~[WIP] Tail Recursive Package Resolution~~ Tail Recursive Package Resolution Dec 5, 2016

cloudRoutine force-pushed the overflow branch 6 times, most recently from 2b784cb to b14dcc3 Compare December 5, 2016 23:57

use correct cli version on travis

804165c

cloudRoutine force-pushed the overflow branch from b14dcc3 to 804165c Compare December 5, 2016 23:59

cloudRoutine force-pushed the overflow branch from ea07d8f to 9292f52 Compare December 12, 2016 20:14

back to magic mode

b5b1491

cloudRoutine force-pushed the overflow branch from 9292f52 to b5b1491 Compare December 12, 2016 20:18

don't run 'dotnet pack'

7d6f95c

forki merged commit 7d6f95c into fsprojects:core3 Dec 13, 2016

cloudRoutine mentioned this pull request Dec 13, 2016

paket4 install crashes with stack-overflow exception #2030

Closed

Navigation Menu

Repository navigation

Tail Recursive Package Resolution - #2066

Tail Recursive Package Resolution#2066
forki merged 12 commits into
fsprojects:core3from
cloudRoutine:overflow

cloudRoutine commented Dec 3, 2016

Uh oh!

cloudRoutine commented Dec 3, 2016

Uh oh!

cloudRoutine commented Dec 4, 2016

Uh oh!

forki commented Dec 4, 2016 via email

Uh oh!

forki commented Dec 12, 2016

Uh oh!

cloudRoutine commented Dec 12, 2016

Uh oh!

forki commented Dec 12, 2016 via email

Uh oh!

cloudRoutine commented Dec 13, 2016

Uh oh!

forki commented Dec 13, 2016

Uh oh!

cloudRoutine commented Dec 13, 2016

Uh oh!

forki commented Dec 13, 2016

Uh oh!

cloudRoutine commented Dec 13, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

cloudRoutine commented Dec 3, 2016

Uh oh!

cloudRoutine commented Dec 3, 2016

Uh oh!

cloudRoutine commented Dec 4, 2016

Uh oh!

forki commented Dec 4, 2016 via email

Uh oh!

forki commented Dec 12, 2016

Uh oh!

cloudRoutine commented Dec 12, 2016

Uh oh!

forki commented Dec 12, 2016 via email

Uh oh!

cloudRoutine commented Dec 13, 2016

Uh oh!

forki commented Dec 13, 2016

Uh oh!

cloudRoutine commented Dec 13, 2016

Uh oh!

forki commented Dec 13, 2016

Uh oh!

cloudRoutine commented Dec 13, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants