[Ready] Fix/refactor/improve framework restriction system, add performance metrics and pre-load requests by matthid · Pull Request #2336 · fsprojects/Paket

matthid · 2017-05-13T15:55:40Z

start fixing #2317 because #2333 shows that its relevant.

Changes:

Remove all magic version strings
Restrictions can now be as complex as wanted - the simplifier almost always makes them look as "simple" as before
Changed Syntax in Lockfile from "Framework: " to "Restriction: " in order to be backwards compatible and allow new syntax (see below).
Restrictions now work nicely in combination with Portable profiles
Removed a various amount of hacks
Remove support for V4_Client -> if complains come up we need to find a general solution as -client and -full exists for all framework versions
Adds a performance overview.

TODO:

Fix tests (part of Parallel requests #2356)
Performance
Lots of Json Exceptions seem to happen in the background...
Write parser for the new syntax
Is Syntax OK?

Example:

NUGET
  remote: https://www.nuget.org/api/v2
    CommonServiceLocator (1.3) - restriction: || (== sl5) (>= net40) (>= portable-net45+monoandroid+xamarinios+win8+wp8+wpa81)
    MvvmLightLibs (5.2)
      CommonServiceLocator (>= 1.0) - restriction: || (== net35) (== sl4)
      CommonServiceLocator (>= 1.3) - restriction: || (== sl5) (>= net40) (>= portable-net45+monoandroid+xamarinios+win8+wp8+wpa81)

The general syntax is

OP (operand1) (operand2) ... for ||, NOT and && and
OP tfm for ==, >= and <.
operandN itself can be a operator again, for example || (&& (>= net3) (< net4)) (== sl4)

Future Work:

Don't punish custom portables as hard (sometimes they need to be prefered. see http://nugettoolsdev.azurewebsites.net/4.0.0/get-nearest-framework?project=xamarinios10&package=portable-net45%2Bwin8%0D%0Aportable-net45%2Bmonoandroid10%2Bmonotouch10%2Bxamarinios10%2BXamarin.Mac20%0D%0Aportable-net40%2Bsl4%2Bwin8%2Bwp71%2Bwpa81%0D%0Aportable-net45%2Bwin8%0D%0Anet45%0D%0Anet40%0D%0Amonotouch%0D%0Awin8%0D%0Awpa81%0D%0Amonoandroid)
I'm not doing that as part of this PR
-> I'd suggest rewriting that "penalty" part from scratch

forki · 2017-05-14T08:23:15Z

You should be able to release this yourself with the release script that I gave you for fake. Am 13.05.2017 5:55 nachm. schrieb "Matthias Dittrich" < notifications@github.com>:

…

start fixing #2317 <#2317> because #2333 <#2333> shows that its relevant. ------------------------------ You can view, comment on, or merge this pull request online at: #2336 Commit Summary - start fixing #2317 because #2333 shows that its relevant. File Changes - *M* src/Paket.Core/Dependencies/PackageResolver.fs <https://github.com/fsprojects/Paket/pull/2336/files#diff-0> (1) - *M* src/Paket.Core/PaketConfigFiles/ProjectFile.fs <https://github.com/fsprojects/Paket/pull/2336/files#diff-1> (4) - *M* src/Paket.Core/Versioning/FrameworkHandling.fs <https://github.com/fsprojects/Paket/pull/2336/files#diff-2> (91) - *M* src/Paket.Core/Versioning/Requirements.fs <https://github.com/fsprojects/Paket/pull/2336/files#diff-3> (64) - *M* tests/Paket.Tests/InstallModel/Xml/System.Security.Cryptography. Algorithms.fs <https://github.com/fsprojects/Paket/pull/2336/files#diff-4> (3) - *M* tests/Paket.Tests/Versioning/FrameworkCompatibilitySpecs.fs <https://github.com/fsprojects/Paket/pull/2336/files#diff-5> (8) Patch Links: - https://github.com/fsprojects/Paket/pull/2336.patch - https://github.com/fsprojects/Paket/pull/2336.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2336>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AADgNOv409IYpY6QOkZ4vg7MijqXSEJuks5r5dJ9gaJpZM4NaHEa> .

matthid · 2017-05-14T09:15:54Z

I don't think I can finish this in the next days, it will be quite invasive in the code base (and remove so many hacks that it might hurt). These changes just show the direction it's going.... For now, I just demonstrate how the low-level code is currently broken (and quite a lot of code sits on top of this)

…eed to detect the group restrictions in a future compatible way.

…fier easier and only generated 'simple' formulas).

…ed to solve it for all version). Remove magic strings (not finished jet)

…uite.

matthid · 2017-05-22T21:13:30Z

I think this is ready for review. Note that integration-tests probably need to be adapted, but I need some days break from this. If one of you wants to finish this let me know (like I said I think its just fixing the tests). Unittest-suite is green already. See my edited first post for more info.
/cc @cloudRoutine @forki

forki · 2017-05-23T08:43:03Z

files changed: 80 +2,601 −1,895
ouch

…nts. Push Async down into the resolver to make the resolver aware that these are costly.

…o wait.

… combination with resolverstrategy.

…ew Thread).

Parallel requests

matthid · 2017-05-24T21:52:39Z

I can try to merge and release tomorrow.

start fixing #2317 because #2333 shows that its relevant.

80e94ed

matthid and others added 18 commits May 14, 2017 16:11

initial (probably broken) compiling version.

624c71e

we need to find all platforms supporting us in restriction logic.

1e187c8

parsing nuspec is a bit more complicated than expected. We properly n…

a97805b

…eed to detect the group restrictions in a future compatible way.

generating a generic description is not as easy as it should be :/

b246942

proper handling in addFrameworkRestrictionsToDependencies.

127a175

refactor

98d3ab8

make sure formulars are always in dnf (which makes writing the simpli…

7f86897

…fier easier and only generated 'simple' formulas).

refactor FrameworkIdentifier, remove V4_Client (if still needed we ne…

7f27225

…ed to solve it for all version). Remove magic strings (not finished jet)

refactor handling of portable profiles

c91e856

properly implement SupportedPlatforms for PCL and start to fix test s…

4623d5d

…uite.

fix several bugs and tests

d1646af

cleanup and more test fixes

e4c0f89

increase cache version

6951693

more cleanup and more test fixes.

8d15286

cleanup and support for unknown portable profiles.

281b221

cleanup and fix almost everything but the parser.

d02e5a0

unit tests green again

9318160

Cleanup and some code-simplifications.

8245a16

matthid requested review from cloudRoutine and forki May 22, 2017 21:13

matthid mentioned this pull request May 22, 2017

Paket is slow on HDD because of creating the packages folder #2333

Closed

matthid added 2 commits May 23, 2017 02:01

use sets for targetprofiles.

9d3831f

add some very basic profiling capabilites.

20c1ae6

matthid added 2 commits May 23, 2017 16:28

performance improvements by using sets and caches

4dd7398

Merge remote-tracking branch 'origin/master' into fix_restrictions

4d2adeb

matthid added 22 commits May 23, 2017 19:19

more performance by using sets and removing some conversations

1390a07

REVIEW: select another conflict

613b750

REVIEW: compare restrictions at least (because it is slowest)

1d6088d

Change Profile-Code to work in parallel and thread changing environme…

2fff14b

…nts. Push Async down into the resolver to make the resolver aware that these are costly.

preload dependency information, such that the resolver doesn't need t…

e6a7cf6

…o wait.

change the keys for the cache and delete an old cache.

7373ae6

bugfix in new stats

a4e4d77

add getPreferredVersionF to properly handle caching and preloading in…

a481dea

… combination with resolverstrategy.

let the user overwrite restriction globally

d7d39f8

fix some tests and a bug in the simplifier.

e5fe351

improve performance output

4a4090c

more performance data & fix test

306e06d

another test fixed

9dd5336

fix a parser bug and another test

09220ea

another test

dfbd104

Fix Stackoverflows in DEBUG mode by increasing the Stack-Size (in a n…

364a5f4

…ew Thread).

fix tests

cb2f0ee

add warning when something is not working as it should and fix caching.

ea40687

fix test code

0ffa1d6

fix test code

3874a3b

Cleanup stuff

bba0419

Merge pull request #2356 from fsprojects/parallel_requests

cf5c947

Parallel requests

matthid changed the title ~~[Ready for Review] Fix/refactor/improve framework restriction system~~ [Ready for Review] Fix/refactor/improve framework restriction system, add performance metrics and pre-load requests May 24, 2017

matthid changed the title ~~[Ready for Review] Fix/refactor/improve framework restriction system, add performance metrics and pre-load requests~~ [Ready] Fix/refactor/improve framework restriction system, add performance metrics and pre-load requests May 24, 2017

update documentation and RELEASE_NOTES.

148d73b

matthid merged commit e817322 into master May 25, 2017

matthid deleted the fix_restrictions branch May 25, 2017 09:30

This was referenced May 26, 2017

Problem with request preloading #2369

Closed

Cleanup and Fix Framework restrictions #2317

Closed

Navigation Menu

Repository navigation

[Ready] Fix/refactor/improve framework restriction system, add performance metrics and pre-load requests - #2336

[Ready] Fix/refactor/improve framework restriction system, add performance metrics and pre-load requests#2336
matthid merged 49 commits into
masterfrom
fix_restrictions

matthid commented May 13, 2017 •

edited

Loading

Uh oh!

forki commented May 14, 2017 via email

Uh oh!

matthid commented May 14, 2017

Uh oh!

matthid commented May 22, 2017

Uh oh!

forki commented May 23, 2017

Uh oh!

matthid commented May 24, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

matthid commented May 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

forki commented May 14, 2017 via email

Uh oh!

matthid commented May 14, 2017

Uh oh!

matthid commented May 22, 2017

Uh oh!

forki commented May 23, 2017

Uh oh!

matthid commented May 24, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

matthid commented May 13, 2017 •

edited

Loading