V2 is different than V3 by matthid · Pull Request #2632 · fsprojects/Paket

matthid · 2017-08-19T12:47:54Z

For example consider https://api.nuget.org/v3-flatcontainer/machine.specifications/index.json?semVerLevel=2.0.0 (and V3 API)
it returns "0.10.0-unstable0014". Paket uses this string currently on a V2 API,

returns "0.10.0-Unstable0014". And in fact https://www.nuget.org/api/v2/Packages(Id='machine.specifications',Version='0.10.0-Unstable0014') works

The problem with this change is in fact that it degrades performance. As the V3 GetPackageDetails needs to API calls to get the required information, which the V2 API generally is only a single call.

matthid · 2017-08-19T12:51:31Z

Also this change uncovers some bugs on our v3 implementation, which were hidden because it was never actually used. For example if I let it run on FAKE (paket update) with v3 urls:

Paket failed with
-> The NuGet source http://www.nuget.org/api/v2 for package Microsoft.Bcl was not found in the paket.dependencies file with sources [NuGetV3 {Url = "https://api.nuget.org/v3/index.json";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fake";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fsharp-formatting";
             Authentication = None;}]

As well as some "Unable to parse framework .NETPlatform5.4" (and other warnings). It seems like V3 returns some (including Portable-) profiles in a format we cannot currently parse.

matthid · 2017-08-19T13:56:33Z

The new warning:

$ ../Paket/bin/paket.exe update --group main
Paket version 5.85.8
Updating group main in C:\Proj\FAKE\paket.dependencies
Resolving packages for group Main:
 - Microsoft.Web.Administration is pinned to 7.0.0
 - Argu is pinned to 3.3.0
 - Microsoft.AspNet.Razor is pinned to 2.0.30506
 - Microsoft.AspNet.WebPages is pinned to 2.0.30506
Could not detect any platforms from '.NETPortable4.5-Profile259' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0
 - log4net is pinned to 2.0.5
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
Could not detect any platforms from '.NETPlatform5.4' in Microsoft.AspNet.Razor 4.0.0-rc1-final
 - serilog.sinks.nlog is pinned to 1.5.4
Could not detect any platforms from '.NETPortable4.5-Profile259' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview2-25401-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview2-25401-01
 - Machine.Specifications is pinned to 0.9.3
Could not detect any platforms from '.NETPortable4.5-Profile259' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.5-Profile111' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from '.NETPortable4.6-Profile151' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview1-25301-01
Could not detect any platforms from 'UAP10.1' in NETStandard.Library 2.0.0-preview1-25301-01
 - Machine.Specifications.Should is pinned to 0.8
 - RavenDB.Client 2.5.25023
 - Mono.Cecil 0.10.0-beta6
 - Paket.Core 5.86.0
 - bootstrap 3.3.7
 - FluentMigrator.Runner 1.12.0
 - FsCheck 2.9.0
 - FsCheck.Xunit 2.9.0
 - FSharp.Compiler.Service 13.0.0
 - FSharp.Core 4.2.3
 - HashLib 2.0.1
 - Knockout 0.0.1
 - Knockout.Mapping 2.4.0
 - Machine.Specifications.Runner.Console 0.9.3
 - Microsoft.DotNet.PlatformAbstractions 2.0.0
 - Microsoft.Net.Http 2.2.29
Could not detect any platforms from '.NETPortable0.0-Profile7' in Microsoft.DiaSymReader.PortablePdb 1.2.0
Could not detect any platforms from '.NETPortable0.0-Profile7' in Microsoft.DiaSymReader.PortablePdb 1.2.0
Could not detect any platforms from '.NETPortable0.0-Profile7' in Microsoft.DiaSymReader.PortablePdb 1.2.0
 - Mono.Web.Xdt 1.0.0
 - Nancy.Authentication.Forms 1.4.1
 - Nancy.Authentication.Stateless 1.4.1
Could not detect any platforms from '.NETPortable0.0-Profile7' in System.Reflection.Metadata 1.5.0
Could not detect any platforms from '.NETPortable4.5-Profile111' in System.Reflection.Metadata 1.5.0
 - Nancy.Hosting.Aspnet 1.4.1
 - Nancy.Hosting.Self 1.4.1
 - Nancy.Serialization.JsonNet 1.4.1
 - Nancy.Testing 1.4.1
 - Nancy.Viewengines.Razor 1.4.3
 - Newtonsoft.Json 10.0.3
 - NuGet.CommandLine 4.1.0
 - Nuget.Core 2.14.0
 - NUnit 3.7.1
 - NUnit.Console 3.7.0
 - Octokit 0.24.0
 - RavenDB.Server 3.5.4
 - SSH.NET 2016.0.0
 - System.Web.Razor.Unofficial 2.0.2
 - toastr 2.1.1
 - Unquote 3.2.0
 - xunit 2.2.0
 - xunit.runner.console 2.2.0
 - xunit.extensibility.execution 2.2.0
 - jQuery 3.1.1
 - Serilog 1.5.14
 - xunit.assert 2.2.0
 - xunit.core 2.2.0
 - SshNet.Security.Cryptography 1.2.0
 - System.ValueTuple 4.4.0
 - runtime.native.System 4.3.0
 - runtime.native.System.IO.Compression 4.3.0
 - System.Buffers 4.4.0
 - System.Collections 4.3.0
 - System.Diagnostics.Debug 4.3.0
 - System.Diagnostics.Tracing 4.3.0
 - System.Globalization 4.3.0
 - System.IO 4.3.0
 - System.IO.Compression 4.3.0
 - System.Linq 4.3.0
 - System.Reflection 4.3.0
 - System.Reflection.Emit 4.3.0
 - System.Reflection.Extensions 4.3.0
 - System.Reflection.Primitives 4.3.0
 - System.Reflection.TypeExtensions 4.4.0
 - System.Resources.ResourceManager 4.3.0
 - System.Runtime 4.3.0
 - System.Runtime.Extensions 4.3.0
 - System.Runtime.Handles 4.3.0
 - System.Runtime.InteropServices 4.3.0
 - System.Security.Cryptography.Algorithms 4.3.0
 - System.Text.Encoding 4.3.0
 - System.Text.Encoding.Extensions 4.3.0
 - System.Threading 4.3.0
 - System.Threading.Tasks 4.3.0
 - Microsoft.CSharp 4.4.0
 - System.ComponentModel.TypeConverter 4.3.0
 - System.Runtime.Serialization.Formatters 4.3.0
 - System.Runtime.Serialization.Primitives 4.3.0
 - System.Xml.XmlDocument 4.3.0
 - System.Runtime.Loader 4.3.0
 - System.Threading.Thread 4.3.0
 - System.Diagnostics.FileVersionInfo 4.3.0
 - System.Diagnostics.Process 4.3.0
 - System.Diagnostics.TraceSource 4.3.0
 - System.Security.Cryptography.ProtectedData 4.4.0
 - System.Xml.XDocument 4.3.0
 - System.Xml.XPath.XDocument 4.3.0
 - System.Xml.XPath.XmlDocument 4.3.0
 - System.Linq.Expressions 4.3.0
 - System.Text.RegularExpressions 4.3.0
 - System.AppContext 4.3.0
 - System.Net.Sockets 4.3.0
 - System.Net.Requests 4.3.0
 - System.Threading.ThreadPool 4.3.0
 - System.Diagnostics.Tools 4.3.0
 - System.Linq.Queryable 4.3.0
 - System.Runtime.Numerics 4.3.0
 - System.Threading.Tasks.Parallel 4.3.0
 - System.Threading.Timer 4.3.0
 - System.IO.FileSystem 4.3.0
 - System.IO.FileSystem.Primitives 4.3.0
 - System.Console 4.3.0
 - AspNetMvc 4.0.20710
 - System.Runtime.InteropServices.RuntimeInformation 4.3.0
 - System.Security.Cryptography.Csp 4.3.0
 - System.Net.NameResolution 4.3.0
 - NUnit.ConsoleRunner 3.7.0
 - NUnit.Extension.NUnitV2Driver 3.7.0
 - NUnit.Extension.NUnitProjectLoader 3.6.0
 - NUnit.Extension.NUnitV2ResultWriter 3.6.0
 - NUnit.Extension.VSProjectLoader 3.6.0
 - NLog 4.4.12
 - Microsoft.Web.Xdt 2.1.1
 - xunit.abstractions 2.0.1
 - knockoutjs 3.4.2
 - FluentMigrator.Runner 1.6.2
 - FluentMigrator 1.6.2
 - NETStandard.Library 2.0.0
 - Nancy 1.4.4
 - System.Reflection.Metadata 1.5.0
 - CsQuery 1.3.4
 - System.Collections.Immutable 1.4.0
 - Microsoft.DiaSymReader.PortablePdb 1.2.0
 - Microsoft.Bcl 1.1.10
 - Microsoft.DiaSymReader 1.1.0
 - Microsoft.NETCore.Platforms 2.0.0
 - Microsoft.NETCore.Targets 2.0.0
 - Microsoft.Bcl.Build 1.0.21
 - NUnit.Extension.TeamCityEventListener 1.0.2
 - Microsoft.Web.Infrastructure 1.0.0
 - Microsoft.CompilerServices.AsyncTargetingPack 1.0.1
 - Chessie 0.6.0
 - FSharp.Compiler.Tools 4.1.23
 - Microsoft.AspNet.Mvc 4.0.40804
 - xunit.extensibility.core 2.2.0
 - System.Net.Http 4.3.2
 - System.Dynamic.Runtime 4.3.0
 - Microsoft.Win32.Primitives 4.3.0
 - System.Collections.Concurrent 4.3.0
 - System.Globalization.Calendars 4.3.0
 - System.IO.Compression.ZipFile 4.3.0
 - System.Net.Primitives 4.3.0
 - System.ObjectModel 4.3.0
 - System.Security.Cryptography.Encoding 4.3.0
 - System.Security.Cryptography.Primitives 4.3.0
 - System.Security.Cryptography.X509Certificates 4.3.0
 - System.Xml.ReaderWriter 4.3.0
 - System.ComponentModel 4.3.0
 - System.ComponentModel.Primitives 4.3.0
 - System.Reflection.Emit.ILGeneration 4.3.0
 - System.Reflection.Emit.Lightweight 4.3.0
 - System.Security.Principal.Windows 4.4.0
 - runtime.native.System.Security.Cryptography.Apple 4.3.0
 - runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.debian.8-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.fedora.23-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.fedora.24-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.opensuse.13.2-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.opensuse.42.1-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.rhel.7-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.ubuntu.14.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.ubuntu.16.04-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.ubuntu.16.10-x64.runtime.native.System.Security.Cryptography.OpenSsl 4.3.1
 - runtime.osx.10.10-x64.runtime.native.System.Security.Cryptography.Apple 4.3.0
 - runtime.native.System.Net.Http 4.3.0
 - System.Diagnostics.DiagnosticSource 4.4.1
 - System.Globalization.Extensions 4.3.0
 - System.Security.Cryptography.OpenSsl 4.3.0
 - System.Security.Cryptography.Cng 4.4.0
 - System.Security.Claims 4.3.0
 - System.Security.Principal 4.3.0
C:\Proj\FAKE\paket.lock is already up-to-date
Installing into projects:
 - Creating model and downloading packages.
Downloading FSharp.Formatting 3.0.0-beta07 (Build)
Downloading FSharp.Formatting.CommandTool 3.0.0-beta07 (Build)
Performance:
 - Resolver: 1 minute, 22 seconds (1 runs)
    - Runtime: 904 milliseconds
    - Blocked (retrieving package details): 1 minute, 11 seconds (55 times)
    - Blocked (retrieving package versions): 9 seconds (14 times)
    - Not Blocked (retrieving package details): 119 times
    - Not Blocked (retrieving package versions): 249 times
 - Average Request Time: 581 milliseconds
 - Number of Requests: 838
 - Runtime: 1 minute, 26 seconds
Paket failed with
-> The NuGet source http://www.nuget.org/api/v2 for package FSharp.Core was not found in the paket.dependencies file with sources [NuGetV3 {Url = "https://api.nuget.org/v3/index.json";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fake";
             Authentication = None;};
    NuGetV2 {Url = "https://ci.appveyor.com/nuget/fsharp-formatting";
             Authentication = None;}]

…2495

matthid · 2017-08-19T14:40:01Z

I think now we have too many warnings again :)

…s in the framework detection logic and makes the detection behave more like NuGet.

matthid · 2017-08-19T22:42:29Z

Ok the error message from above is an unrelated bug: #2634

matthid · 2017-08-19T22:44:52Z

This is basically ready (maybe some integration tests need to be updated -> CI), however I expect it to degrade performance a bit.

matthid · 2017-08-19T22:49:34Z

                None
        else Some { Name = path; Platforms = platforms })

+let extractPlatforms warn path =


The general guideline for warning parameter is:

true is always OK

false is only OK when either the calling method prints a better warning OR when the calling method has it's own warning parameter

This way warnings should not get lost. I don't know if there is better way to do this.

matthid · 2017-08-19T22:53:48Z

@dmunch just FYI I removed this monoandroid44w thing. Is there any actual package using this? Because afaics nuget doesn't even support it either: http://nugettoolsdev.azurewebsites.net/4.3.0-preview4/parse-framework?framework=monoandroid44w

forki · 2017-08-20T10:50:23Z

what's the status here?

matthid · 2017-08-20T11:30:52Z

Failing test is spurious, no idea what's going on

matthid · 2017-08-20T12:41:16Z

OK found the problem: There is no wpa80 apparently, fix is under way.

- Add net50 to project files (+update baselines) - Add test to ensure KnownTargetProfiles-lists are updated - Fix wpa parsing: Was parsed as wpa80 but that one doesn't exist (at least not in NuGet). wpa81 is the correct one. - Add hack for uap10.1 to keep project files constant for '.NetCore' (I don't know the correct condition to generate anyway)

matthid · 2017-08-20T13:02:24Z

+        let tagReader = FSharp.Reflection.FSharpValue.PreComputeUnionTagReader(typeof<'t>)
+        let cases = FSharp.Reflection.FSharpType.GetUnionCases(typeof<'t>)
+        checkListEx tagReader cases l
+    checkList KnownTargetProfiles.DotNetFrameworkVersions


matthid · 2017-08-20T14:07:30Z

I feel like this is ready.

dmunch · 2017-08-20T17:35:12Z

For me it's fine to remove it. Since the list is generated by a script contained in a comment on the top in the list it might however be worth adding the reason why this particular moniker has been removed. Le dim. 20 août 2017 à 00:53, Matthias Dittrich <notifications@github.com> a écrit :

…

@dmunch <https://github.com/dmunch> just FYI I removed this monoandroid44w thing. Is there any actual package using this? Because afaics nuget doesn't even support it either: http://nugettoolsdev.azurewebsites.net/4.3.0-preview4/parse-framework?framework=monoandroid44w — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2632 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEcbXHjn1cA7DYFLkVyMme0OxSVGbUiKks5sZ2d-gaJpZM4O8UtG> .

dmunch · 2017-08-20T17:36:32Z

I might remember wrong about the script, can't easily check the code here. Still would find it good to add the reason why it has been removed somewhere for future reference Le dim. 20 août 2017 à 19:34, Daniel Munch <dani.munch@gmail.com> a écrit :

…

For me it's fine to remove it. Since the list is generated by a script contained in a comment on the top in the list it might however be worth adding the reason why this particular moniker has been removed. Le dim. 20 août 2017 à 00:53, Matthias Dittrich ***@***.***> a écrit : > @dmunch <https://github.com/dmunch> just FYI I removed this > monoandroid44w thing. Is there any actual package using this? Because > afaics nuget doesn't even support it either: > http://nugettoolsdev.azurewebsites.net/4.3.0-preview4/parse-framework?framework=monoandroid44w > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#2632 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AEcbXHjn1cA7DYFLkVyMme0OxSVGbUiKks5sZ2d-gaJpZM4O8UtG> > . >

matthid · 2017-08-20T18:14:29Z

@dmunch technically the string is not a valid tfm. Nuget.exe is not detecting it either.

I don't want to break anything, but can you show an actual package using this string and is nuget.exe working with it?

0x53A · 2017-08-20T18:57:53Z

Note: uap10.1 is also detected as unsupported by paket, but IS detected by nuget:
http://nugettoolsdev.azurewebsites.net/4.3.0-preview4/parse-framework?framework=uap10.1

matthid · 2017-08-20T19:03:52Z

@0x53A I thought I added that with this PR?

matthid · 2017-08-20T19:05:23Z

+[<Test>]
+let ``Can detect uap10.1``() =
+    let p = PlatformMatching.forceExtractPlatforms "UAP10.1"
+    p.ToTargetProfile false |> shouldEqual (Some (SinglePlatform (FrameworkIdentifier.UAP UAPVersion.V10_1)))


ah, sorry, I just saw the warning in another build log, and searching in github found the log at the top, where it was also missing, but I didn't actually look at the code.

no problem. This pr "fixes" tfm parsing to be more like nuget, but it makes supporting monoandroid44w a bit cumbersome. That's why I have removed it.

use v2 when v2 was specified and v3 when v3 was specified...

9f63cd0

Improve 'Could not detect any platforms from' warning, references #2495

13384b7

track warnings until we have enough information to report them, fixes #…

70838d7

…2495

matthid added 4 commits August 19, 2017 17:11

fix some more instances of warnings without context

0ed043a

fix an instance of using exceptions for control flow.

58f1213

fix warnings detected with previous commits. Basically fixes some bug…

58d18f9

…s in the framework detection logic and makes the detection behave more like NuGet.

add some backwards compat quirks

1df50e3

matthid changed the title ~~[WIP] V2 is different than V3~~ V2 is different than V3 Aug 19, 2017

matthid commented Aug 19, 2017

View reviewed changes

get rid of 44w in tests and project files (fix integration-tests)

aac3976

matthid commented Aug 20, 2017

View reviewed changes

matthid mentioned this pull request Aug 20, 2017

paket restore/install resulted in "The profile 'portable-net45+win8+wp8+wp81+wpa81' is not a known profile. Please tell the package author." #2495

Closed

forki merged commit 5e29efd into master Aug 21, 2017

matthid deleted the v2_vs_v3_data branch July 2, 2019 17:16

Conversation

matthid commented Aug 19, 2017

Uh oh!

matthid commented Aug 19, 2017

Uh oh!

matthid commented Aug 19, 2017

Uh oh!

matthid commented Aug 19, 2017

Uh oh!

matthid commented Aug 19, 2017

Uh oh!

matthid commented Aug 19, 2017

Uh oh!

matthid Aug 19, 2017

Choose a reason for hiding this comment

Uh oh!

matthid commented Aug 19, 2017

Uh oh!

forki commented Aug 20, 2017

Uh oh!

matthid commented Aug 20, 2017

Uh oh!

matthid commented Aug 20, 2017

Uh oh!

matthid Aug 20, 2017

Choose a reason for hiding this comment

Uh oh!

matthid commented Aug 20, 2017

Uh oh!

dmunch commented Aug 20, 2017 via email

Uh oh!

dmunch commented Aug 20, 2017 via email

Uh oh!

matthid commented Aug 20, 2017

Uh oh!

0x53A commented Aug 20, 2017

Uh oh!

matthid commented Aug 20, 2017

Uh oh!

matthid Aug 20, 2017

Choose a reason for hiding this comment

Uh oh!

0x53A Aug 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

matthid Aug 20, 2017

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

0x53A Aug 20, 2017 •

edited

Loading