The Wayback Machine - https://web.archive.org/web/20051114154320/http://www.kuro5hin.org:80/?op=displaystory;sid=2001/6/19/05641/7357

Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership | k5 store

[P]
Microsoft, TCP/IP, Open Source, and Licensing (Internet)

By adamba
Tue Jun 19th, 2001 at 05:50:32 AM EST

Software

A Wall Street Journal article titled Microsoft Uses Open-Source Code Despite Denying Use of Such Software recently caused a ruckus on the Web.

The article stated that Microsoft was running FreeBSD on some of the servers used by Hotmail, its free Web email service. It also claimed that "software connected with the FreeBSD open-source operating system is used in several places deep inside several versions of Microsoft's Windows software, such as in the `TCP/IP' section that arranges all connections to the Internet."


I worked at Microsoft for ten years, most of it on the core Windows NT/2000 (hereafter referred to as NT) networking code. As such I briefly dealt with the Hotmail team, mostly to hear them complain about the lameness of the telnet daemon in NT (a valid point). I do know that when Microsoft bought Hotmail, the email system was entirely running on FreeBSD, and Microsoft immediately set about trying to migrate it to NT, and it took many years to do so. Now it seems that the transition is not complete. Well, what are you gonna do.

On the other hand, I know a lot about the TCP/IP stack that is running on NT. Here is a short history of it (some of this may also be told in the book How the Web Was Won, but I haven't read it):

The original plan for NT was that a few members of the core NT team (which numbered about 15 developers) would write all the networking code. However, in 1990 a small team was started up in the LAN Manager group at Microsoft to do some of that NT networking work. Eventually that team moved over to be a part of NT (this coincided with the IBM-OS/2 "divorce", if anyone is interested).

Microsoft's networking software at the time ran over a network protocol called Netbeui, but it was decided that TCP/IP was gaining in importance, and should be included in NT. In addition, the user-mode API associated with Netbeui, which was called Netbios, was too Netbeui-specific and couldn't be adapted to allow user-mode access to TCP/IP. As a result, the decision was made:

1) To put a TCP/IP stack in NT

2) To adapt the sockets user-mode API for NT

#1 was solved by licensing code from a company called Spider Systems. However, Spider's TCP/IP stack was written to run within an environment called STREAMS, which was a wrapper that specified how the various parts of the stack would communicate with each other (TCP/IP is really several pieces of code -- two of which are TCP and IP -- layered on top of each other. Most network protocols are like that, which is why they are referred to as "stacks"). As a result, STREAMS also had to be ported to NT.

#2 involved the creation of the winsock API, which persists today.

It was recognized that using Spider's stack was a temporary measure, because nobody really wanted a stack that depended on STREAMS and its associated overhead. So, a short time after this, work was begun on a new version of TCP/IP, written entirely by Microsoft.

Along with Spider's stack came versions of various TCP/IP-related utility programs, such as ftp, rcp and rsh. Those were ported from BSD sockets to winsock (not a huge change) and bundled with NT.

Now, some of Spider's code (possibly all of it) was based on the TCP/IP stack in the BSD flavors of Unix. These are open source, but distributed under the BSD license, not the GPL that Linux is released under. Whereas the GPL states that any software derived from GPL'ed software must also be released under the GPL, the BSD license basically says, "here's the source, you can do whatever you want, just give credit to the original author."

Eventually the new, from scratch TCP/IP stack was done and shipped with NT 3.5 (the second version, despite the number) in late 1994. The same stack was also included with Windows 95.

However, it looks like some of those Unix utilities were never rewritten. If you look at the executables, you can still see the copyright notice from the regents of the University of California (BSD is short for Berkeley Software Distrubution, Berkeley being a branch of the University of California, for some reason referred to as "Berkeley" on the East Coast and "California" on the West Coast...and "Berkeley" is one of those words that starts to look real funny if you stare at it too long - but I digress).

Keep in mind there is no reason to rewrite that code. If your ftp client works fine (no comments from the peanut gallery!) then why change it? Microsoft has other fish to fry. And the software was licensed perfectly legally, since the inclusion of the copyright notice satisfied the BSD license.

I won't even swear on a stack of bibles that the "new" TCP/IP now shipping in NT/2000/XP and Windows 95/98/Me is completely free of the old code from Spider. Since I don't work there I don't have access to the source code. Certainly some parts of TCP (the checksum calculation comes to mind) are the same everywhere and once someone has written an optimized version, why rewrite it? And once again, this would be perfectly legitimate for Microsoft to do under the license.

But it is certainly misleading of the Wall Street Journal to say that BSD code is used "deep inside" the NT networking code, unless they mean the STREAMS wrapper itself, which I believe is still there in case someone wants to write a transport using it (I think there is an OSI TP4 STREAMS transport lurking somewhere out there, if anyone cares - but I just checked, nobody does). But the TCP/IP in NT certainly doesn't use STREAMS.

And implying that the TCP/IP stack uses BSD code is also false. As I said above there may be small vestiges of it in there, although I doubt it. Anyway the FreeBSD programmers who reported all this to the Wall Street Journal can't see the NT TCP/IP source either, so they can't have been referring to that.

But whatever! It isn't the first time Microsoft has been maligned by the press, and it won't be the last.

However, this history does illustrate Microsoft's view of the GPL. As you may recall, Microsoft VP Craig Mundie recently claimed that the GPL is anti-competitive, un-American, flea-infested, locust-plagued, etc. This followed up on similar claims by another VP, Jim Allchin, so it appears that this is the angle Microsoft has chosen to attack Linux.

On the face of it, Microsoft complaining about the source license used by Linux is like the event horizon calling the kettle black. Microsoft has no source license at all to speak of (for the general public anyway)! Anyway when did the source license of a piece of software become an issue for the average Joe who wants to buy it? As a former Microserf, I'll be the first to admit that Mundie's arguments make no sense for the industry in general.

Still from Microsoft's point of view, the source license of software does matter, because Microsoft might want to license the code. As it did in the case of the TCP/IP code it got from Spider, which was under the BSD license. If Spider's code had been under the GPL, Microsoft couldn't have used it. In other cases, Microsoft has licensed software from companies that were not open source at all, but Microsoft was able to negotiate a private license for its own use.

Again I'm not defending Microsoft's viewpoint. For any given piece of code, there may be five companies in the world that want to license it, 500 programmers who want to modify it, and 5 million users that want to use it. So the GPL is a good thing in general. Microsoft's main goal with the anti-GPL rhetoric is to discredit Linux, not to make it easier for it to license third-party software.

Still you can see how Microsoft's stance, from its own weird, warped perspective, almost makes some little iota of sense.

Sponsors
Voxel dot net
o Managed Servers
o Managed Clusters
o Virtual Hosting


www.johncompanies.com
www.johncompanies.com

Looking for a hosted server? We provide Dedicated, Managed and Virtual servers with unparalleled tech support and world-class network connections.

Starting as low as $15/month
o Linux and FreeBSD
o No set-up fees and no hidden costs
o Tier-one provider bandwidth connections

Login
Make a new account
Username:
Password:

Note: You must accept a cookie to log in.

Related Links
o Microsoft Uses Open-Source Code Despite Denying Use of Such Software
o FreeBSD
o Hotmail
o branch of the University of California
o the GPL is anti-competitive
o More on Software
o Also by adamba


View: Display: Sort:
Microsoft, TCP/IP, Open Source, and Licensing | 73 comments (68 topical, 5 editorial, 0 hidden)
MSFT uses linux on its zone.com servers (none / 0) (#73)
by strawdog on Wed Jun 27th, 2001 at 11:14:22 AM EST

They use linux on the statistics server for the zone (stats.zone.com) netcraft report here

Kerberos (none / 0) (#54)
by oohp on Thu Jun 21st, 2001 at 04:22:32 AM EST
(oohp@gotik.nu) http://devnull.ro

Can anyone point out what happened to Kerberos? I understand it was distributed too under a MIT/BSD license, Microsoft took the code, modified the protocol and never released any specifications (or they did release something). There had been a thread on bugtraq about this.

Screw BSD TCP/IP stack, what about MICA rumors? (4.50 / 4) (#52)
by bediger on Thu Jun 21st, 2001 at 01:48:46 AM EST
http://www.users.qwest.net/~eballen1/

Enough with apologizing for MS using BSD networking code - anyone with any sense does so. It's solid.

What about the rumors that NT is essentially MICA, the OS that Dave Cutler was working on at DEC before he joined MS in '88? I heard it 3rd-hand, but my source said that DEC got a sweetheart deal for Alpha NT because the NT source code was identifiably based on MICA.


-- I am Spartacus.
Thanks for this! (none / 0) (#46)
by leonbrooks on Wed Jun 20th, 2001 at 08:28:14 AM EST
(leon.brooks@fdns@net) http://users.smileys.net/~leonb/

On the face of it, Microsoft complaining about the source license used by Linux is like the event horizon calling the kettle black.

Congratulations! You are now a member of that most exclusive and elite of clubs, ``authors in my signatures database'' (-:
-- If at first you don't succeed, try a shorter bungee

Couldn't (3.50 / 4) (#38)
by ichimunki on Tue Jun 19th, 2001 at 03:49:14 PM EST

You say that if Spider's code had been under the GPL that Microsoft couldn't have used it. But what you need to say is wouldn't have used it. And there is a huge difference between ability to do something and willingness to do so.

I would think that not one single MS user would have been affected at all, and that MS would have had an easy time complying with the GPL, by simply having a folder on one of the install CDs containing the source code to any GPL tools used in the OS.

Microsoft's recent statements about Open Source and Linux seem to me to be a deliberate misstatement about the nature of the license and the Free Software movement. Recall that Bill Gates penned the "Open Letter to Hobbyists", and to me this says more about his motives than anything. He has gone from making "less than $2 an hour" to being one of the richest men in the world. His decision to avoid "cancerous" software has done him a lot of good apparently, and from what I hear his employees don't do too shabby either... but his users? Well, that's for each user to answer, I guess. But I always get a sense of sticker shock when I see what Windows and office licenses cost compared to the value they might add to my life.

i did some work related to the hotmail team (3.50 / 8) (#34)
by lb on Tue Jun 19th, 2001 at 12:36:56 PM EST

I don't see what the big deal is. There are a few employees at Microsoft who are open source fans. Quite a few of the people I worked with actually. These people will try to leverage open source in their daily work.

I wrote quite a few scripts in Perl to automate the build process on the project I was on. (From what I hear they're still using them. :) The machines were Suns, but only because the code for that specific project was inherited half-complete, already designed for Solaris.

I don't see why this is a big deal, and I don't really see how Microsoft has egg on it's face here.

Yes, I had a linux box plugged into the Microsoft redwest network. Who cares?

-lb

STREAMS (3.66 / 3) (#31)
by ksandstr on Tue Jun 19th, 2001 at 12:23:05 PM EST
(ksandstr@gmail.com)

ICBW, but isn't STREAMS supposed to be more like a generic interface to an operating system kernel's internals (and the underlying hardware), so that when a proprietary chickenshit company needs to write a driver for solaris, fooBSD and other x86-with-STREAMS compatible systems they'd only have to implement the driver for STREAMS, once?

Also, didn't the TCP/IP stack in NT 4 have a fingerprint alarmingly similar to the one in wossnameBSD "reno"?



What has taken up residence in the Princess's rectal cavity?
Modules using BSD code. (3.00 / 8) (#26)
by ucblockhead on Tue Jun 19th, 2001 at 11:05:31 AM EST
(ucblockhead at is.worsethanhitler.org) http://www.ucblockhead.org/journal

Searching for the string "Regents" in all files in the \winnt directory tree of Windows 2000 gives you just these five files:

finger.exe
ftp.exe
nslookup.exe
rcp.exe
rsh.exe

Hardly earth-shattering.
-----------------------
This is k5. We're all tools - duxup

Microsoft could license GPL code (4.10 / 10) (#25)
by darthaggie on Tue Jun 19th, 2001 at 10:56:57 AM EST
(sy_nttvr@gurcragntba.pbz)

If Spider's code had been under the GPL, Microsoft couldn't have used it.

If Spider had been the sole copyright holder of the code, they could have worked out a seperate agreement with M$. Why do people presume that if code is licensed under GPL, it can not be licensed under different terms by the copyright holder? GPL is viral, but it isn't that viral.

Now, if there are multiple copyright holders, licensing GPL would be an adventure.

I am BOFH. Resistance is futile. Your network will be assimilated.

Astroturf? (1.83 / 6) (#14)
by agent on Tue Jun 19th, 2001 at 09:00:28 AM EST
(ntrag@fqs.ybarfgne.bet) http://agent.freeshell.org

Why should this be of any surprise at all? Of course Microsoft would rewrite the network stacks somewhat with each upgrade of Windows. Given that 90% of the systems run some version of Windows, I'd rather have a BSD derived IP stack communicating than a newly coded, buggier stack that was denied the benefits of 'Standing on Giants.' Atleast it looks like this is one open protocol that will not be embraced, extended, and proprieterized.
--
ROT13 my email address to contact me
AIM: TheAgen7
A few points (3.33 / 12) (#13)
by jd on Tue Jun 19th, 2001 at 08:58:19 AM EST

First, anyone can "fingerprint" a TCP/IP stack, without having access to the source itself. There are a variety of algorithms, but they mostly seem to depend on testing behaviour under a variety of nonsensical condtions.

Because no two people are likely to implement -EXACTLY- the same behaviour under conditions that will never arise under normal usage, those behaviours are pretty good indicators of what software is being used, where.

*Behaviour in this context may refer to a number of things, depending on the exact fingerprinting technique used. I believe nmap uses a mix of error codes and other responses. However, that is not the only method, or even necessarily the best. (There is at least one article linked to, from nmap's own webpages, which takes nmap to task for using older fingerprinting techniques.)

Second, as I understand it, NT was tried in Hotmail, but was withdrawn after it completely failed to survive. Windows 2000, NOT NT, is the latest attempt to migrate Hotmail from *BSD to a Microsoft OS.

Last, but by no means least, NT IS NOT WINDOWS 2000! They wrote Windows 2000 from scratch (near enough). All 15 million or so lines of it. It contains essentially no pre-existing Microsoft code. That the Windows 2000 developers might have taken short-cuts by scrounging existing *BSD code (which they already knew was superior to their own Wolverine-based TCP/IP stack) comes as no great shock. They were behind schedule, the cost over-runs must have been truly horrible, and the PR people must have been begging them for some good news. If I had been in their shoes, I'd have opted for quick-fixes, too.

I am shocked, shocked to hear that MS has lied! :) (4.20 / 10) (#12)
by Zukov on Tue Jun 19th, 2001 at 08:57:26 AM EST

The exact details of what stack was used where are not important. The people who decide what software a company uses normally know nothing about the details of the software.

The paper version of the WSJ (which is as pro-business a paper as you can get) has a summary on page one, column 2, above the fold:

"Microsoft has been using open-source computer code, despite denying it did so and campaigning against the software"

People who have been authorizing the purchase of MS products might think to themselves

  • Open-source is good enough for MS
  • Open source can be free (cost)
  • Why are we paying MS for software instead of using open-source?
  • Why does MS bash open-source and then use it in the products we pay them for?

    MS would have done better to ignore open-source, or say it is too difficult for the average user, etc.

    The way this has played out is the worst possible public relations disaster for MS- open source gets mentioned on the front page of the WSJ, and MS is shown to be reselling software it publicly bashes.

    ¿ëë±È¶ ±Hæñ ¥ØÜ (§^Ð

    Yes, I have just bumbled upon Gnome Character Map. Please ! me.

  • Winsock: not a Bad Thing... (3.91 / 12) (#8)
    by SpaceHamster on Tue Jun 19th, 2001 at 05:56:47 AM EST
    (nick@nospam.msn2.net) http://www.msn2.net

    as far as Win32 APIs go.

    Its 3am. I just spent the previous 4 hours of my life debugging Winsock code. Turned out to be my own fault (forgot a select() call, code would spin when it shouldn't have). Yet, even though my network layer is pretty damn complex and looks like spagheti, it works solidly. This is not something I can say of the rest of the API. Jeez. I've spent too many hours in the last couple years pulling my hair out when functions don't work the way the're documented to. Or even worse, when the documentation is bad, and the headers are not self explanatory, just figuring out the calls to make (and in what order, and with what flags) can be a multi-day task.

    But in the case of Winsock, most all the BSD socket calls are there, with most of the options intact. Its pretty well documented. Heck, I learned socket programming in order to hack some old MUD code that compiles with the same network layer on almost every *nix and win9x/nt. The calls don't change over time, and (unlike a LOT of other Win32 calls) don't work differently under different versions of Windows.

    So you know what, if MS wants to lift some network code from BSD, more power to them for not wasting time reinventing the wheel. Every windows user/programmer benefits, no one is hurt, so please don't anyone bitch about it.

    Stacks; licensing (3.80 / 10) (#7)
    by sigwinch on Tue Jun 19th, 2001 at 05:22:48 AM EST

    And implying that the TCP/IP stack uses BSD code is also false. As I said above there may be small vestiges of it in there, although I doubt it.
    FWIW, there are persistent rumors that Windows 2000 did include BSD networking code.
    Anyway the FreeBSD programmers who reported all this to the Wall Street Journal can't see the NT TCP/IP source either, so they can't have been referring to that.
    They can't? <grin> How naive. ;-)
    On the face of it, Microsoft complaining about the source license used by Linux is like the event horizon calling the kettle black.
    I'm stealing this for my .sig.
    If Spider's code had been under the GPL, Microsoft couldn't have used it.
    They most certainly could have. They would have chosen not to because their central goal is to use technology to make their customers artificially dependent on them. Some people sell razors. Other people sell razor blades. Microsoft sells shaving cream containing 5% hair growth hormone.

    --
    I don't want the world, I just want your half.

    What you seem to miss... (4.07 / 13) (#4)
    by sombragris on Tue Jun 19th, 2001 at 02:42:27 AM EST
    (nomail@spam.die.die.die)

    is that Microsoft used the GPL-bashing as a broader attack against Open Source. The main thrust of the article is that while MS bashed Open Source, it was busy porting lots of BSD code in the back end. For us, that was near the obvious, but it is relevant that the press could realize that and run this story.

    Great article! (3.71 / 21) (#2)
    by MoxFulder on Tue Jun 19th, 2001 at 01:33:07 AM EST
    http://myxo.css.msu.edu/danimal

    This article should make both the knee-jerk MS-bashers and the knee-jerk MS-defenders stop and think for a moment. As the author explained very convincingly, Microsoft has used open source BSD code quite selfishly, for its own ends, but has done so quite legally.

    There isn't anything apparently illegal about what Microsoft has done with the BSD code, but it is perhaps hypocritical, considering their recent open-source bashing.

    This article made me think about my own opinions of the BSD and GPL licenses. I think I tend to come across as a bit of a GPL zealot, but I can see how in some cases, BSD is a better choice. If the BSD sockets code had been under the GPL, Microsoft wouldn't have used it, and likely as not they would have developed a completely incompatible TCP/IP programming interface. As it is, sockets have become something of a standard, and Unix and Windows programmers can understand and modify each other's TCP/IP code without too much trouble. So in this case, the BSD code has won a victory for interoperability.

    Thanks to adamba for the informative, level-headed article!


    "If good things lasted forever, would we realize how special they are?"
    --Calvin and Hobbes


    Microsoft, TCP/IP, Open Source, and Licensing | 73 comments (68 topical, 5 editorial, 0 hidden)
    View: Display: Sort:

    kuro5hin.org

    [XML]
    All trademarks and copyrights on this page are owned by their respective companies. The Rest © 2000 - 2005 Kuro5hin.org Inc.
    See our legalese page for copyright policies. Please also read our Privacy Policy.
    Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
    Need some help? Email help@kuro5hin.org.
    If you can read this, you are sitting too close to your screen.

    Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories! K5 Store by Jinx Hackwear Syndication Supported by NewsIsFree