From: "Doug Bissett" <virtualized_ecs_users@2rosenthals.com>
Message-ID: <000.80ea0d00c7eb234b.028@telus.net>
To: "Virtualized eCS Users Mailing List" <virtualized_ecs_users@2rosenthals.com>
Date: Sat, 12 Dec 2009 12:15:19 -0700 (MST)
In-Reply-To: <list-1986857@2rosenthals.com>
References: <list-1986857@2rosenthals.com>
Priority: Normal
User-Agent: PMMail/3.07 (os/2; U; Warp 4.5; en-CA; i386; ver 3.07.04.1489)
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Subject: Re: [Virtualized eCS] TAP driver killing TCP/IP bandwidth

On 2009-12-11, at 21:23:28, Lewis G Rosenthal wrote:
>
>Hi, Doug...
>
>[long post warning: this message contains a number of stats in the body]

[I snipped a lot of the detail - anyone interested should look at the
message that this replies to]

...snip...
>>> Indeed, I thought of the auto-sensing circuitry, but don't know if the 
>>> T43 or the new Intel chip (still trying to get GenMAC to work with it) - 
>>> an 82567LF-2 - do auto-sense. The switch surely does, which is why I no 
>>> longer stock crossover cables. :-)
>>>     
>>
>> I did my tests with a standard cable (probably CAT5) that is about 5
>> feet long. No trouble with not using a crossover cable.

Yes, it is a CAT5 cable, but the short length should make that less of
a problem.
   
>I was able to successfully connect a CAT-6 straight through between a 
>workstation and the ThinkPad. The interesting discovery I made was that 
>even though I have power management at a bare minimum, evidently, while 
>on battery power, my wired NIC is throttled down. The best I could get 
>(no TAP driver) was about 480Mbps on battery. Plugged into AC, however, 
>I was able to eek out >830Mbps.

I wonder if it is the NIC that is throttled down, or if it is slower
because the processor is throttled down? It would be interesting to see
what that other OS does.

>There was only a minor difference when going through the switch 
>(actually, a well functioning switch *should* improve overall 
>performance - and indeed, with the NDIS2 driver, it really did for me), 
>however, the TAP driver was a killer. With an active VM, it got even 
>worse (see below)...

I would think that going through the switch would not make much
difference, unless there are a number of active systems fighting for
time.

>>>>> Has anyone else seen this type of performance hit cuased by the TAP driver?
...snip...
>My tests using the B57 driver were just as abysmal under AC as they were 
>under battery power (no TAP driver loaded). That said, on the ThinkPad I 
>got similar results to what I saw on the server, so I suspect it is just 
>a poorly written device driver base from Broadcom (different versions, 
>but similar lack of optimization). Does anyone have an NDIS2 driver 
>running at close to 1Gbps?

Apparently mine doesn't. I suspect that to get full speed, the systems
would need to be pretty powerful, and highly optimized.

...snip...
>> The really interesting numbers are in TCP, and NETBIOS, with and
>> without the TAP driver installed. Sending seems to be limited to what I
>> see with my 100 Mbs hub in the network. With a direct connection, and
>> no TAP driver, send and receive seem to be about the same. The
>> interesting thing is that receiving doesn't seem to be affected by
>> whatever is slowing down sending.
>>
>>   
>If you used a CAT-5 cable vs a CAT-5e or a CAT-6, this might account for 
>the results. Also, without locking the NICs down to 100Mbps, and with no 
>100Mbps switch in between, they might be trying to shift up to 1000Mbps, 
>which will likely fail over a CAT-5 connection, causing them to resend 
>and/or renegotiate. It's hard to tell without seeing the actual packets.

I don't think a short (5 foot) cable would have too much trouble.
Looking around, I think I have a CAT5e cable, that I can try (when I
find the time). I suspect that the T43 is the bottle neck in my setup.
I forgot that it was throttled back to keep the temperature down. I
also have SWITCH.OS2 (for Virtual PC, if I remember correctly)
installed on both sides. Have you checked that to see if it affects
speed?

>I didn't test UDP; only TCP:
...snip...
>
>Ugh... Perhaps some RAM tuning for the guest might tweak the numbers a 
>bit; I didn't try. Even though I have 2GB in the T43, I can't take the 
>guest up to 1GB, due to memory fragmentation and lack of memory in the 
>shared arena. I would expect, however, that slightly more RAM devoted to 
>the host might improve performance here, rather than vice-versa.

I have found that it is necessary to give a guest as much memory as
possible, to be sure that it never has to page memory in the guest. I
am also not very surprised that the guests seem to be throttled to
something relatively slow. However, I don't see any reason why the host
should be throttled by the TAP$ driver. This is somewhat reminiscent of
the early 100 Mbs support, where some systems would set up to run at 10
Mbs, "just because".

>> As an aside, I discovered that the MPTS in eCS 2.0 Silver has added a
>> feature to install LLAeCS.EXE into \MPTN\BIN\MPTSTART.CMD, every time
>> it runs (even if you don't change anything). This took me on a wild
>> goose chase for a while, since LLAeCS.EXE hangs at boot, unless I start
>> it much later than in MPTSTART.CMD.
>>
>>   
>For the record, I *hate* that M$ LLA nonsense. If I have no static 
>address and no DHCP server, then by g-d, I don't want a fake IP address. 
>If I get a bunch of machines together to chatter amongst themselves, 
>they're either statically assigned or one of them is running DHCP. IMO 
>LLA is more trouble than it's worth, and a real hassle for our stack to 
>deal with, on top of everything else.

LLAeCS seems to help to make it possible to switch between wired, and
wireless. The problems are in the initial startup, where it seems to
get involved before the standard (DHCP) setup has finished doing what
it does. The fake address seems to be necessary to prevent NETBIOS over
TCP/IP from getting hung up with a 0.0.0.0 address, when DHCP fails to
assign one for one reason or another. Of course, if you understand what
is going on, it is easy enough to work around it. LLAeCS is for hose
who don't understand what is going on (probably about 90% of users),
and for those who don't want to be bothered with it (another 9%). I
don't understand why, but I was told that LLAeCS is also "necessary"
for machines with a single, wired, NIC.  I didn't bother to ask why. I
mentioned it to warn others that the line is restored to MPTSTART.CMD,
when you run the Adapters and Protocols program, whether you want it or
not. I haven't tried to make the file read only, to see if that stops
that problem. When LLAeCS is started from MPTSTART.CMD, on my systems
(all of the ones where I have installed eCS 2.0 Silver), the system
hangs during boot. It seems to be okay, when it gets started much later
in the sequence.

>> This is somewhat limited testing, but I think that it shows that the
>> TAP driver does limit send speed to something near 100 Mbs, even when
>> using a gigabit NIC. In different configurations, the results could be
>> different.
>>
>>   
>Note mine... :-(
>> I never tried wireless, but I will do that, when I find some time to
>> play with it again.
>>   
>Good luck getting the TAP driver to bind to the wireless NIC. The trick 
>which Andy showed me (and this was under VPC, before we had VBox) was to 
>keep it bound to the wired NIC, and enter a static route to allow 
>traffic to bounce back and forth off of the wired interface to the 
>wireless. To get this right, you really need to "think like a packet," 
>as a friend of mine likes to say. Also, this arrangement obviously 
>doesn't work in a bridged environment (all of my tests were bridged). We 
>might want to try a NAT configuration, I guess...

I don't know why binding to a wireless NIC would be any different than
binding to a wired NIC. It would, of course, limit speeds to what the
wireless can do. Perhaps the problem is that the TAP driver sets the
speed to 100 Mbs, no matter what speed the actual connection can run
at, and then has trouble because it doesn't want to slow down to
wireless speeds.

NAT is something that needs to be tested, but that should only affect
guest systems, I think.

>Cheers/2, and thanks for testing this with me. Much appreciated.

More testing to do, with so little time :-)

At the moment, I am going to spend some time with the latest ACPI 318
(FULL) package that was put into the testzone a few days ago. With eCS
2.0 GA pending, it seems logical to work on that, since there is some
(slim) possibility of getting problems fixed before it is released.
Getting something fixed in the TAP driver would seem to be unlikely,
unless we can find a way to do it ourselves.
-- 
****************************
From the eComStation of Doug Bissett
dougb007 at telus.net
****************************
... I stopped pursuing the American dream, when I found out it means I'd need to move to France.