Weird Issue with Broadcom NetXtremeII

vahtryn · n00b Joined: 12 Jul 2005 Posts: 27 Location: SF

I have just the Broadcom NetXtremeII driver compiled into my kernel (non module as I have no modules). The nic seems to come up just fine after a reboot, ifconfig looks fine, default route is added in properly. The thing is it can't pass traffic what so ever. Doesn't receive arp from anything what so ever either. A quick /etc/init.d/net.eth0 restart allows for it to have network connectivity.

The machine is a dell 1950, here's what Dell has to say about the built in NICs: Dual embedded Broadcom® NetXtreme IITM 5708 Gigabit Ethernet NIC with fail-over and load balancing

here's my /etc/conf.d/net:

cat /etc/conf.d/net
config_eth0=( "10.160.65.23/16" )
routes_eth0=(
"default via 10.160.0.1"
)

Anybody seen this before?

-edit
I'm using the current gentoo-sources which is 2.6.16-r9

Kronos · Posted: Wed Jul 05, 2006 8:50 pm Post subject:

What are you doing to see you're not getting arp?

What does dmesg say about it?

Can you check /var/log/messages or whatever the relevant log file is?

What does ethtool eth0 say?

To be sure, you're saying it works fine after a restart of net.eth0?

vahtryn · n00b Joined: 12 Jul 2005 Posts: 27 Location: SF

I'm doing an arp -an, I get an incomplete arp entry from the default gateway of the network. I also do not receive an arp entry on my mbp when pinging to the server, and I should as I'm on the same vlan (I have an arp entry now since I'm sshed into it).

dmesg brought up this:

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.4.31 (January 19, 2006)
bnx2: eth0: using MSI
bnx2: eth0 NIC Link is Up, 100 Mbps half duplex

ethtool output:

ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Half
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: d
Wake-on: d
Link detected: yes

from /var/log/messages:

Jul 5 08:27:26 prod-everything Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.4.
31 (January 19, 2006)
Jul 5 08:27:26 prod-everything ACPI: PCI Interrupt 0000:09:00.0[A] -> GSI 16 (level, lo
w) -> IRQ 169
Jul 5 08:27:26 prod-everything eth0: Broadcom NetXtreme II BCM5708 1000Base-T (B1) PCI-
X 64-bit 133MHz found at mem f4000000, IRQ 169, node addr 001372f7f2c9
Jul 5 08:27:26 prod-everything ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 16 (level, lo
w) -> IRQ 169
Jul 5 08:27:26 prod-everything eth1: Broadcom NetXtreme II BCM5708 1000Base-T (B1) PCI-
X 64-bit 133MHz found at mem f8000000, IRQ 169, node addr 001372f7f2c7
<b>this is my /etc/init.d/net.eth0 restart</b>
Jul 5 08:36:25 prod-everything sshd[3246]: Received signal 15; terminating.
Jul 5 08:36:26 prod-everything bnx2: eth0: using MSI
Jul 5 08:36:26 prod-everything sshd[4173]: Server listening on 0.0.0.0 port 22.
Jul 5 08:36:28 prod-everything bnx2: eth0 NIC Link is Up, 100 Mbps half duplex
Jul 5 08:40:14 prod-everything sshd[4435]: Accepted keyboard-interactive/pam for root f
rom 10.160.51.1 port 60321 ssh2

Finally yes, after restarting /etc/init.d/net.eth0 it works fine.

Kronos · Posted: Wed Jul 05, 2006 9:13 pm Post subject:

Can we get a post of what ethtool says before you do an /etc/init.d/net.eth0 restart?

vahtryn · n00b Joined: 12 Jul 2005 Posts: 27 Location: SF

cat eth.tool
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: Unknown! (65535)
Duplex: Unknown! (255)
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: d
Wake-on: d
Link detected: no

I just piped it into that flat file from the console. Hrm, odd seems that it can't auto negotiate properly.

Kronos · Posted: Wed Jul 05, 2006 10:28 pm Post subject:

What is the other end of eth0 plugged into?

vahtryn · n00b Joined: 12 Jul 2005 Posts: 27 Location: SF

Kronos · Posted: Wed Jul 05, 2006 10:44 pm Post subject:

Check if the port is flapping on the cisco. Maybe it is misconfigured. Or it's a flapper. When you see that from ethtool it often means your port on the other end is either admin down or was in a down flaped state or something. Or a really weird config prob I've seldom seen.

You could sho int gx/yy and sho log | include GigabitEthernetx/yy or whatever your setup is there.

Kronos · Posted: Thu Jul 06, 2006 12:44 pm Post subject:

Of course that would just be coincidence. Have you had a chance to check on the network gear? Considering a network restart always fixes this, it could just be coincidence.

You could always do a watch -n 0.1 ethtool eth0 after boot and watch for a while. See if anything (like speed, duplex, link) flaps. A less /var/log/messages | grep eth0 would show that too though.. anything there?

Other than that it sounds like it is an issue with the module not setting/detecting speed/duplex right. Of course, this could be an issue if you have net.eth0 on a different runlevel than default. rc-update show default will show you everything in the default runlevel. net.eth0 should be in there.

Let me know as you progress...

vahtryn · n00b Joined: 12 Jul 2005 Posts: 27 Location: SF

Well can't figure out why it won't start up on boot. So I've just added into the /etc/conf.d/local.start /etc/init.d/net.eth0 restart.

Temporary fix while I continue to work on this issue but I don't ahve to go to the terminal after a reboot for testing purposes so I don't really mind too much.

Kronos · Posted: Fri Jul 07, 2006 3:49 am Post subject:

Yeah but unfortunate you have to do that, and other folks probably will have the issue eventually. Maybe there's something wrong with the module... who knows. Did you look at the log on the network device?

vahtryn · n00b Joined: 12 Jul 2005 Posts: 27 Location: SF

sgao · Tux's lil' helper Joined: 22 Apr 2006 Posts: 149

I saw the same problem here.

With PowerEdge 1950, try installing with Gentoo 2006.0 x86_64 mini install CD. Could not get ip through dhcp. Manually set ip address, but could not connect to anywhere. The driver used is bnx2.

It seems working ok with Gentoo 2006.0 x86 install CD.

Anyone made any progress on this issue?

Simon

Suicidal · l33t Joined: 30 Jul 2003 Posts: 959 Location: /dev/null

annunaki2k2 · Posted: Fri Aug 11, 2006 4:21 pm Post subject:

Has anyone found a solution to this problem?

I have 8 new servers waiting to have Gentoo installed on them, but have the same problem as described above.

They are all Dell 2950s with the integrated Broadcom 5708 (NetXtreme II).

The bnx2 driver is loaded, and the eth{0,1} devices become available. But ethtool always shows the link as down:

GentooStarter · n00b Joined: 03 Jan 2004 Posts: 65

I had problems with this (onboard) NIC as well... The strangest effects and errors.. very unstable etc.

If you select the broadcom tigon driver instead of the netextreme in your kernel driver all you problems will go away :-)

I have tried this on 3 different (HP) machines using this chipset.. works on all of then !

Splink · Apprentice Joined: 09 Feb 2005 Posts: 295

There is a documented problem within Alcatel worldwide about broadcom cards connecting to their 6600 series switches. The issue is with trying to set autonegotiation - the card will just appear to hang, until you force it to goto 100/half 100/full.

We have about 400 of them here all with the same issue - solution - change vendor

_________________
It said "Requires Windows XP or better"
So I installed gentoo.

aZZe · Posted: Wed Sep 20, 2006 6:25 pm Post subject:

I have the same problem on a DELL Power Edge 2950. The problem is quite the same as described in the upper threads. Ethtool tells me that the network card is not connected. When I use a Gentoo 2006.1 amd64 Live-CD everything is fine. Additionally i have three Intel Dual Port Pro/1000 network cards in this server and these cards don't work too. So including the onboard Broadcom card I have 6 NICs. Again all 6 NICs work fine with the Gentoo Live CD! When I boot the system from harddisk I cannot send any data to the network. So I would like to know the difference between the Live CD and my native system. I use the modules e1000 for the Intel cars and the module bnx2 for the Broadcom NetXtremeII onboard card.

We don't use Cisco Switches. We use HP ProCurve switches. The switch is 100% o.k. The log files are clean.
_________________
ASUS 4A785TD-V EVO with AMD Phenom(tm) II X4 965
4GB RAM DDR3-1333
ATI Radeon HD4870

aZZe · Posted: Wed Sep 20, 2006 6:39 pm Post subject:

Dark_Angel · n00b Joined: 17 Mar 2006 Posts: 37

Hi, I have this ethernet controller, havn't so much read the thread, but in respounce to the message, I have both the tg3 and bnx2 loaded to control it.

aZZe · Posted: Fri Sep 22, 2006 2:05 pm Post subject:

aZZe · Posted: Sat Sep 23, 2006 1:16 pm Post subject:

The tg3 module can't be used with the Broadcom NetXtremeII card. When you compile tg3 and bnx2 as modules and don't bind them statically into the kernel you will see that you won't have any NICs when the tg3 module is loaded and the bnx2 module unloaded. So the right module for this card is bnx2.

I also searched in the kernel config for some missing modules that might be needed but I don't have a guess. So if one of you guys has a result please post it.

Maybe I should say that I build a 64Bit System (IntelEMT64). Would be interesting if one of you had tried the same or used a 32Bit System.
_________________
ASUS 4A785TD-V EVO with AMD Phenom(tm) II X4 965
4GB RAM DDR3-1333
ATI Radeon HD4870

sepiriz2005 · n00b Joined: 09 Oct 2006 Posts: 2

Same problem here on a DELL PowerEdge 2900
Same network adapter

The difference is that it half works.
Let me explain:
I have two gentoo systems installed: one is classically install from the lice-cd x86 and works fine; the other one in a kernel compiled from xen-source.
Both have bnx2 but only first one works.
On second ethtool says same as seen in above posts.
The /etc/init.d/net.eth0 restart tip doesn't work for me.
Anyone has a clue?

Here is what zcat /proc/config.gz says for working system:

sepiriz2005 · n00b Joined: 09 Oct 2006 Posts: 2

Just found a solution.
Changed bnx2 support in my kernel from built-in to module and works like a charm.

aZZe · Posted: Mon Nov 20, 2006 8:43 pm Post subject: