[PATCH] Enabling (more) MSI usage for AMD SB700

[PrakashP]

I poked around with my chipset, so that more devices can use MSI und thus I don't have shared interrupts anymore:

[asturm]

That's finally a good reason to start my AMD system again. I think it should have the SB750 on it.

[zbiggy]

Great job PrakashP,

if I will have AMD chipset mainboard I will use your patch. Why you do not use MSI for radeon?
Now I have nForce 720/Geforce8200 chipset and apic/acpi is putting everything on interrupts 20-23. ohci is hanging after few minutes of use - I had to turn off apic and it works now. I was able to offload ahci, hd audio and eth0 to msi so my system is more stable and looks to be faster. Unfortunately I was not able to move nvidia binary blob to MSI using nvidia NVreg_EnableMSI=1 parameter. It hangs after few minutes of using. However this happens on my integrated Geforce 8200. People using discrete Geforces are fine.

I was unable to turn on MSI for this devices:

[PrakashP]

[roarinelk]

Hello,

Here's an updated patch against 3.3-rc1. I reimplemented the OHCI/EHCI MSI enable code
since the old one was interfering with the XHCI controllers' MSI-X.

It boots and runs on my SB850.

[DaggyStyle]

if this indeed introduces gains, why not submitting it to the kernel mailing list?
_________________
Only two things are infinite, the universe and human stupidity and I'm not sure about the former - Albert Einstein

[roarinelk]

There are no real gains as far as I can see. Yes it's easier to direct an MSI interrupt
to a certain CPU when the line isn't shared, but who does that anyway?

MSI support is usually disabled because it is more or less untested and buggy
(I've already had ALSA complain about non-responding audio hardware).

Consider the patch a proof-of-concept that MSI on SB700/800 southbridges
works but that it is disabled usually for a reason.

[PrakashP]

@roarinelk

Nice, I will try your changes to the usb stack. Out of curiosity, could you post your /proc/interrupts and lspci -nnv?

What is the problem with xhci? Doesn't is work using MSI? Might it need quirk_msi_intx_disable_ati_bug quirk?

@DaggyStyle

Some parts of the patch are obviously hackish, so won't get accepted. The kernel maintainers also don't want the hassle with introducing MSI support in the ohci and ehci drivers. The reason bios disables most of the MSI stuff is due to AMD advising so - if you look into the datasheets. Usually they don't give a reason why, though. I guess it is because of the needed quirks that the legacy interrupt pin must not be disabled when using MSI, as then the AMD chipsets (at least older ones like mine) won't give any interrupts anymore. So for every device on the affected AMD chipsets you need to set the quirk_msi_intx_disable_ati_bug quirk. I doubt kernel maintainers would like it done this way. So I guess the only part of the patch which might have to chances to get included is the Azalia/Intel HDA part, as it is least hackish. (To be honest it is not documented in the public SB700 datasheet, but in the Sb800 one, so I tried my luck. ) If I can make sure, it works for different SBx00 versions, I might try to submit that part.

[roarinelk]

xhci calls usb_add_hcd() for its ehci companion, _after_ it has enabled MSI-X for itself.
Placing pci_enable_msi() in usb_hcd_pci_probe() screws up xhci int handling; the ehci is part
of the xhcis register space which means it has the same struct pci_dev.
in the list below, xhci is bus2, its ehci companion is bus1. the ati controllers are 3-9.

[DaggyStyle]

[PrakashP]

http://en.wikipedia.org/wiki/Message_Signaled_Interrupts

It's up to you to use it, I cooked it up for my personal interest and just shared my result. Why not use the chipset's full potential, if it is possible?

[trubicoid]

Did you tried some "benchmarks", like latency or CPU load?

I have two affected computers, so I'll try it. Now they don't use MSI a lot:

[PrakashP]

Nope, I didn't try to write any benchmarks. BTW, I suggest you try roarinelk's patch. It's more robust than mine. (Update) My version now includes roarinelk's changes.

I find it it interesting that both your AHCI controllers have MSI disabled. I remember there was some erratum with S3 or alike. Anyway, I can look up how to enable MSI on AHCI, as well.

[potenzial]

[PrakashP]

Ok, I updated the patch to inlcude roarinelk's changes and also added SATA MSI enabling. Therefore I disable D3 power management to be on the safe side. So using S1 Standby is no problem. The RPR actually says to disable MSI on older revisions, but I guess that's just because of the INTX disable quirk. It works fine on my system.

@potenzial

Did you already apply the patch and then dumped lspci? Because then I wouldn't understand why MSI for OHCI wouldn't be enabled. What is your southbridge chipset? Nice system, btw.

[potenzial]

[PrakashP]

Well, it would be helpful to tell me the chipset you motherboard has.

[potenzial]

It'a ASUS Crosshair 5 and the chipset is AMD 990FX/SB950
But with your patch everything is using msi.
Thanks for the help.

/proc/interrupts

[j_c_p]

I've tried the patch too :

[s4e8]

I work beatiful here. But I'd like to have 64 bit DMA in ehci controller either. It think 64bit DMA is more important than MSI interrupt. Intel ehci controller support 64bit DMA, but AMD/ATI not. In order to enable 64 bit DMA, you must check and set dma_mask before ehci_mem_init(). The kernel source has some commented code in ehci_run(), it's wrong place, because ehci_mem_init() allocate/map DMA buffer before ehci_run().

[roarinelk]

[maudeb]

PrakashP's patch works like a charm here too:

- Debian Squeeze/Sid w/ Liquorix 3.3.6 kernel
- Asrock 870 Extreme 3 (AMD 870 + SB850) + Phenom II X4 965

...but I still can't see HPET_MSI-edge (BLK_DEV_FD is not set). I'd really like if someone could port this on the kernel tree.

Thanks! Mau

[roarinelk]

[_______0]

[PrakashP]

I am always running a relatively new kernel from the git tree and thus forward-port the patch. And yes it works (for me), but I haven't posted any updated patches. I cannot guarantee though that no bugs emerge. Only AMD can give that guarantee.