Nvidia Wrong Resolution Solution

[Graymi]

Way back when i started using Gentoo on my laptop, i soon discovered that using the "proper" nvidia-drivers from portage caused me the hassle of not detecting my display properly, and giving me a resolution of 800x600 and not the full 1024x768.

I soon discovered ignoring my EDID solved this.

However a not-so-recent rebuild meant that nvidia-legacy drivers had gone, and unfortunately ignoring the edid with non legacy drivers lead to what i shall call the rainbow effect, where my xserver simply drew pretty colours on the screen (pretty useless colours at that)

after hours of tweeking, with various tips not working,i eventually discovered that through mode-lines and ignoring some of my edid, i could get my display running at 968x768 resolution (i think that's what it was) obviously not ideal, owing partly to the annoying fact of a black bar running down the right side of my screen.

well those pixels are once again active! I finally did some reading up on the subject.

and here's how to get your laptop running at full resolution. (definately works on my toshiba sp6100... and if i undestand what i've done correctly don't see why shouldn't work on others)

/----------------------------------\
\----------------------------------/

Firstly, we know our EDID is wrong. but it's actually a very simple change to make to correct it. Step one is to use the nivdia-settings application to create a copy of the incorrect edid somewhere on your computer. I chose /etc/X11 simply to keep everything together.

once you've done this, you need to load the file into your favourite hex editor.

Now look for the following Bytes: 56 and 58 take the upper part of 58 (the first of the two hex-digits) and put it infront of the whole of 56 to create a 3 digit hex number. in my case 3C9. Convert this to a decimal number, and what do you get? well, the wrong horizontal resolution, that's what (969... such a daft number)

Well, time to correct that. Using your favourite decimal-hex converted program (calc on my windows box in this case I'm afraid folks... I'm stuck in console on the laptop) tap in the correct horizontal resolution, and convert it to hex (in my case 1024, becomes 400)

Now the tricky part. take the first digit, and replace the first digit of byte 58 with that (4) and the remaining two digits, and replace byte 56 with those (00)

save the edid file

now pop open xorg.cfg

Remove any ignoreEdid or useEdid false config options. and put the following in

[spindle]

Sweet. Nice trick. Thanks for sharing this (from here). I'll probably try this in a few days myself and get up to date on my nvidia-drivers.

In the console you could have used vim for your binary edit (vim -b file.bin). But that may require a lot more beer if you're not used to vim, but it's so worth it in the end, trust me.

[Graymi]

I'm a big vim fan. Haven't got a clue how to use it properly in the console (still haven't figured out copy nd paste )

but it replaced notepad at work ages ago

[spindle]

This doesn't seem to quite work for me I don't have those 56 and 58 values in my edid.bin.

[Graymi]

ah, a slight overview on my behalf... 56 and 58 are bytes 38 and 3a (dec->hex... whoops lol)

so, the two values you'd be interested in are underlined here...

0000030: b300 81c0 8bc0 2139 9030 621a 2740 68b0 ......!9.0b.'@h.

seems your monitor thinks its horizontal value is 690 (1680 in normal numbers)

[spindle]

[spindle]

Nevermind, I got this working. Mine had a checksum mismatch problem I'm pretty sure.

[Graymi]

glad ot hear you fixed that. checksum error, nasty.

as for a file spec, i vaugly remeber something lurking around wikipedia, but really cba to go looking for the link right now as its past my bed-time

just try searching for EDID

[spindle]

Searching for EDID found that wikipedia article you mention I assume. Looks like there's a link to the spec there in the external links section at the bottom. Here's a direct link to the pdf.

Thanks for the info. I might not need the spec for now as this seems to be working great for me now. I'll note it as it might make some interesting reading sometime and if anyone else needs it.

[rockdw]

I actually came across a less "low-level" way to fix this on my Dell Inspiron 8200 (GeForce2 Go)

I had the same problem with the 800x600 and ended up using xorgcfg to create a new xorg.conf (it's referenced in the X config HOWTO) That in itself wasn't all that exciting, except for the fact that everything was now working! Upon closer inspection, I discovered a very small difference in the file from what I had before. In my Monitor section, I now had the following:

[Graymi]

i'm speaking more from what little experiance i gleamed, but it seems when X starts it tries to validate things such as refresh against edid and other such fanciness. therefore putting in invalid refresh and sync values tends to lead to stuff not working, or simply invalidating modes.

i just installed a big ol' nasty 1080p telly () and use edid modes only on that - no refresh rates specified in xorg.conf - works every time

so try sticking with edid is my advice. unless its corrupt. like my laptop.