PlayStation 2 - Major progress, help me finish!

[Chewi]

Ah. Probably not a good idea to use the word compiling in that sense when we're talking about compilers.

Nothing's changed recently beyond what I've done. Even if I manage to get gcc 4 working, I highly doubt they'd merge my work because the machine is just so weird. You have to make awkward adjustments to what would otherwise be simpler code. This is even more true for binutils. I don't mind maintaining my own patches for the foreseeable future though.

[Maximus32]

Hi Chewi,

I love what you are doing. I also attempted to strip the ps2 patches from the mvl a while back. I came a long way but not as far as you have, and finally quit becouse of a lack of time, an interest. I am very happy to see an active attempt, and such an amount of interest for linux on the ps2. I would love to help out!

I don't know much about gcc and binutils. I currently use binutils 2.9 and gcc 3.0.3 (from playstation-linux). I also know ps2dev uses binutils 2.14 and gcc 3.2.2 (available from there subversion server). You are right about merging gcc/binutils to the main tree, they don't want that, attempts have been made already and failed.

Where I can help out is:
- linux kernel
- development environment
- versioning systems (CVS/Subversion)

[Chewi]

Ah someone who knows what they're talking about! A rare find. I'm hoping that finishing support for the second pipeline to gcc and fixing the short loop length problem in binutils is all I have left to do to get the new toolchain working. Support for the 128-bit instructions can probably come later. This means that it may not be too long before I start looking at kernel 2.6. If you can help with that, that'd be fantastic. I'm not certain yet but it looks like we may incorporate ps2sdk so if you want to make yourself useful in the meantime, you could get familiar with that. I've built it using gcc 4 but that's all I've really done with it so far. Building the other libraries that compliment it is a painful mess of makefiles and environment variables but we shouldn't need those.

[Maximus32]

I am already familiar with ps2sdk

The EE part of ps2dev is sort of a mini operating system (threads, file io, etc, etc...) so that part is not usefull to us (besides from providing great examples of how to communicate to the ps2 hardware).

The IOP part however could be very usefull. It is sort of a RTE for ps2sdk. It uses a totally different interface than used by the RTE, but perhaps we could create linux drivers that interface with the ps2dev modules, eliminating the use of the RTE. That way we could use normal elf loaders to run linux on the ps2. Modifying the modules to provide linux usable interfaces would also be an option. I'll look into the possibilities. I have some linux driver experience so it's worth a try.

ps2sdk built without a problem under windows with the ps2sdk compilers. I'll look into building it under linux as soon as I can. I already have the EE compiler (3.0.3), but I'll also need to install the IOP compiler. I don't think that will be a problem though, since it's a standard mips3000 (ps1) cpu.

Does anyone know if it's possible to load linux with a normal elf loader already? Just seeing a few linux debug messages would be enough to start from. Ofcourse it crashes when trying to communicate to the RTE.

[Chewi]

Even better! I thought you might be but I wasn't sure.

I thought the EE part might hold code relating to the GS but I noticed there's also a library called gsKit? I also thought newlib provided a lot of that "mini OS" stuff.

The IOP part would indeed be very useful. Eliminating the RTE is the general idea. If you check the thread over at ps2dev.org, you'll see that alexp managed to start kernel 2.6 without the RTE so it is certainly possible.

Building ps2sdk itself wasn't too bad, it was the libraries such as gsKit, dreamgl and libjpg that were a pain. Part of the problem was that I was using a CHOST of mips64r5900el-scei-elf rather than their choice of ee. It was also down to changes over the years such as ps2sdk changing its name from ps2dev and files moving locations. I also had to make adjustments to the code because gcc 4 is stricter. It's not just the EE part I built using gcc 4 but the IOP part as well. Fortunately I was able to get IRX support working in binutils 2.16.1. At least I think it works anyway!

I have been using ps2link to test out my gcc patches. I'm not too sure but do you think it would be possible to run Linux through it, at least up to a point?

[Maximus32]

The GS part of ps2dev is really tiny. I've already tried using it a while back, it is for initialization only, there is no interface for drawing or 3d. For this you would need gsKit or dreamgl as you mentioned. OpenGL support would be really nice, DreamGL could be usefull for this but it's far from finished. gsKit is more mature but provides a different interface. If I'm correct linux only provides 2d accelerated drivers, no opengl support (/dev/fb?). For OpenGL I think xorg drivers are needed. Do you know how opengl is implemented under linux?

newlib provides standard c library functionality, like glibc. In ps2dev newlib is ported to communicate to ps2sdk. For us this is totally useless since we would want it to communicate to the linux kernel. Besides, we already have a working c library.

All the separate libraries are a mess indeed, I had the very same problems with the build scrips. Is it possible to commit improvements to ps2dev to their repository?

About loading linux with ps2link... If I'm correct both the ps2dev executables and the linux kernel are ELF binaries, so in theory it should work. Probably some adjustments have to be made to the linux makefiles/linkscript, or to ps2link to get them compatible. I'll try when I get the time.

[Chewi]

I do have OpenGL support in mind but obviously that's a way off yet. The first step would be to get a basic framebuffer driver going. At best, I think these drivers can provide only a basic level of 2D acceleration. For better 2D acceleration, there is DirectFB, which I am still very interested in because it covers more bases than, say, SDL. SDL can use DirectFB natively, X can use it through XDirectFB and GTK+ can use it (without X!) through DFBGTK+, plus some applications such as mplayer can use DirectFB directly. As for 3D, I had DirectFBGL in mind, but it uses a stagnant embedded branch of Mesa that only works with Matrox hardware. I checked Mesa's homepage recently to see that a new project had appeared called glFBDev. Perhaps this is the way we should go instead but most 3D applications require some degree of X support as well. DirectFBGL was able to work in conjunction with XDirectFB but I'm not sure how glFBDev would tie in. Perhaps DirectFBGL could be rewritten to use glFBDev instead of Mesa embedded. I'm trying to rule out the Xorg server and even X entirely where possible because we're very short on RAM. This is all a bit hazy in my head at the moment but I'm looking forward to tackling it. Up till now, the most I've done with OpenGL is write a simple 3D maze program in C.

I don't have commit access to the ps2dev repository. To be honest, I'm not too interested in bare metal development, I've only been checking it out to test the new toolchain and to see if we can use some of the code in Linux. If the new toolchain turns out well, I'll offer the necessary patches and discuss cleaning up the scripts while we're at it.

For the record, ps2sdk is licensed under the Academic Free License v2, which is incompatible with GPL-2. This means we are allowed to use code from ps2sdk in Linux but we're not allowed to distribute any resulting binaries. Only the source code or patches can be distributed.

[Maximus32]

Do you know ps2gl? Full OpenGL support for ps2 , it is supposed to be on the official ps2linux distribution:
http://ps2gl.playstation2-linux.com/
Not using X sounds very good to me. It's BIG and slow, especially on a machine with only 32MiB. There is also window managers who don't need X to run, and run directly on the frame buffer, like Enlightenment (also used in PS3 yellow dog linux).

I tried running a linux kernel with ps2link.
The bad news: The original 2.2.1 kernel does nothing, I keep seeing the ps2link screen and nothing happens. I also tried the 2.4.17-mvl, something did happen, the screen turned black.
The good news: The files are recognized, I can see the files loading at address 80010000, and entry points at 800103a0 & 8001076c, which makes sense to me.

So the only question is: Why doesn't it show something on screen? Perhaps because of missing parameters, but most likely because of the missing RTE.

[Chewi]

Ah I wasn't aware of that. I've been to the ps2stuff site before but that was ages ago and 3D wasn't on my mind at the time. This kernel patch it needs sounds suspiciously like archaic 2.2 stuff that probably isn't needed for 2.6 but I could be wrong. I think existing software expects to be built against Mesa so perhaps the library itself isn't that useful but the code could be just what we need.

I wouldn't expect the old kernels to run through ps2link because they do rely on the RTE, which provides the necessary IOP modules. If you build alexp's kernel exactly as he says, maybe you'll get the same result he did.

[Chewi]

I'm proud to announce that gcc 4 is... WORKING!

http://www.aura-online.co.uk/~chewi/ps2-gcc4.png (that's the PS2 being shown through mplayer, not mplayer running on the PS2)

It's not perfect yet but damn, I'm pleased.

[Maximus32]

That's great news!!! Having a gcc 4 compiler for the ps2 is really breakthrough (now we'll be able to compile all of the new goodies ). I'd love to start testing/working with it when you think it's finished.

[Maximus32]

I am having some problems getting the latest 2.4.17-ps2 kernel. I use the following:

http://www.linux-mips.org/pub/linux/mips/kernel/v2.4/linux-2.4.17.tar.gz
http://gps2.aura-online.co.uk/distfiles/10-linux-2.4.17-ps2.patch.bz2

Then when I try to patch using:
patch -p1 < ../10-linux-2.4.17-ps2.patch

I get a lot of "Hunk #xxx FAILED at xxx" and then it fails. Am I using the wrong patch or kernel version?

[Chewi]

Something must be wrong somewhere because that is the right patch for that kernel. It changes the vanilla MIPS kernel into the MontaVista PS2 kernel. It's not the minimalistic PS2 patch I created, if that's what you wanted. I didn't commit that one because I never fixed the USB support. Doing the following works just fine for me.

[Maximus32]

That is indeed what I wanted, the minimalistic patch. I don't mind usb is not working. Since I'm trying to get linux to boot from a regular elf loader. I'm already happy if it shows something on the screen . I don't want to drag all the other montavista stuff along with debugging the kernel and getting it to work from a loader without the RTE. What's the latest patch I can use for that?

Turns out someone is already pretty far in booting linux without the RTE. He goes by the name of Mega Man on ps2dev and has written a kernel loader which does exactly what we would need:
http://forums.ps2dev.org/viewtopic.php?p=57330#57330
http://freenet-homepage.de/ps2dev/kernelloader0.2.tgz

From README:

[Chewi]

This is what you'll need then. Be aware that most of it was clueless guesswork so don't assume that this is a problem-free kernel. Apart from the USB, I didn't find anything wrong with it though.

http://www.aura-online.co.uk/~chewi/ps2-kernel-7.patch

Thanks for reminding me about Mega Man's loader. I'd seen it aaages ago and had totally forgotten about it since. I'm confused though. alexp has shown that he can start to boot 2.6 without any kind of loader at all. Are you only interested in the old RTE-style 2.4 kernels for now? I think the only reason we might need some kind of loader for 2.6 is to start the kernel with some parameters and for initrd support. Parameters aren't essential for the moment and initrds are really only needed for specific reasons. We wouldn't want to waste any of the PS2's RAM.

[Maximus32]

If you read the forum at ps2dev carefully you will see that alexp does not have the kernel running in kernel mode (kseg0). I am not focusing on 2.6 kernel for the ps2, I'll leave that up to you and alexp . Instead I will be focusing on replacing the RTE. For this purpose IRX modules will have to be created, and linux drivers to communicate with them.

Perhaps ps2dev modules can be used/modified for this purpose. The TGE source already provides usefull, for it includes an SBIOS (ee), also included in the kernelloader. The kernelloader also automatically loads the SBIOS.

The IRX modules will (ofcourse) be compatible with linux 2.6. The drivers will need to be ported, but I don't think that will be much work.

PS: You're right about me not using gentoo , I did use it a couple of times, so I know a little about it. Currently I am using ubuntu & windows. My ps2 development environment runs in vmware (also ubuntu). This way I can develop in both windows and linux, and reinstalling my pc doesn't require me to reinstall my development environment.

[Chewi]

Aah okay, I thought he fixed that issue in the end but I guess he didn't. I was planning to have the necessary IRX modules loaded by the kernel itself rather than a loader. I think Sony only did it using the loader so that we couldn't mess with it. If you can think of any reasons why this might not work, please let me know. We need to make this decision soon because I suspect it may make a big difference to the way the drivers are written.

[Maximus32]

It is possible, but not for boot devices (hdd/mc/dvd/usb). For these devices at least some IRX modules need to be loaded to load the IRX modules . Using initrd would be a possibility though. But since there is no need for an initrd otherwise (boot devices are static and should be compiled in the kernel), I would prefer not to use initrd at all.

Also, the loader already needs to load at least some IRX modules for itself to work, so why not let it load the modules needed by the kernel?

I would like to see some changes to the loader though. Currently what the loader loads is hard coded. I would like to have a config file for this purpose. So the loader will load whatever the config file specifies. Also, since loading the kernel takes some time (at least from mc/network it does), it would be nice for a user to see some progress indication, just like the sony RTE shows the progress of loading the kernel. This is no top priority however.

Besides from the boot devices, it would be nice for the kernel to also be able to load IRX files. So it will be possible from a running ps2linux environment to load/test IRX modules. For this I think a kernel module would be nice (irxloader.ko?). This module will need an interface for users to load modules (irxinsmod test.irx / irxmodprobe test ?), and for drivers to load modules. Drivers who need irx modules will then need this irxloader to load their irx modules, boot device drivers will either not be able to load irx modules or, the irxloader module will have to detect whether the module is already loaded by the kernel loader or not (I would prefer the last).

To Sum it up:

- modify loader to support a config file. (ask Mega Man at ps2dev)
- modify loader to show progress indication. (low priority)
- create kernel module that loads IRX files. (irxloader.ko?)
- create application users can use to load IRX files. (irxmodprobe, irxinsmod?)

Let me know what you think about all this.

EDIT: By doing it like this we will be able to choose if we let the loader load the irx files, or if we let the kernel/drivers load the irx files, using initr.

[Chewi]

I see your point. And believe it or not, I had almost exactly the same idea for the IRX modules? lsirx and irxprobe would be cool. I think that would only really be useful for development beyond Linux though. Like you said, we might as well let the loader load all the required modules.

I'm having to take a step back for the minute because further testing has revealed that the toolchain still isn't perfect yet. I tried Mega Man's Ballion game and it built (with some help) but left ps2link hanging. I then tried madplay (since Ballion uses libmad) and that failed with a "TLB load/inst fetch" exception. I built ps2sdk with the old toolchain as a comparison. The assembly for the function that fails is identical in both versions so I'm going to have to dig deeper. Additionally, I've realised that binutils isn't producing valid IRXs.

Old toolchain...

[Maximus32]

What version of gcc/binutils are you using to compile the IRX modules? I thought the IOP uses standard mips3000 compiler, nothing special like the EE/mips5900?

[Chewi]

Same as I'm using for the EE, gcc 4.2.0 and binutils 2.16.1. I don't have to, I suppose, but I'd like to be free of the old stuff. Extremely little is added to gcc. A bit more is added to binutils for IRX support. I did have some trouble linking IRXs initially but I fixed that.

[Maximus32]

I'm currently trying to get linux to boot using kernelloader. Since kernelloader includes an example kernel I can see that the loader works. But I still can see no life from linux. I've gone back to using version 0.1 of the loader since version 0.2 doesn't even show me the test kernel. Perhaps this is a problem with where things get loaded. I used both the high and the low version of ps2link. Also I like that version 0.1 of the kernelloader has graphics support. This is somehow dropped in version 0.2.
I got all weekend to get this thing to boot , I'll keep you posted.

[Chewi]

Heh how's your progress?

This IRX problem is proving tricky but I've managed to work out that the problem is gcc because using the new binutils and the old gcc works fine. That's really strange though because very few changes need to be made to gcc.

[Maximus32]

Version 0.2 of the somehow won't work. I've tried lots of things but not even the test kernel included will boot. So back to 0.1 where the test kernel boots I tried to get linux to boot. Since the kernel loader is located at 3MiB and linux gets loaded at 64KiB I thought the loader might overwrite itself with a big kernel as linux. So I changed the linkfile to let the loader start at 5MiB. Then not even the loader would boot. Turns out the loader uses a static heap at 4MiB, so I changed the heap to using the address specified in the linkfile. This time it booted, but with the same results as before .

Then I got a little sidetracked and included a nice looking progress bar, and a config file where the file names can be specified. All works and looks cute, but linux still won't boot.

So today I'm going to focus on the linux kernel. I think I'm going to embed a tiny ps2 example program into the kernel and let that boot first, just to see if the kernel gets called at all, and to see if all compiler settings are working.

What is the difference between the standard compiler and the compiler for the iop? Only thing I know of that differs is the file format, IRX. But I thought a separate tool turns elf files into irx files?

[Maximus32]

WOW! I got linux running up to the point where it can't continue without a root fs. Giving me the message "<0>Kernel panic: I have no root and I want t...".
http://www.bricks-os.org/shared/BOOT01.JPG
Next I need to make an initrd so I can see if it really continues to boot!

PS: How do I include a picture so it shows in the forum?