mmal on rpi64

[axl]

So I asked point blank Sakaki & anyone else (here - https://forums.gentoo.org/viewtopic-p-8370868.html#8370868) if mmal in 64 is really that important.

I was expecting either sakaki or neddy to come and tell me I'm wrong.

Listen. aarch64 is great. works a little bit faster. i'm not sure it will fix all problems 6j and 7a had or maybe 8a will have, but it did show some improvements in terms of performance vs armv32. But that being said, in 64 bits the mmal components doesn't work.

I just want someone to say it out loud. It's not my incompetence, you can't make any multimedia thing work in 64 bits.

I wanted to repeat this question in other threads, but I thought it was impolite to stain a thread that is not mine.

So I made my own thread. I doubt sakaki will visit it, but anyone is invited to contribute.

My premise is that most of the video/3d things work only in 32 bits. 64 bits does work, but works best for HEAD-LESS setup.

In some weird sense, everybody is excited for the aarch64, but in reality, it has no practical application. I can't think of one.



Let me me something very clear. I'm ALL for "because i can". and aarch64 is exactly that. because i can.

In terms of raspberry pi gpu, you simply can't in 64 bits and someone should just say it out loud.


That's just my opinion... what's yours? I'd love to correct what I done wrong...

[NeddySeagoon]

axl,

mmal is supposed to work on the Pi in 64 bit mode. I've not knowingly tested it myself.

The GPU on the Pi is 32 bit only, so any 64 bit code has to speak 32 bit to the GPU and understand 32 bit responses.
Its a hardware feature. The VC4 is 32 bit.

mmal is on my list of things to poke at but its very low down.
I unexpected got a 4k image out of my Pi4. My Pi is connected via HDMI on the display and the display is not supposed to support 4k on the HDMI ports.
The desktop was very slow in 4k and it shouldn't have been.

With mmal, the Pi should be able to play 4k videos.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[Gavinmc42]

You have to read a bunch of 64bit Raspbian posts but buried in there is the answer.
It should be possible but it was not on the RPF list of important things at the time.

As Raspbian Buster is moving towards 64bit things like camera etc are starting to work.
With the PI4 being such a great device more smart people will figure things out.
And now it is probably on RPF's list of things to do.
Wish we had some manuals.
Time will tell.

I do miss the OpenVG
But I get Shaders as a consolation prize
_________________
Don't get Pi's if you are scared of learning.

[axl]

@neddy 2 different things. vc4 is one thing, mmal is another. vc4 is the gpu itself used for rendering, and it's handled through kernel & mesa. Hmmm, I didn't know that worked in 32 bits as well, but hey, that was no problem getting it to work, now or in previous attempts. It's true I wasn't trying to get 4k to work, as I have no interest in that whatsoever, however omxplayer & kodi do interest me. And since mmal doesn't work in 64 bits, raspberry-pi flag is disabled in ffmpeg and since that is disabled ... no hardware decoding.

@Gavinmc42 in order for the camera to work, they changed the way it is used. Now instead of it being managed by raspberry pi userland, it is managed by v4l in kernel. that means instead of using raspistill and raspivid, you have to use /dev/video0.

My point is that before embarking into this long long ride, people should at least know what to expect at the end of it.

PS and based on the discussions I've read, the mmal port to 64 bits wont happen easily. Or soon. And it kinda is important.

[Gavinmc42]

If I have used mmal I was probably not aware of it.

I use Baremetal, it is based on Ultibo so I use the fpc based libraries and don't have much idea of how they work except it's OpenGLES/VG/EGL/Dispmanx.
Yep was aware of the cameras via V4L in the Pi4 but have not tested that.
dev/video0 is mainstream so will probably just work.
Emerge Cheese tonight, let you know tmr?
Got cameras here on the desk somewhere

SDL 2.0 seems to work, can it substitute?

The entire VC4 blob is not going to be ported to Pi4.
In fact RPF seem to be avoiding doing that.

What do people use mmal for?
_________________
Don't get Pi's if you are scared of learning.

[axl]

http://www.jvcref.com/files/PI/documentation/html/


MMAL (Multi-Media Abstraction Layer) is a framework which is used to provide a host-side, simple and relatively low-level interface to multimedia components running on VideoCore.

The short answer is that mmal is responsible for hardware video decoding/encoding. Keyword here is "hardware".

[axl]

A little correction. I was wrong about the name of the useflag. it's not raspberry-pi, it's actually mmal. Funny, I thought they changed it, but it seems I was remembering wrong or they changed it back.

[NeddySeagoon]

axl,

Enable the mmal flag and tell us what happens. We all want to know and someone has to be first.
This time, it could be you. :)
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[axl]

?!

let me rephrase that. it is blocked, not disabled. mmal is blocked on aarch64 because it expects you to have raspberrypi-userland and that doesn't work as well. Well, some parts of that work, but mmal is not one them.

it's not just about enabling a flag.

[NeddySeagoon]

axl,

I've just looked at the ffmpeg ebuild. You are correct
I recall someone posting here about mmal on 64 bit working.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[axl]

That would be awesum. Maybe I missed something. Reading through the issues on github for raspberrypi-userland, i doubt that completely though.

It's like a known issue, for years even (since the first 64bit pi dropped) that mmal is going to be a pain in the behind. Nobody is working on it. No efforts whatsoever have been done for this. It's just dead in the water. In fact if you read through the issues on raspberry pi github you can get a distinct idea that it's not ever supposed to work in 64bits. It's a given that it does, but not because it's supposed to.

Gavinmc42 mentioned the new interface for the pi camera, which in fact is the old interface for video. v4l. I don't know exactly if /dev/video0 is able to spit out h264 coded video. I doubt it can. which raspivid did. as far as I know, /dev/video0 provides uncompressed video. Which if it was a pc, would be usable. But on a PI... be it pi4... it's still not enough to process that data. /dev/video0 at those resolutions without mmal is just pointless. That was the whole thing of the rpi camera. it was realtime encoded in h264. and in case none of you know how, it was through mmal.

mmal is central to _hardware_ video encoding/decoding. if you are going the 64 bits way, you can say goodbye to hardware encoding/decoding. and without that... raspberry pi is just a very slow 64bits atom computer. it's whole hook was the hardware video encoding/decoding ability. I'm not saying you can't use it for other things, but mmal is brilliant. Shame it doesn't work in 64 bits.

Anyway, people trying 64 bits should know: kodi, omxplayer, pi camera and other alternatives of the sort are severely handicapped.

[axl]

When I say hardware video encoding/decoding I am talking about alternatives to cpu. We all know ffmpeg (which is the central piece of any video software) can process things through the cpu. even multi core. Amazing how many advances have been made.

But there's also alternatives to NOT use the cpu. Like vdpau for nvidia rendering. Or nvenc for nvidia encoding. I don't know amd, coz I don't have any, but I do know that Intel has it's own thing. Was it vaapi? Not sure, I don't have any intel either, I just picked it up while messing with mesa I think.

A while back, like 2 decades ago, I received a bundled creative cdrom and a creative dxr2 card. That card (you can still see the dxr2 flag in ffmeg) was able to video decode mpeg streams. You would put your vga out from video card into dxr2 input, and dxr2 output to vga output to monitor and basically it was an overlay. So old. I still have that card somewhere. Should be put in a museum. It was BAAAAD at least with the software of the time.

But on the other hand did you see how videos look on the pi? The colors, the not tearing, the framerate... everything is crystal clear. I can connect my nvidia 960 through hdmi to the same tv where I have one of my pi's and there's just no comparison. I use mostly mplayer & smplayer and I tried xv, gl_nosw, sdl, vdpau... you can't make a pc with linux and nvidia show a video as beautiful as raspberry does it. You just can't. That's hardware video decoding at work. the mmal component does that.

[NeddySeagoon]

axl,

[axl]

[axl]

and I'm sorry to myself for derailing my own thread, but talking about old hardware accelerated platforms, I had the original creative video blaster.

During that time, windows 3.1 was a novelty. Most things worked in dos.

And this thing would be able to take video in analogue signal and put it on screen. You couldn't record videos as far as I remember, but you could make pics. 320x240 pics (TGA format). I think. It was something ridiculously small.

But it was so cool. I remember we used to connect either the vcr to watch a movie on the computer, or the camera to monkey around. I might have been 12 at the time.

The computer holding it was a 486se. 8 megs of ram. 500 megs of hdd. had cdrom, had sound blaster. had tape. it was a beast at the time.

I can't believe how far we've come in my lifetime.

[NeddySeagoon]

axl,

I was just on linux :)
It uses the em8300 driver, which still appears to be in the kernel.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[axl]

[axl]

I wonder... if you have a 64 bits kernel, and a 32 bits userland, could you use mmal? or if you boot 64bits, and chroot into a 32bits distro. could you mmal then? (if you mounted dev proc sys). if the answer is true...

maybe aarch64 could benefit from a multilib setup. one where you compile most things 64 bits, but not ffmpeg, it's dependencies, and anything that follows ffmpeg.

Should be fairly easy to test. just add 64 bits kernel to raspbian. it would boot and work, but is kodi still working? Maybe I'll try that tomorrow.

[axl]

Maybe I should try armv7a. Haven't done that in a while. On the pi4 should be a breeze.

Uhm, I'm especially curious if chromium (youtube) uses mmal to render. I think it does. Uses ffmpeg. There's no reason it shouldn't. Did you see how gpu heavy chrome/chromium is? they even have a chrome://gpu tab.

It's been at least one year since I wanted see what would that look on armv7a. I seen it in aarch64 and I'm not impressed.

[Gavinmc42]

axl , you want mmal and VC4 hardware for h.264 encoding/decoding?
Then I defiantly use the h.264 encode, that's used for motion detection on my Pikrellcam cameras.
Guess that will limit Pikrellcam to VC4 Pi's, for the moment.

Years ago the Pi guys told me h.265 would be faster decoding on the Arm cores than the VC4 GPU.
I think that is what they have gone for in the PI4.
Arm decoding of h.264 and 265.
I was under the impression ffmpeg had that already.

RPF guys say Browsers are a bit laggy for >1920x1080 videos, so I went looking for a standalone YT viewer.
Freetube YouTube viewer works, that might be easier to dig into than Chromium/Firefox.
I think you are right about them using ffmpeg, but they do other stuff as well.

With the new mesa releases we get the OpenGLES 3.1 driver which has compute shaders.
OpenCL is supposed to work too.
Someone with some clues might be able to accelerate some stuff using those on the VC6.

Wish we had the manuals.
_________________
Don't get Pi's if you are scared of learning.

[axl]

You know, this is not exactly clear. They did say the new rpi4 supports x265 as it is 4k. and as far as i know x265 was designed especially to push x264 into 4k. but people dont always use x265 for 4k streams.

What is not clear to me atm is if the ability to decode x265 is also backported to older pi's through the magic of mmal.

I guess the answer is in ffmpeg. which is central to any video rendering app. regardless of what app it is. even chrome.

Or things I could test if I could be bothered. Other than PI 1, I still have one of each. But they're in their place. It's not me that can't be bothered... I can't bother the girls. I call them by girls names. I'm weird. But I have a 4 and a 3b which are not "in their place". I'll experiment on those.

Honestly, this discussion helped. put things into perspective. it's hard to run tests without knowing what tests you want to run. which is not to say more input is not welcomed.

[Gavinmc42]

Looks like we don't have OpenCL either.

While the hardware might be capable of lots of stuff without the software it is not usable.
Tried to emerge CLinfo but it is masked.
Have not got to the masking part of the Gentoo manual yet.

OpenMAX is omx. = omxplayer, Is that fully open now?

omx101 for dummies?
https://kwasi-ich.de/blog/2017/11/26/omx/

https://www.yodeck.com/news/the-raspberry-pi-4-ultimate-review/

A few posts about OpenMax on Pi4, gooblygook to me.
The more I learn the less I know.
Starting to think I can never learn everything about Pi's.
Oh for the old days of <100 page manuals and one double sided laminated plastic sheet for coding.
_________________
Don't get Pi's if you are scared of learning.

[Gavinmc42]

I followed a white rabbit and found this.
https://www.raspberrypi.org/forums/viewtopic.php?f=38&t=246837

I am so glad I don't do complex video stuff or have large res screens.
Getting a straight answer is hard.

Maybe the Kodi guys have a clue?
Wish there were some manuals, so I could RTFMs.
_________________
Don't get Pi's if you are scared of learning.

[Gavinmc42]

Looks for the reasons mmal on Sakaki's wiki

[Sakaki]

As far as I know, the current 64-bit bcm2711_defconfig kernel supports MMAL, when called from a 32-bit userland (see e.g. here), but I don't believe the required userland libraries exist in a 64-bit form, yet.

The (optional) camera module, and (built-in) hardware video codecs can however be accessed via the V4L2{,M2M} framework from a 64-bit userland (this needs a patched ffmpeg to work, such as the version shipped on the current gentoo-on-rpi-64bit image). See e.g. my notes here.

When capturing from the camera, you can specify the use of e.g. H264: