| View previous topic :: View next topic |
| Author |
Message |
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
 Posted: Fri Aug 18, 2023 11:50 pm Post subject: Gentoo_stage_for_CUDA |
 |
|
| hello forum. if everyone says that it is impossible to run the Linux kernel on video cards, well, or just difficult, then why not just make Gentoo stage specifically for assembly with the CUDA compiler. I don't like approach or solution such as CUDA, I think that the kernel could work on a video card much more natively, but what we have as they say. is it possible to create such a stage for CUDA? there is for MIPS, there is for arm, but not for CUDA. why? |
|
| Back to top |
|
 |
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Sat Aug 19, 2023 9:51 pm Post subject: Re: Gentoo_stage_for_CUDA |
|
|
Gentoo has a lot of problems, but the biggest problem is the time-consuming compilation. GPU can help a lot here. but everyone will have a choice of what to put and what to assemble either on the CPU or on the GPU. you just choose stage and no one infringes on you in anything. everyone will have a choice. he's not really here right now. for some reason, the entire Linux community easily agrees to the introduction of any useless and meaningless rubbish, but to refuse what could really be useful. this could make Gentoo a really flexible and fast powerful system. why not take a step towards it? I think Gentoo is ideal for running on GPU or CUDA cores. this is harmony for this distribution. there is no other easy improvement and it will not be for many years. this will expand the possibilities. only fools can get rid of it. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22602
|
| Posted: Sun Aug 20, 2023 3:36 am Post subject: |
|
|
| What steps need to be completed for this to work? If I wanted to start from an existing Gentoo install, how would I do what you are proposing? |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 1:28 am Post subject: |
|
|
| Hu wrote: | | What steps need to be completed for this to work? If I wanted to start from an existing Gentoo install, how would I do what you are proposing? |
there's nothing you can do. this should be done by those who have already developed for Gentoo. they know all their mistakes, this is their handwriting of development, it will be difficult for you to understand their scribbles. imagine that I'm baking a pie. no one knows what I put there except myself. so those gentlemen who took a shit in this wonderful distribution know their shit best of all, so no one will do anything better than themselves. if they condescend to us mere mortals and create STAGE3 FOR CUDA and write in manul all the steps that need to be done to assemble it with the NVCC compiler, this is the easiest way to get Gentoo on the GPU. she doesn't have enough CPU power. I think that a pair of 3060TI could assemble Gentoo very quickly. very very fast. we need to move in this direction. other paths for this distribution are dead ends. |
|
| Back to top |
|
|
ian.au
l33t
Joined: 07 Apr 2011
Posts: 604
Location: Australia
|
| Posted: Mon Aug 21, 2023 1:53 am Post subject: |
|
|
| Gentoopc wrote: | | Hu wrote: | | What steps need to be completed for this to work? If I wanted to start from an existing Gentoo install, how would I do what you are proposing? |
there's nothing you can do. this should be done by those who have already developed for Gentoo. they know all their mistakes, this is their handwriting of development, it will be difficult for you to understand their scribbles. imagine that I'm baking a pie. no one knows what I put there except myself. so those gentlemen who took a shit in this wonderful distribution know their shit best of all, so no one will do anything better than themselves. if they condescend to us mere mortals and create STAGE3 FOR CUDA and write in manul all the steps that need to be done to assemble it with the NVCC compiler, this is the easiest way to get Gentoo on the GPU. she doesn't have enough CPU power. I think that a pair of 3060TI could assemble Gentoo very quickly. very very fast. we need to move in this direction. other paths for this distribution are dead ends. |
Why so obnoxious? CUDA is NVIDIA right - proprietary and closed-source? So get them to document a way forward for you, good luck with that. Unloading on people who assist with resourcing Gentoo because of your frustrations won't help. |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 2:38 am Post subject: |
|
|
you accepted Intel microcode into the distribution kit as well as other firmware, this did not frighten you. Well, Nvidia CUDA is evil. but you can easily make a Stage_GPU that will just run on the GPU. not only on Nvidia GPU, but also AMD and possibly on pure GPU from China. Do software engineers really not have the mind to make the kernel and Gentoo work on the GPU as a whole? I have already said that you can simply programmatically make the GPU look like the CPU so that every software thinks so. |
|
| Back to top |
|
|
wjb
l33t
Joined: 10 Jul 2005
Posts: 629
Location: Fife, Scotland
|
| Posted: Mon Aug 21, 2023 8:56 am Post subject: |
|
|
| Quote: | | I have already said that you can simply programmatically make the GPU look like the CPU so that every software thinks so. |
Unfortunately, that's not how it works. Why not try some of the various CUDA tutorials to get a better feel for the possibilities? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54577
Location: 56N 3W
|
| Posted: Mon Aug 21, 2023 9:09 am Post subject: |
|
|
Gentoopc,
| Gentoopc wrote: |
... but you can easily make a Stage_GPU that will just run on the GPU. ... |
Everything is easy if you are not the one doing it.
As it's 'easy', please contribute the stage you are advocating.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22602
|
| Posted: Mon Aug 21, 2023 12:35 pm Post subject: |
|
|
| Gentoopc wrote: | | there's nothing you can do. this should be done by those who have already developed for Gentoo. |
Then describe what someone who has already developed for Gentoo should do. The point of my post was to solicit instructions that someone other than you can follow, so that we can begin a discussion of how difficult or easy this is. | Gentoopc wrote: | | they know all their mistakes, this is their handwriting of development, it will be difficult for you to understand their scribbles. |
It may be difficult, but not impossible. Also, do not assume that because the code was done wrong, the author knows what was done wrong and just needs your advocacy to do better. Perhaps the developer is unaware of the error, or lacks the time to do it better. Describing how the author should do it may spark their interest, or show them a better way. | Gentoopc wrote: | | so those gentlemen who took a * in this wonderful distribution know their * best of all |
Mind your language, please. There is no need for that here. | Gentoopc wrote: | | so no one will do anything better than themselves. |
I disagree. The people who wrote the code may know it best now, but there is no guarantee that someone more experienced and more clever cannot come along and show a better way. Perhaps you would show us the better way, and prove yourself more clever than the original authors? | Gentoopc wrote: | | if they condescend to us mere mortals and create STAGE3 FOR CUDA and write in manul all the steps that need to be done to assemble it with the NVCC compiler, this is the easiest way to get Gentoo on the GPU. |
Yes, having a documented path to do this would be nice. Will you write that manual for us, so that we do not need to wait for the Gentoo experts to do this? I suspect that if there were any experts who wanted to undertake this project, we would know about it, so either nobody knows how, or nobody wants to work on it. Either way, that says we need a new contributor to get this working. | Gentoopc wrote: | | she doesn't have enough CPU power. I think that a pair of 3060TI could assemble Gentoo very quickly. very very fast. we need to move in this direction. other paths for this distribution are dead ends. |
I am doubtful of this conclusion, but I would be willing to be proven wrong. Do you have an example benchmark of running a modern C++ compiler on a GPU and getting good results? I hardly think relying on a CPU is a dead end. It may not be the most efficient path possible, but it is very viable. I am not aware of a generally available and currently usable path that is more efficient. Are you? If so, please tell us where we can read how to use this path. |
|
| Back to top |
|
|
grknight
Retired Dev
Joined: 20 Feb 2015
Posts: 1901
|
| Posted: Mon Aug 21, 2023 1:30 pm Post subject: |
|
|
This is a bit of a dream at this point.
CPUs and GPUs are fundamentally different in their architecture. CPUs are very good at a serial set of complex instructions and good scheduling. GPUs are very good at lots of simple, parallel computations.
NVCC is mentioned but nVidia's documentation itself says that NVCC passes all non-GPU based instructions from a .cu file to gcc or g++. All pure C++ and C files run through NVCC are directly dumped to gcc anyway.
A stage based on nvcc gains nothing at this time. One day a better compiler than gcc may appear or gcc may include GPU offloading for large data tasks. When that occurs, I'm sure more people will take advantage of it. |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 7:40 pm Post subject: |
|
|
| grknight wrote: | | When that occurs, I'm sure more people will take advantage of it. |
when that happens Gentoo will be gone. everything has time, and the main thing is to get there for sure! that's how everyone took their niches. Gentoo can turn from a distribution for nerds into a distribution for more adult and serious purposes. The way I told you. you don't lose anything. and how to do it, let the professionals think, it's their job they occupy this place. for example, I am a simple farmer, but I ask you to do my work for me, I just take and do what I have to do, because this is my job. so are you, just do your job. I can only say as a simple user from the position of a simple user what this distribution, which has been on the verge of death for many years, lacks. do you want the best for him? well, you know what to do. if you just take up space, and don't want difficulties, and therefore immediately kill all the offers, then I won't even waste time explaining what you should understand anyway, but you just don't want to, but you can understand, you only do what those who pay you will tell you. and corporations pay. and they don 't need a distribution that will be extremely cool .they need a bad buggy Gentoo with systemd. |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 7:51 pm Post subject: |
|
|
linus Torvalds once said that the core will never be able to work on M1 chips. but it works. this speaks volumes. guys, there is a goal, but how to achieve it is up to you. you need to make the kernel run on the GPU. I have already written about the clumsiness of the scheduler in other topics. and that only the huge parallelization that the GPU can give could save everything. also, the transition to GPU will make it possible to integrate artificial intelligence into the distribution. sooner or later it will happen. although it is already happening in the android OS. we need to create a foundation for this. if you deny it and do not accept it, then you are fools and there is no point in having a conversation anymore. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54577
Location: 56N 3W
|
| Posted: Mon Aug 21, 2023 7:53 pm Post subject: |
|
|
Gentoopc,
| Quote: | | ... what this distribution, which has been on the verge of death for many years, ... |
Since I joined, over 20 years ago :)
| Quote: | | .. just do your job. ... |
Nobody is paid by Gentoo to work on Gentoo. It's a hobby.
There are one or two devs that are paid by their employer (not Gentoo) to work on Gentoo as the employer directs.
If what you suggest was both easy and beneficial to the distro, one of the distros with lots of paid developers would have done it already as it would provide a competitive advantage. e.g. Red Hat.
| Quote: | | ... immediately kill all the offers ... |
I did not see any offer(s) only a suggestion that others do work for dubious benefit.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
wjb
l33t
Joined: 10 Jul 2005
Posts: 629
Location: Fife, Scotland
|
| Posted: Mon Aug 21, 2023 8:58 pm Post subject: |
|
|
M1 is a CPU, so no surprise there. The point is that GPUs are effectively co-processors, they're superb at what they do (vector maths) but branching ain't it.
You're making the mistake of thinking that if there's a C/C++ compiler producing code for a GPU it'll all just work. Not true. Take a look at at what GPUs are geared up for and most people (programmers anyway) would notice real fast they're aimed at doing maths, in parallel. And, sadly, that's not a lot of use for compiling software.
Take a look at the Gentoo packages with cuda use-flags or requiring cuda; they're all (without exception) maths-based data processing. Anything that can use a GPU is already using a GPU. |
|
| Back to top |
|
|
stefan11111
l33t
Joined: 29 Jan 2023
Posts: 934
Location: Romania
|
| Posted: Mon Aug 21, 2023 9:33 pm Post subject: |
|
|
| Gentoopc wrote: |
Gentoo can turn from a distribution for nerds into a distribution for more adult and serious purposes.
|
How is gentoo not suitable for "serious" purposes?
| Gentoopc wrote: |
which has been on the verge of death for many years
|
And when was that?
| Gentoopc wrote: |
and corporations pay. and they don 't need a distribution that will be extremely cool .they need a bad buggy Gentoo with systemd. |
From my understanding, systemd is embraced by corporations because it helps them circumvent the gpl.
_________________
My overlay: https://github.com/stefan11111/stefan_overlay
INSTALL_MASK="/etc/systemd /lib/systemd /usr/lib/systemd /usr/lib/modules-load.d *udev* /usr/lib/tmpfiles.d *tmpfiles* /var/lib/dbus /usr/bin/gdbus /lib/udev" |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 9:43 pm Post subject: |
|
|
[quote="stefan11111"] | Gentoopc wrote: |
From my understanding, systemd is embraced by corporations because it helps them circumvent the gpl. |
why does gentoo use systed then? did it become a corporation? |
|
| Back to top |
|
|
stefan11111
l33t
Joined: 29 Jan 2023
Posts: 934
Location: Romania
|
| Posted: Mon Aug 21, 2023 9:47 pm Post subject: |
|
|
| Gentoopc wrote: | | stefan11111 wrote: |
From my understanding, systemd is embraced by corporations because it helps them circumvent the gpl. |
why does gentoo use systed then? did it become a corporation? |
Gentoo gives the user the option to use many init systems, including systemd.
I use openrc and avoid systemd like the plague, but someone else might like systemd(don't know why, but such people exist).
I don't have any of it's shims either:
| Code: | $ eix -I systemd
No matches found |
_________________
My overlay: https://github.com/stefan11111/stefan_overlay
INSTALL_MASK="/etc/systemd /lib/systemd /usr/lib/systemd /usr/lib/modules-load.d *udev* /usr/lib/tmpfiles.d *tmpfiles* /var/lib/dbus /usr/bin/gdbus /lib/udev"
Last edited by stefan11111 on Mon Aug 21, 2023 9:48 pm; edited 1 time in total |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 9:48 pm Post subject: |
|
|
| wjb wrote: | M1 is a CPU, so no surprise there.
|
M1 is a SOC
and the point is not that it is, but that the main boss of the kernel said that it could not be done.
Last edited by Gentoopc on Mon Aug 21, 2023 9:50 pm; edited 1 time in total |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22602
|
| Posted: Mon Aug 21, 2023 9:49 pm Post subject: |
|
|
Gentoo offers systemd as one of several choices, because some users choose to run it. They may do this because they prefer how it works. They may do it because they want to use Gentoo as a testing ground to learn systemd before being stuck with it on some other, less choice-oriented distribution. They may do this because they insist on running some software package which only works properly with systemd. Some people may have reasons I did not list here. Gentoo does not at this time require systemd, because there are still Gentoo maintainers who choose to support other init systems.
I note you did not address my earlier request for guidance on how a Gentoo developer should modify Gentoo to run on a CUDA GPU. I suggest that providing that information would help your case, as it seems no one else here (including me) believes it to be practical or a worthwhile use of time. Please prove us wrong. |
|
| Back to top |
|
|
stefan11111
l33t
Joined: 29 Jan 2023
Posts: 934
Location: Romania
|
| Posted: Mon Aug 21, 2023 9:55 pm Post subject: |
|
|
| Hu wrote: | | They may do this because they insist on running some software package which only works properly with systemd. |
In this case, they should ask here.
Someone, I think the funtoo guys, removed systemd from gnome. This shows that no matter how big, bloated and complex something is, with dedication one can rid it of systemd.
_________________
My overlay: https://github.com/stefan11111/stefan_overlay
INSTALL_MASK="/etc/systemd /lib/systemd /usr/lib/systemd /usr/lib/modules-load.d *udev* /usr/lib/tmpfiles.d *tmpfiles* /var/lib/dbus /usr/bin/gdbus /lib/udev" |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Mon Aug 21, 2023 9:59 pm Post subject: |
|
|
| wjb wrote: |
You're making the mistake of thinking that if there's a C/C++ compiler producing code for a GPU it'll all just work. Not true. Take a look at at what GPUs are geared up for and most people (programmers anyway) would notice real fast they're aimed at doing maths, in parallel. And, sadly, that's not a lot of use for compiling software.
Take a look at the Gentoo packages with cuda use-flags or requiring cuda; they're all (without exception) maths-based data processing. Anything that can use a GPU is already using a GPU. |
I already wrote to you above that it is necessary to create a foundation for the future of this distribution for the introduction of AI. if software engineers don't know how to make the GPU, thanks to the presence of a huge number of cores with an ingenious architecture, give a new round and increase performance, then there is nothing more to say. they cannot take a step towards the new, they follow the knurled. they are limited. such will not be able to deploy the distribution on the GPU to open up new horizons. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3408
|
| Posted: Mon Aug 21, 2023 10:28 pm Post subject: |
|
|
| Quote: | | if everyone says that it is impossible to run the Linux kernel on video cards, well, or just difficult, then why not just make Gentoo stage specifically for assembly with the CUDA compiler |
If what you mean by "assembly with cuda compiler" is "using gpu's power for building software", it's not going to happen with a MAYBE exception for someone's drunken challenge.
Why? There is no good reason to do it, except for proving it's technically possible (if it is: press X to doubt). GPUs are not even fast. They just can do basic math on registers 1000 times bigger* than CPU while being only 10 times slower. Exactly the opposite of what building software requires.
| Quote: | | such will not be able to deploy the distribution on the GPU to open up new horizons. |
Of course they won't be able to run a distribution on a gpu. Just like you won't be able to deliver my pizza by driving an intercontinental cargo ship.
It's a wrong tool for the job at hand.
*This is an oversimplification, but I think it reflects strengths and weaknesses coming from lockstep and memory access modes pretty well, and sufficiently for the matter at hand.
A cargo ship can carry a LOT of stuff from point A to B. Pizza delivery from A to B, C, D and E works better with an agile bike though, even if you carry only 1 box at a time. |
|
| Back to top |
|
|
Gentoopc
Guru
Joined: 25 Dec 2017
Posts: 364
|
| Posted: Tue Aug 22, 2023 2:01 am Post subject: |
|
|
| szatox wrote: | | Quote: | | if everyone says that it is impossible to run the Linux kernel on video cards, well, or just difficult, then why not just make Gentoo stage specifically for assembly with the CUDA compiler |
If what you mean by "assembly with cuda compiler" is "using gpu's power for building software", it's not going to happen with a MAYBE exception for someone's drunken challenge.
Why? There is no good reason to do it, except for proving it's technically possible (if it is: press X to doubt). GPUs are not even fast. They just can do basic math on registers 1000 times bigger* than CPU while being only 10 times slower. Exactly the opposite of what building software requires.
| Quote: | | such will not be able to deploy the distribution on the GPU to open up new horizons. |
Of course they won't be able to run a distribution on a gpu. Just like you won't be able to deliver my pizza by driving an intercontinental cargo ship.
It's a wrong tool for the job at hand.
*This is an oversimplification, but I think it reflects strengths and weaknesses coming from lockstep and memory access modes pretty well, and sufficiently for the matter at hand.
A cargo ship can carry a LOT of stuff from point A to B. Pizza delivery from A to B, C, D and E works better with an agile bike though, even if you carry only 1 box at a time. |
weak cores fast cores are all the words of a child. as you build the logic of the software on the GPU, so it will be. if you are smart enough to do well, then everything will fly. if there is no mind, then at least superfast cores will slow down and everything will fly out. it all depends on who will do it. but the glaring plus of the GPU is a huge parallelization opportunity. I'm already tired of telling everyone about it. read the messages at least sometimes. a huge number of cores will smooth out the problem with queues in the kernel, with the kernel policy of killing processes that will stand in the queue and exhaust their CPU time. this is not a threat on the GPU. and I repeat once again for the hundredth time that this is the foundation for AI. everything from Photoshop to games is already tied to AI. if you are not able to realize this , then please do not try to make meaningless what is already simply necessary . being able to work on a Gentoo GPU is just a must for survival. she knows how to work on all sorts of rubbish like mips, sparc, alpha, ppc and other nonsense that 2 people out of 1000 use. but you don't want to make stage_GPU. you're insane. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3408
|
| Posted: Tue Aug 22, 2023 9:37 am Post subject: |
|
|
| Quote: | | if you are smart enough to do well, then everything will fly. |
Fine,show us your code.
| Quote: | | but the glaring plus of the GPU is a huge parallelization opportunity |
The glaring plus of a cargo ship is a huge capacity, but the dude with a bike delivered a 100 pizzas already and I'm still waiting for mine.
| Quote: | | a huge number of cores will smooth out the problem with queues in the kernel, |
We're telling you this is not how GPUs work.
| Quote: | | and I repeat once again for the hundredth time that this is the foundation for AI. everything from Photoshop to games is already tied to AI |
Your foundation is wrong and AI software already runs on GPU without it.
| Quote: | | being able to work on a Gentoo GPU is just a must for survival |
Yes, yes... and we have a decade to do something about climate change. Just like we had 10 years ago, 20 years ago, and 70 years ago.
I don't know who told you that, but you're being a drama lama right now.
| Quote: | | but you don't want to make stage_GPU. you're insane. |
Back to square 1: show us your code |
|
| Back to top |
|
|
stefan11111
l33t
Joined: 29 Jan 2023
Posts: 934
Location: Romania
|
| Posted: Tue Aug 22, 2023 10:41 am Post subject: |
|
|
| Gentoopc wrote: | | she knows how to work on all sorts of rubbish like mips, sparc, alpha, ppc and other nonsense that 2 people out of 1000 use. |
Gentoo is known, among other things, for supporting so many arches.
It's one of gentoo's selling points.
If arm and riscv is among that "other nonsense", then you're simply wrong.
Should we become like arch and only support amd64?
_________________
My overlay: https://github.com/stefan11111/stefan_overlay
INSTALL_MASK="/etc/systemd /lib/systemd /usr/lib/systemd /usr/lib/modules-load.d *udev* /usr/lib/tmpfiles.d *tmpfiles* /var/lib/dbus /usr/bin/gdbus /lib/udev" |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Kernel & Hardware
