View previous topic :: View next topic |
Author |
Message |
The_Document Apprentice
Joined: 03 Feb 2018 Posts: 275
|
Posted: Wed Mar 21, 2018 1:05 am Post subject: How do binary distros do their kernel? |
|
|
I have been wondering about how binary distros like ubuntu and fedora do their kernel. Do they really enable ALL options? How do only applicable modules load? Does the software probe hardware EVERYTIME it is booted to find appropriate modules to load?(Seems like it already does even with monolithic kernels) How would I go about making a kernel in the same fashion as say ubuntu does but with gentoo? Is there a guide to this kernel fashion?
Couldn't find any answers on my own so I made this topic. I am accustomed to selecting applicable kernel options as built in, I don't use modules except for audio as I have some issues with it being built in. |
|
Back to top |
|
|
audiodef Watchman
Joined: 06 Jul 2005 Posts: 6656 Location: The soundosphere
|
|
Back to top |
|
|
The_Document Apprentice
Joined: 03 Feb 2018 Posts: 275
|
Posted: Wed Mar 21, 2018 2:10 am Post subject: |
|
|
audiodef wrote: | I think you want genkernel. Make sure to ~arch keyword it. |
Sure but the kernel thing is still a mystery to me. Not sure what you meant by arch keywording also, I did add it to package.accept_keywords |
|
Back to top |
|
|
Juippisi Developer
Joined: 30 Sep 2005 Posts: 758 Location: /home
|
Posted: Wed Mar 21, 2018 8:00 am Post subject: |
|
|
There's debian-sources available in Funtoo's repository. Just a warning, building it requires over 20 GB space so you can be sure it builds every module ;)
Anyway you can study that ebuild and their kernel eclass to do it on Gentoo as well. It uses genkernel. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10679 Location: Somewhere over Atlanta, Georgia
|
Posted: Wed Mar 21, 2018 11:41 am Post subject: |
|
|
The_Document wrote: | ... Not sure what you meant by arch keywording ... | See the Handbook, Mixing Software Branches topic.
As for your other questions:- Do they really enable ALL options? Pretty much: all non-conflictiing ones.
- How do only applicable modules load? Does the software probe hardware EVERYTIME it is booted to find appropriate modules to load? Yes
- How would I go about making a kernel in the same fashion as say ubuntu does but with gentoo? Is there a guide to this kernel fashion? As audiodef implied, genkernel does this.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
Tony0945 Watchman
Joined: 25 Jul 2006 Posts: 5127 Location: Illinois, USA
|
Posted: Wed Mar 21, 2018 3:01 pm Post subject: Re: How do binary distros do their kernel? |
|
|
The_Document wrote: | How do only applicable modules load? |
In earlier versions of OpenRC and eudev, only modules listed in /etc/conf.d/modules were loaded.
Later, all modules were loaded except those listed in /etc/modprobe.d/blacklist.conf
Generally, modules fail to load if the hardware they support isn't present. I would guess that they try to initialize the hardware and exit when that fails.
Distro's like Ubuntu and RedHat build their kernels with a wide range of modules, try to load them all and the relevant modules manage to load.
I recommend only building modules for hardware that you have. If you have say a built-in nic and an add-in nic, build both modules and blacklist one. That way, it's easy to switch if you have trouble with the hardware, just move your plug, edit the blacklist file and reboot.
https://forums.gentoo.org/viewtopic-t-1053466-start-0.html
https://wiki.gentoo.org/wiki/Kernel_Modules
https://wiki.archlinux.org/index.php/OpenRC
You can find more by googling "openrc load all modules". |
|
Back to top |
|
|
The_Document Apprentice
Joined: 03 Feb 2018 Posts: 275
|
Posted: Wed Mar 21, 2018 9:19 pm Post subject: |
|
|
John R. Graham wrote: | As audiodef implied, genkernel does this. |
No it does not, I checked its .config and most items are not enabled however some very ridiculous things like isdn, wimax ect are enabled, I should try out debians kernel tool. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10679 Location: Somewhere over Atlanta, Georgia
|
Posted: Wed Mar 21, 2018 11:36 pm Post subject: |
|
|
You're right; I misspoke. The kernel can be configured for many purposes, and depending on purpose, many options are on or off. A typical distro sets up the kernel for desktop usage; this is what Genkernel does. But essentially all of the device drivers are set to be built as modules so that the built-in hardware detection can load them if the hardware is present.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
The_Document Apprentice
Joined: 03 Feb 2018 Posts: 275
|
Posted: Wed Mar 21, 2018 11:40 pm Post subject: |
|
|
John R. Graham wrote: | You're right; I misspoke. The kernel can be configured for many purposes, and depending on purpose, many options are on or off. A typical distro sets up the kernel for desktop usage; this is what Genkernel does. But essentially all of the device drivers are set to be built as modules so that the built-in hardware detection can load them if the hardware is present.
- John |
Following the genkernel guide from here https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel ended up not having GPU acceleration because no DRM was enabled and AMDGPU wasen't selected also and Im sure even if it was selected, it would have never even loaded the applicable firmware files because does not install firmware by default and even if it were firmware file names would proably still would need to be manually added to kernel config. I recommend dumping genkernel and using whatever Ubuntu is using, because they obviously got it right. It wold make things simpler for users who don't want to configure their kernels. Not enabling hugely important things such as DRM and AMDGPU is inexcusable especially considering this is for desktop use. Even framebuffer drivers were disabled. Genkernel is a broken tool.
Made a bug:
https://bugs.gentoo.org/651126 |
|
Back to top |
|
|
Tony0945 Watchman
Joined: 25 Jul 2006 Posts: 5127 Location: Illinois, USA
|
Posted: Thu Mar 22, 2018 2:10 am Post subject: |
|
|
The_Document wrote: | and I'm sure even if it was selected, it would have never even loaded the applicable firmware files because does not install firmware by default and even if it were firmware file names would probably still would need to be manually added to kernel config. | I'm willing to be corrected but I was under the impression that if you built a driver as a module, you didn't have to have the firmware in the kernel and if you emerge linux-firmware, the module will find it on it's own. Unless you have an unusual file structure. I've used, on various machines, the radeon and nouveau drivers as modules and never included any special firmware. Likewise the r8169, r8138too and ieee1000 drivers.
Amdgpu is relatively new and might not be supported by genkernel yet. |
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Thu Mar 22, 2018 10:24 am Post subject: |
|
|
they just build the kernel with make allmodconfig mostly and enable some critical ones that "should" appears in most users system and are critical to boot : ie, any generic kernel should have ahci build-in because everyone use such controller, if you want a kernel able to boot most adaptec raid cards, you will buildin aacraid (and this is where ubuntu and friends are bloated, because if user is not using aacraid, it will be load unconditionally as only a module may not be loaded if hardware is not present). For ahci it's like a 99% good bet, and even it might appears little to load aacraid, you have to do that for every controller user "might" have, and the list gets long then, adding little bloat over previous one (which might be lowered thru grub option with some "normal kernel" + "kernel for raid" options and proper initram set).
That's the biggest problem there: what controller you should build-in to get your kernel working fine with most configurations.
You have also usb and at keyboard support, for usb build-in or not, you might bet the computer has some usb ports and it will certainly be use, so build-in usb support allowing to boot an usb keyboard, but at keyboard support is adding bloat for user with usb keyboard...
for the other "non critical" modules to boot, they just let the device manager do the work for them as it will autoload module that are needed for hardware.
for other "software" tools (configs or loop per example), you have to either add them to your init autoload module list, or just use them as build-in (again always loaded even not of any use), but you can leave the task to the user to load them when need.
This gave 3 options: opt-in (user load them), opt-out (you have load them, user have to unload them), or bloat (you don't use them as module but buildin).
I don't know which options they use, i think a mix of opt-out and opt-in, and because some hard dumb, maybe bloat too. |
|
Back to top |
|
|
Naib Watchman
Joined: 21 May 2004 Posts: 6069 Location: Removed by Neddy
|
Posted: Thu Mar 22, 2018 11:00 am Post subject: |
|
|
this is Ubuntu's config
https://gist.github.com/eeyrjmr/279a0930253831a34be53e74ae4311cb _________________ #define HelloWorld int
#define Int main()
#define Return printf
#define Print return
#include <stdio>
HelloWorld Int {
Return("Hello, world!\n");
Print 0;
Last edited by Naib on Thu Mar 22, 2018 11:44 am; edited 1 time in total |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54644 Location: 56N 3W
|
Posted: Thu Mar 22, 2018 11:11 am Post subject: |
|
|
Naib,
Postbin please. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
audiodef Watchman
Joined: 06 Jul 2005 Posts: 6656 Location: The soundosphere
|
|
Back to top |
|
|
Jaglover Watchman
Joined: 29 May 2005 Posts: 8291 Location: Saint Amant, Acadiana
|
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22877
|
Posted: Fri Mar 23, 2018 1:53 am Post subject: |
|
|
Tony0945 wrote: | I'm willing to be corrected but I was under the impression that if you built a driver as a module, you didn't have to have the firmware in the kernel and if you emerge linux-firmware, the module will find it on it's own. | That is mostly true. To be precise about the rules:- Firmware must be available when the module initializes.
- Firmware in the standard place on the filesystem can be found automatically, but only if the filesystem is mounted and usable when the module initializes.
- Modules are normally stored on the filesystem, so they cannot be loaded before the filesystem is ready. This ensures that firmware is also available.
- Modules can be placed in an initramfs. If so, they can load before the filesystem is ready and will not have access to on-disk firmware. There are some other similarly unusual circumstances where a module can load without the main filesystem available and ready.
That last point is the tricky one. It makes your observation incorrect in a very small number of specialized cases, but your statement is accurate for everyone else. |
|
Back to top |
|
|
The_Document Apprentice
Joined: 03 Feb 2018 Posts: 275
|
Posted: Fri Mar 23, 2018 1:59 am Post subject: |
|
|
Will try it, hopefully I wont load a bunch of pata drivers. Can anybody tell me what commands are issued to make and install the initramfs? |
|
Back to top |
|
|
Tony0945 Watchman
Joined: 25 Jul 2006 Posts: 5127 Location: Illinois, USA
|
Posted: Fri Mar 23, 2018 2:49 am Post subject: |
|
|
Hu wrote: | It makes your observation incorrect in a very small number of specialized cases, but your statement is accurate for everyone else. |
Thank you, that was substantially my understanding but i appreciate your detailed clarification. I don't use an initramfs or a separate /usr and sometimes forget that others do. |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9847 Location: almost Mile High in the USA
|
Posted: Fri Mar 23, 2018 3:19 am Post subject: |
|
|
Also keep in mind many of the other distros keep even the rootfs disk driver as a module and loads it upon boot from the initramfs - so no unused drivers remain in memory despite a generic kernel being used. _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
The_Document Apprentice
Joined: 03 Feb 2018 Posts: 275
|
Posted: Fri Mar 23, 2018 6:37 am Post subject: |
|
|
audiodef wrote: | If you do a manual kernel config, this tool is very useful. Use it with lspci -n: https://kmuto.jp/debian/hcl/
If you use genkernel, it pays to RTFM. There are many options, including saving/using custom configs to get the desired outcome.
It looks like with a combination of the tool above, genkernel options, and studying what other distros do, you could come up with a custom genkernel config to reliably maintain and update kernels as desired. If you, please share, others might benefit. |
Pretty useless tool considering its not updated. |
|
Back to top |
|
|
|