| View previous topic :: View next topic |
| Author |
Message |
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
 Posted: Fri Oct 19, 2018 3:20 am Post subject: Tip: speeding up modprobe bash-completion approximately 500x |
 |
|
Symptom:
| Code: | ~ $ time _modules
real 1m34.954s
user 0m6.820s
sys 0m3.727s |
The _modules command is the bashcomp helper function that finds kernel modules. I was getting a bit annoyed at how slow it was and decided to investigate.
In a discovery that'll probably give SteveL an aneurysm, I found it attempting to parse the output of ls -RL. It's scanning the entire kernel source tree in the /lib/modules/$(uname -r)/source symlink just to find module files under /lib/modules. My kernel source happens to be on an NFS mount, so it hits even harder. Not only that, the results end up being completely useless - it lists every single intermediate object file in the kernel tree as a loadable module.
This is a bit of a rubbish situation imo, so I wrote a sane implementation. Drop this in bashrc.d to fix it:
| /etc/bash/bashrc.d/fix_modules_bashcomp.sh: | _modules()
{
local modpath
modpath=/lib/modules/$1
COMPREPLY=( $( compgen -W "$( find -L "$modpath" -type f -xdev -regex '.*\.k?o\(\.[gx]z\)?' 2>/dev/null | \
command sed -e 's@^.*/\(.*\)\.k\?o\(\.[gx]z\)\?@\1@' )" -- "$cur" ) )
} |
Result:
| Code: | ~ $ time _modules
real 0m0.026s
user 0m0.031s
sys 0m0.012s |
I'm not the only Gentoo user to have noticed this, but upstream doesn't seem to care: https://github.com/scop/bash-completion/issues/136 |
|
| Back to top |
|
 |
khayyam
Watchman
Joined: 07 Jun 2012
Posts: 6227
Location: Room 101
|
| Posted: Fri Oct 19, 2018 9:29 am Post subject: Re: Tip: speeding up modprobe bash-completion approximately |
|
|
| Ant P. wrote: | | /etc/bash/bashrc.d/fix_modules_bashcomp.sh: | _modules()
{
local modpath
modpath=/lib/modules/$1
COMPREPLY=( $( compgen -W "$( find -L "$modpath" -type f -xdev -regex '.*\.k?o\(\.[gx]z\)?' 2>/dev/null | \
command sed -e 's@^.*/\(.*\)\.k\?o\(\.[gx]z\)\?@\1@' )" -- "$cur" ) )
} |
|
Ant P. ... hehe, too much to expect a shell to glob for 'files' only, and return only the required part of the name?
| Code: | % print -rl /lib/modules/$(uname -r)/(*~(source|build))/**/*.ko*(.:t:r) | head -n 2
acpi-cpufreq
applesmc |
The '_modutils' zsh completion has the following:
| _modutils: | | modules=( $modules_dir/$kver/(*~(source|build))/**/*.(o|ko|ko.gz|ko.xz)(.:t:r:r) ) |
Anyhow, what I wanted to say was, you can probably improve the speed by excluding 'source' and 'build' from the find.
best ... khay |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Sat Oct 20, 2018 8:31 pm Post subject: |
|
|
Oh, good catch, I was using -xdev to exclude /source but that'd only work on my machine, and I completely missed the /build dir.
Here's a revised version, still not as pretty as I'd like—bash doesn't have an inline equivalent to (.:t:r)—but it's correct, and more importantly, faster :)
| /etc/bash/bashrc.d/fix_modules_bashcomp.sh: |
_modules()
{
COMPREPLY=( $( compgen -W "$(shopt -s globstar failglob
printf '%s\n' /lib/modules/"${1:?}"/!(source|build)/**/*.@(o|ko|ko.gz|ko.xz) | \
sed -e 's@^.*/\(.*\)\.k\?o\(\.[gx]z\)\?@\1@' )" -- "$cur" ) )
} |
| Code: | ~ $ time _modules $(uname -r)
real 0m0.008s
user 0m0.006s
sys 0m0.005s |
Not a bad speedup.
My numbers were a bit off too. The first one is a 4000× improvement, this is ~12000×. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
