Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Anyone know how to add more threads to app-portage/fetchc...
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
leyvi
Tux's lil' helper
Tux's lil' helper


Joined: 08 Sep 2023
Posts: 130

PostPosted: Sun Jan 19, 2025 5:08 pm    Post subject: Anyone know how to add more threads to app-portage/fetchc... Reply with quote

Trying to configure app-portage/fetchcommandwrapper. How do I change the default configuration? /usr/share/fetchcommandwrapper/make.conf isn't very helpful, and I want to add more than 5 download threads. Anyone know how?
Back to top
View user's profile Send private message
rab0171610
Guru
Guru


Joined: 24 Dec 2022
Posts: 455

PostPosted: Sun Jan 19, 2025 7:32 pm    Post subject: Reply with quote

That option is set in the installed file, which for me is located at:
/usr/lib/python3.12/site-packages/fetchcommandwrapper/__main__.py

You could modify this file directly and change the line:
Code:
MAX_STREAMS = 5

to something else. For example:
Code:
MAX_STREAMS = 10


You would have to do this every time you install the file or update/rebuild the package through portage.

For a more persistent solution, you might be able to write a patch that will modify the sources. I am not good with patches. If I am simply modifying one line or string in a source file of a downloaded package, I usually use the portage bashrc file, /etc/portage/bashrc:

https://wiki.gentoo.org/wiki//etc/portage/bashrc

For this particular package you would need to set up your own local repo if you have not already:
https://wiki.gentoo.org/wiki/Creating_an_ebuild_repository

I kept the defaults and mine is located at:
/var/db/repos/localrepo

You will need root privileges.
Create the directory:
Code:
mkdir -p /var/db/repos/localrepo/app-portage/fetchcommandwrapper/files


Copy the files from the Gentoo repo to the local repo:

Code:
 cp -rv /var/db/repos/gentoo/app-portage/fetchcommandwrapper/* /var/db/repos/localrepo/app-portage/fetchcommandwrapper

Go to the files directory you just created within:
Code:
cd  /var/db/repos/localrepo/app-portage/fetchcommandwrapper/files

Use your preferred editor (I use nano) to create a bash script that will use sed to change the MAX_STREAMS in the portage downloaded source file from 5 to the desired value, I am using 10 as an example:
Code:
nano MAX_STREAMS.sh

Insert the following text (note the double underscores in the filename):
Code:
#!/bin/bash
sed -i "s/MAX_STREAMS = 5/MAX_STREAMS = 10/" /var/tmp/portage/app-portage/fetchcommandwrapper*/work/fetchcommandwrapper*/fetchcommandwrapper/__main__.py

Hit ctrl+x to save.
Change the permissions to executable:
Code:
chmod +x  MAX_STREAMS.sh

Change to parent directory:
Code:
cd ../

Build the manifest:
Code:
ebuild fetchcommandwrapper-0.8.4-r5.ebuild digest


Next you would create the portage hook. If it does not exist you can create the file:
/etc/portage/bashrc
Insert the following text (edited to your preferences, this is what works for me):
Code:
#!/bin/bash

###Increase MAX_STREAMS for app-portage/fetchcommandwrapper
if [ "${CATEGORY}/${PN}" == "app-portage/fetchcommandwrapper" ] && [ "${EBUILD_PHASE}" == "configure" ] ; then
  echo -e "  \e[31;1m*** The fetchcommandwrapper sources will be modified.\e[0m"
  bash -c '/var/db/repos/localrepo/app-portage/fetchcommandwrapper/files/MAX_STREAMS.sh'
  echo -e "  \e[31;1m*** The fetchcommandwrapper sources were modified.\e[0m"
fi

This will output red coded text that is easy to identify in the portage output when installing fetchcommandwapper. If there are any errors you will see them printed between the two lines of red text:
Code:

 *** The fetchcommandwrapper sources will be modified.
bash: line 1: /var/db/repos/localrepo/app-portage/fetchcommandwrapper/files/MAX_STREAMS.sh: Permission denied
  *** The fetchcommandwrapper sources were modified.

The above would occur if you forgot to make the bash script executable for example.
You may need to first source the portage bashrc file, but I am not certain of that to be honest:
Code:
source /etc/portage/bashrc

You can check after installation if these changes were successful and troubleshoot accordingly by reviewing the contents of the file:
/usr/lib/python3.12/site-packages/fetchcommandwrapper/__main__.py
If all went well it should say:
Code:
MAX_STREAMS = 10

There may, however, be a reason that the developer set the max streams to 5, so be prepared to monitor things for unexpected consequences. The max streams may be an arbitrary number or it may have been considered the optimum setting after thorough testing.


Last edited by rab0171610 on Sun Jan 19, 2025 7:36 pm; edited 1 time in total
Back to top
View user's profile Send private message
sam_
Developer
Developer


Joined: 14 Aug 2020
Posts: 2096

PostPosted: Sun Jan 19, 2025 7:35 pm    Post subject: Reply with quote

Please encourage people to use https://wiki.gentoo.org/wiki//etc/portage/patches instead.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum