View previous topic :: View next topic |
Author |
Message |
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
Posted: Sat Jun 24, 2023 5:44 pm Post subject: |
|
|
I've updated https://delta-labs.org/portagefilelist/ and it should be fully working now.
Currently there is only the data from my pc available and it would be greate if some of you could share some data to validate the whole thing.
If you want to, install https://packages.gentoo.org/packages/app-portage/pfl and modify the python pfl file to match the alternate endpoint.
The pfl.py file should be in /usr/lib/python-exec/<VERSION>/pfl.py or /usr/lib/<VERSION>/site-packages/pfl/pfl.py. Open it and search for UPLOADURL=. Change the value to https://delta-labs.org/portagefilelist/data.php
Run it and the data should be uploaded to the new endpoint. Please provide feedback, either here or as a PM. After that I will run the import to check if everything is working. _________________ Forum Guidelines
PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire
Last edited by Banana on Tue Jun 27, 2023 7:34 am; edited 1 time in total |
|
Back to top |
|
|
dalek Veteran
Joined: 19 Sep 2003 Posts: 1353 Location: Mississippi USA
|
Posted: Sat Jun 24, 2023 9:20 pm Post subject: |
|
|
I just sent mine. It appears to have worked. On my system, the file was located here:
Code: | /usr/lib/python3.11/site-packages/pfl/pfl.py |
Can you tell if you got it or not? _________________ My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54552 Location: 56N 3W
|
Posted: Sat Jun 24, 2023 9:52 pm Post subject: |
|
|
Banana,
Donated the data from my arm64 build server. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
|
Back to top |
|
|
toralf Developer
Joined: 01 Feb 2004 Posts: 3934 Location: Hamburg
|
Posted: Sun Jun 25, 2023 3:15 pm Post subject: |
|
|
If the pfl script would accept an URL as a parameter I could upload from the tinderbox [1] files to the new location too.
[1] https://zwiebeltoralf.de/tinderbox.html
Last edited by toralf on Tue Jun 27, 2023 6:32 pm; edited 1 time in total |
|
Back to top |
|
|
dalek Veteran
Joined: 19 Sep 2003 Posts: 1353 Location: Mississippi USA
|
Posted: Sun Jun 25, 2023 3:16 pm Post subject: |
|
|
Sounds good. Glad to help. I changed it back after I ran it. I do updates on the weekends, will start those shortly, do you want me to send again after my updates? It is usually done with a cron job on Tuesday night, Wednesday morning, for the older site. I can change it, run it manually for your site, change it back so it updates the old site as well. Sort of update both until the change is official.
I set pfl to update years ago. I'd like to see it continue on. _________________ My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54552 Location: 56N 3W
|
Posted: Sun Jun 25, 2023 3:48 pm Post subject: |
|
|
Banana,
I can post more but I don't want you to be overwhelmed with test data.
How far back do you want to go too?
I have a few old binpackages that could be mined for file names
That may not be useful to your readers an many of the ebuilds will exist only in git history.
The old stuff is only there as I have not needed to recycle the disk space yet.
Do you intend to purge data from ebuilds no longer in the tree? _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
Posted: Mon Jun 26, 2023 8:30 am Post subject: |
|
|
Almost dropped some code on my side. ugh. Testing of the new data will be later.
Historical data, good point. Didn't think of those yet. I've got a snapshot of the original DB and I need to make a migration to get the data into the new DB and format.
toralf, sounds interesting. How does the data look like?
dalek, right now it is testing only. Don't know yet how and when the project takeover will be done. I'll let you know if I need more regular data.
NeddySeagoon, as of now I do log when data is imported and updated. So the information about age is present. It depends on the amount of data how long I can keep them.
For all the interested: http://91.132.146.200:3000/Banana/portagefilelist _________________ Forum Guidelines
PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54552 Location: 56N 3W
|
Posted: Mon Jun 26, 2023 9:03 am Post subject: |
|
|
Banana,
I was thinking that you may want to from time to time to prevent the database growing without limit and keeping current packages only.
Then again, historical data is useful to a few people too.
does not exist yet. It would remove file names belonging to packages no longer in the tree from the database.
Much as does today. I have no idea how to do that. Software is hard.
Port 3000 ... my paranoid firewall won't let that out :)
Keep up the good work. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
Posted: Mon Jun 26, 2023 6:05 pm Post subject: |
|
|
I currently dig through the existing pfl db export. After I've managed to get a postgresql server running and importing the file. Right now it is ~180Gb on disk and not finished yet.
I'm not sure if I can keep all the data. If there is a timestamp, maybe check which timeframe and amount of data can I handle.
The software running on port 3000 is https://gogs.io/. So, nothing dangerous there. Sometime on the future I will mirror it to my github account. _________________ Forum Guidelines
PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54552 Location: 56N 3W
|
Posted: Mon Jun 26, 2023 6:20 pm Post subject: |
|
|
Banana,
I did not intend to imply that your site could be dangerous.
Its just that my firewall starts off by denying everything then required services are permitted.
Its amazing the amount of traffic from android devices that is dropped on the way out.
I have no idea what its for but everything seems to work without it, so it's probably a nice to have. Nice for Google that is.
My (Windows 10) work laptop does the same thing ... to Microsoft.
They are both in an untrusted zone, along with everything else wireless.
Firewalls are drifting off topic. To get to port 3000, I need to add a rule to my firewall. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
Juippisi Developer
Joined: 30 Sep 2005 Posts: 743 Location: /home
|
Posted: Tue Jun 27, 2023 6:06 am Post subject: |
|
|
Banana wrote: | I've updated https://delta-labs.org/portagefilelist/ and it should be fully working now.
Currently there is only the data from my pc available and it would be greate if some of you could share some data to validate the whole thing.
If you want to, install https://packages.gentoo.org/packages/app-portage/pfl and modify the python pfl file to match the alternate endpoint.
The pfl.py file should be in /usr/lib/python-exec/<VERSION>/pfl.py. Open it and search for UPLOADURL=. Change the value to https://delta-labs.org/portagefilelist/data.php
Run it and the data should be uploaded to the new endpoint. Please provide feedback, either here or as a PM. After that I will run the import to check if everything is working. |
And here's a .patch file you can drop onto /etc/portage/patches/app-portage/pfl/ directory:
Code: |
diff -Naur a/pfl/pfl.py b/pfl/pfl.py
--- a/pfl/pfl.py 2021-07-25 18:47:15.000000000 +0300
+++ b/pfl/pfl.py 2023-06-27 09:04:13.086468794 +0300
@@ -23,7 +23,7 @@
else:
INFOFILE = '%s/.pfl.info' % HOME;
-UPLOADURL='https://upload.portagefilelist.de/data.php'
+UPLOADURL='https://delta-labs.org/portagefilelist/data.php'
if DEBUG:
print('Portage Version: ', portage.VERSION)
|
then re-emerge pfl. Make sure the patch is applied. This will corret the URL regardless of your PYTHON_COMPAT setting.
Last edited by Juippisi on Wed Jun 28, 2023 5:25 am; edited 2 times in total |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
Posted: Tue Jun 27, 2023 7:53 am Post subject: |
|
|
Neddy, all good. Just wanted to clarify what it is.
Juippisi, thx for the patch but I've made a typo. The new value is https://delta-labs.org/portagefilelist/data.php
The already provided uploads are tested and imported. Had an error in the already imported data and fixed it while testing.
Feel free to provide more uploads.
Thx again. _________________ Forum Guidelines
PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire |
|
Back to top |
|
|
dalek Veteran
Joined: 19 Sep 2003 Posts: 1353 Location: Mississippi USA
|
Posted: Tue Jun 27, 2023 3:24 pm Post subject: |
|
|
I reset the last run to 0 so it would send a complete list. I have KDE but also quite a bit of Gnome installed plus a few other desktops for backup. As you can tell, it's a lot of packages. I won't update again until next weekend. Let me know if you want me to send each time I finish a update or wait until you get a better grasp on things. Tail end of output.
Oh, on first send, I forgot to use the newer URL. Sent twice but last one is the more complete one.
Hope that helps. _________________ My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
Posted: Thu Jul 06, 2023 11:18 am Post subject: |
|
|
I can now confirm that I will take over the repos and website from the current owner tuxmainy.
The domain transfer could lead to some downtime. (not sure when yet) I hope I have everything needed and looking forward what may come. _________________ Forum Guidelines
PFL - Portage file list - find which package a file or command belongs to.
My delta-labs.org snippets do expire
Last edited by Banana on Fri Jul 07, 2023 7:12 am; edited 1 time in total |
|
Back to top |
|
|
Juippisi Developer
Joined: 30 Sep 2005 Posts: 743 Location: /home
|
Posted: Fri Jul 07, 2023 6:08 am Post subject: |
|
|
Awesome :) |
|
Back to top |
|
|
tuxmainy n00b
Joined: 15 Nov 2016 Posts: 8
|
Posted: Sat Jul 08, 2023 1:36 pm Post subject: |
|
|
Banana wrote: | I can now confirm that I will take over the repos and website from the current owner tuxmainy.
The domain transfer could lead to some downtime. (not sure when yet) I hope I have everything needed and looking forward what may come. |
So, here I am dropping some tears on a project I have managed 19 years But it is as it is and not having time to maintain the project won't help PFL anyway. I am pretty sure Banana will continue PFL in good
Banana, as already said by mail, I wish you all the best. Thanks for taking care of PFL.
Dear Gentoo community, thank you for making such a project possible. Without your uploads PFL wouldn't exist. Please continue to support Banana as best as you can. And of course, some of you not just uploaded files but also developed some code, reported bugs or dedicated otherwise. Thank you.
Last but not least thanks to Daniel for maintaining the PFL package.
So Long, and Thanks for All the Fish
Daniel |
|
Back to top |
|
|
Juippisi Developer
Joined: 30 Sep 2005 Posts: 743 Location: /home
|
Posted: Sun Jul 09, 2023 6:24 am Post subject: |
|
|
Thank you both. Glad to see this project live on! |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
|
Back to top |
|
|
dalek Veteran
Joined: 19 Sep 2003 Posts: 1353 Location: Mississippi USA
|
Posted: Sat Jul 22, 2023 8:50 am Post subject: |
|
|
At one point we were asked to upload to a different URL. Can I assume that is no longer needed? Are we to just use the old URL until something changes with a normal software update?
Just want to be sure I'm sending to the right place. _________________ My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
|
Back to top |
|
|
grknight Retired Dev
Joined: 20 Feb 2015 Posts: 1852
|
Posted: Mon Aug 28, 2023 7:26 pm Post subject: |
|
|
I'm sorry to hear that this has become such a poor result.
Not having a lookup that can process valid file names diminishes the value of this site and I will be using it less.
The common underscore is disallowed and I have been blocked from using this site searching for files because of this shortcoming.
The term should not be filtered to remove such things. |
|
Back to top |
|
|
Goverp Advocate
Joined: 07 Mar 2007 Posts: 2130
|
Posted: Mon Aug 28, 2023 9:57 pm Post subject: |
|
|
grknight wrote: | ...
Not having a lookup that can process valid file names diminishes the value of this site and I will be using it less.
... |
As a bypass you could try an "*", but that's clearly overkill. Better would be "?", but that doesn't work.
Hopefully the function will be enhanced. _________________ Greybeard |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1663 Location: Germany
|
|
Back to top |
|
|
|