View previous topic :: View next topic |
Author |
Message |
ppurka Advocate
Joined: 26 Dec 2004 Posts: 3256
|
Posted: Sat Sep 05, 2015 2:38 pm Post subject: |
|
|
Update Got my answer. Renamed the gentoo.conf.rsync to .gentoo.conf.rsync and it worked. It seems the rsync configuration was overriding the git configuration. Strange, because I thought it only reads files ending in .conf
I can not seem to sync using git. Can someone explain the reason?
Code: | ~» mount | grep '/usr/portage' # portage is on a ext2 formatted file
/mnt/usb/portage/portageFile on /usr/portage type ext2 (rw,noatime)
~» ls -a /usr/portage # the entire directory is empty
. .. lost+found
~» l /etc/portage/repos.conf # the repository information is on gentoo.conf
total 8.0K
4.0K gentoo.conf 4.0K gentoo.conf.rsync
~» cat /etc/portage/repos.conf/gentoo.conf # the contents
[DEFAULT]
main-repo = gentoo
[gentoo]
location = /usr/portage
sync-type = git
sync-uri = git://anongit.gentoo.org/repo/gentoo.git
auto-sync = yes
~» grep -i repo /etc/portage/make.conf # nothing else in make.conf
-fam -ruby -harfbuzz -crashreporter -cvs avx -mercurial gtk2 \
~» emaint sync -a
>>> Syncing repository 'gentoo' into '/usr/portage'...
!!! getaddrinfo failed for 'rsync.gentoo.org': [Errno -2] Name or service not known
>>> Starting rsync with rsync://rsync.gentoo.org/gentoo-portage...
| Why does emaint still try to use rsync? _________________ emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/ |
|
Back to top |
|
|
hasufell Retired Dev
Joined: 29 Oct 2011 Posts: 429
|
Posted: Sat Sep 05, 2015 5:14 pm Post subject: |
|
|
I don't know if it can properly initialize the repo if the "/usr/portage" directory already exists. In addition, if it's gone, the "profiles/" subdirectory of the portage tree is missing which means you don't even have a working profile. That can break/confuse some tools.
I'd do the initial clone manually.
edit: tested in a docker environment, it should work if you remove /usr/portage first |
|
Back to top |
|
|
ppurka Advocate
Joined: 26 Dec 2004 Posts: 3256
|
Posted: Sun Sep 06, 2015 1:35 am Post subject: |
|
|
hasufell wrote: | I don't know if it can properly initialize the repo if the "/usr/portage" directory already exists. In addition, if it's gone, the "profiles/" subdirectory of the portage tree is missing which means you don't even have a working profile. That can break/confuse some tools.
I'd do the initial clone manually.
edit: tested in a docker environment, it should work if you remove /usr/portage first | Yes. It works with a completely empty /usr/portage. In my case, it wasn't even using the git protocol. It was because I had two files
Code: | ~» l /etc/portage/repos.conf # the repository information is on gentoo.conf
total 8.0K
4.0K gentoo.conf 4.0K gentoo.conf.rsync | The second one was a backup of the rsync config. Unfortunately, emaint reads all files in /etc/portage/repos.conf - not just those files which are ending in .conf.
Once I removed the gentoo.conf.rsync (actually renamed it to .gentoo.conf.rsync), it started using git. But then it again failed because I had newly formatted /usr/portage and the formatting created a lost+found folder. I had to delete that folder, and then git sync worked. And git sync was incredibly fast - I eventually used the github repo. _________________ emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/ |
|
Back to top |
|
|
WWWW Tux's lil' helper
Joined: 30 Nov 2014 Posts: 143
|
Posted: Fri Sep 11, 2015 10:49 am Post subject: |
|
|
I was reading about wtf was $harepoint, and apparently can be used for code versioning. |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 445
|
Posted: Fri Sep 11, 2015 11:22 am Post subject: |
|
|
WWWW wrote: | I was reading about wtf was $harepoint, and apparently can be used for code versioning. |
How is this related? Are you trying to fake you post count for some reason? |
|
Back to top |
|
|
Yamakuzure Advocate
Joined: 21 Jun 2006 Posts: 2305 Location: Adendorf, Germany
|
Posted: Wed Sep 16, 2015 8:30 am Post subject: |
|
|
Just a question:
Is it normal that when ever I sync I get full profile updates? Code: | Performing Global Updates
(Could take a couple of minutes if you have a lot of binary packages.)
.='update pass' *='binary update' #='/var/db update' @='/var/db move'
s='/var/db SLOT move' %='binary move' S='binary SLOT move'
p='update /etc/portage/package.*'
/usr/portage/profiles/updates/1Q-2010..........................................
................................
/usr/portage/profiles/updates/2Q-2010.......................................
/usr/portage/profiles/updates/3Q-2010............
/usr/portage/profiles/updates/4Q-2010............................
/usr/portage/profiles/updates/1Q-2011..........................................
.................................
/usr/portage/profiles/updates/2Q-2011..........................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
...............................................................................
......................................................
/usr/portage/profiles/updates/3Q-2011..........................
/usr/portage/profiles/updates/4Q-2011..........................................
...................................................................
/usr/portage/profiles/updates/1Q-2012.......................
/usr/portage/profiles/updates/2Q-2012........................
/usr/portage/profiles/updates/3Q-2012..........................................
...........
/usr/portage/profiles/updates/4Q-2012..........................................
...................
/usr/portage/profiles/updates/1Q-2013..........................................
........................................................
/usr/portage/profiles/updates/2Q-2013...............
/usr/portage/profiles/updates/3Q-2013............
/usr/portage/profiles/updates/4Q-2013......
/usr/portage/profiles/updates/1Q-2014.......
/usr/portage/profiles/updates/2Q-2014..
/usr/portage/profiles/updates/3Q-2014................
/usr/portage/profiles/updates/4Q-2014.............
/usr/portage/profiles/updates/1Q-2015..........................................
.........................
/usr/portage/profiles/updates/2Q-2015..........................................
...............................................................................
...............................................................................
.........................................
/usr/portage/profiles/updates/3Q-2015......................
# | It finishes in about a second, but how is this triggered and how can I tell the system that those updates are already performed?
Oh, and btw: where are all ChangeLog files gone?
Edit: Got my answer: hasufell wrote: | miket wrote: | (and lots of luck sorting it out if the committer commited several things at once) |
There is no luck involved. Git can show you _any_ commit message that has touched a given file (or directory), no matter if it has been part of a bigger commit. It is far superior to any ChangeLogs we had. | Yes, that is of course right. The ChangeLog files are just a bit cleaner and easier to get with a structure nicer to read.
But as that is a pure personal preference and has nothing to do with the information level, which, of course, isn't any higher in ChangeLog files than the git log, I am completely content with the ChangeLog files removal. _________________ Edited 220,176 times by Yamakuzure |
|
Back to top |
|
|
dweezil-n0xad Apprentice
Joined: 30 Oct 2006 Posts: 156 Location: Ostend, Belgium
|
|
Back to top |
|
|
charles17 Advocate
Joined: 02 Mar 2008 Posts: 3685
|
Posted: Thu Sep 17, 2015 11:58 am Post subject: |
|
|
So we could finally use this without any postsync stuff. Code: | /etc/portage/repos.conf/gentoo.conf
[DEFAULT]
main-repo = gentoo
[gentoo]
location = /usr/portage
sync-type = git
sync-uri = https://github.com/gentoo-mirror/gentoo.git
auto-sync = yes |
Is this sync-uri unique or will there be uris to select as are for sync selected by mirrorselect -i -r -o >> /tmp/foobar? |
|
Back to top |
|
|
hasufell Retired Dev
Joined: 29 Oct 2011 Posts: 429
|
Posted: Thu Sep 17, 2015 12:07 pm Post subject: |
|
|
charles17 wrote: |
So we could finally use this without any postsync stuff. | If you don't care about git history and gpg signatures... |
|
Back to top |
|
|
Yamakuzure Advocate
Joined: 21 Jun 2006 Posts: 2305 Location: Adendorf, Germany
|
Posted: Thu Sep 17, 2015 12:09 pm Post subject: |
|
|
Oh! I am still using hasufell's postsync stuff, so that might have caused the constant repeating of the update checks, right? _________________ Edited 220,176 times by Yamakuzure |
|
Back to top |
|
|
hasufell Retired Dev
Joined: 29 Oct 2011 Posts: 429
|
Posted: Thu Sep 17, 2015 12:15 pm Post subject: |
|
|
Yamakuzure wrote: | Oh! I am still using hasufell's postsync stuff, so that might have caused the constant repeating of the update checks, right? |
I doubt it, haven't experienced that here.
The updates are part of the main repository, no matter what repo source you use. |
|
Back to top |
|
|
steveL Watchman
Joined: 13 Sep 2006 Posts: 5153 Location: The Peanut Gallery
|
Posted: Sat Sep 19, 2015 7:10 am Post subject: |
|
|
hasufell wrote: | If you don't care about git history and gpg signatures... |
Funny, just on the previous page, you were saying that the spiffing new git repo does not, and will not, have history:
hasufell wrote: | The old history will not be re-added to the current development repository |
Now here you are claiming how wonderful it is, "because history." |
|
Back to top |
|
|
hasufell Retired Dev
Joined: 29 Oct 2011 Posts: 429
|
Posted: Sat Sep 19, 2015 9:37 am Post subject: |
|
|
steveL wrote: | hasufell wrote: | If you don't care about git history and gpg signatures... |
Funny, just on the previous page, you were saying that the spiffing new git repo does not, and will not, have history: |
That is wrong, the repo does have history, it was just reset. Please refrain from spreading your negativity, hate and drama for no reason. You are neither productive, nor helpful. You should seriously reconsider your behavior.
PS: I'm going to unsubscribe from this topic... if anyone needs further help wrt the scripts, create an issue on github |
|
Back to top |
|
|
ct85711 Veteran
Joined: 27 Sep 2005 Posts: 1791
|
Posted: Sat Sep 19, 2015 3:43 pm Post subject: |
|
|
I noticed, even though it's in the tree on github, it's still not being pulled.
According to the .gitignore file in metadata
Code: |
# These are added during tree generation only.
/cache
/dtd
/glsa
/md5-cache
/news
/herds.xml
/timestamp
/timestamp.chk
/timestamp.x
/pkg_desc_index
|
They are still on the ignore list, seems that .gitignore needs to be updated so that those directories will start being pulled once again. |
|
Back to top |
|
|
miket Guru
Joined: 28 Apr 2007 Posts: 497 Location: Gainesville, FL, USA
|
Posted: Sat Sep 19, 2015 3:51 pm Post subject: |
|
|
I wish I had had enough time to review the patching-in technique you described for the older history. Here is the big point, though, one I wish you would have kept in mind: older history matters.
It is not acceptable to have the older history available. Unless that git-replace technique you described can do a seamless merge of the old history, you're not done.
hasufell wrote: | That is wrong, the repo does have history, it was just reset. Please refrain from spreading your negativity, hate and drama for no reason. You are neither productive, nor helpful. You should seriously reconsider your behavior.
PS: I'm going to unsubscribe from this topic... if anyone needs further help wrt the scripts, create an issue on github |
Great way to avoid the issue. Recognizing that we need the old history is not being negative or counterproductive. |
|
Back to top |
|
|
ct85711 Veteran
Joined: 27 Sep 2005 Posts: 1791
|
Posted: Sat Sep 19, 2015 4:38 pm Post subject: |
|
|
miket, think it more of falls down to time vs value. Right now, how I am seeing it, doing all that work doesn't payoff the time spent. Now if this was a development tree, I can see where it would pay off for the extra amount of time however for this tree it doesn't.
Then it also runs into, how many people really need that long of history for the portage tree? Making the assumption that a change now makes something wrong does not also make the past wrong too (especially at the scale you are wanting). |
|
Back to top |
|
|
steveL Watchman
Joined: 13 Sep 2006 Posts: 5153 Location: The Peanut Gallery
|
Posted: Sat Sep 19, 2015 5:17 pm Post subject: |
|
|
hasufell wrote: | Please refrain from spreading your negativity, hate and drama for no reason. |
WTF are you talking about? It was a simple statement, alluding to the content miket posted above in response to your negativity.
Projecting your issues on me, then asserting that I behave in manner X, does not make it true.
Though you're not alone in trying it.
Funny how often it happens with formerly-lucid developers who become paludroids; normally after about 12-18 months indoctrination in the myth that the cult-members are a "persecuted minority".
Typically they then start kicking up a fuss about how no-one wants paludis as a default (boo-hoo.)
There were several years of very good reasons why McCreesh was forcibly ejected from Gentoo, and why his behaviour thereafter formed the background to the Developer CoC, including its effect on developers after they leave Gentoo.
You really should reconsider both your approach, your manners, and the people you associate with.
I'll return to the actual topic next week; I had no intention of even posting to this thread until then, then I found out you'd done your usual "ban him, because I don't like him, let me count the made-up reasons" nonsense.
Consider whether you are not just a troll, however wonderful you might think you are (before you actually have to run your ideas by others, let alone real-life.) |
|
Back to top |
|
|
miket Guru
Joined: 28 Apr 2007 Posts: 497 Location: Gainesville, FL, USA
|
Posted: Sat Sep 19, 2015 5:29 pm Post subject: |
|
|
I'll concede that not often will people need to look back 10 years, but it does occur quite often that people need that history from just a few months ago. I see that kind of thing a lot in the forums when someone is trying to go back to some version no longer in the tree so that they can unbreak their machines. Or, as in my case, try to figure out where something went wrong so that I can file a meaningful bug report.
Didn't, as I understand, someone already go through the big effort to make that happen? What happened to that? |
|
Back to top |
|
|
ct85711 Veteran
Joined: 27 Sep 2005 Posts: 1791
|
Posted: Sat Sep 19, 2015 6:21 pm Post subject: |
|
|
I do know they did try to go through the effort to port the history over, when this was originally thought up. I couldn't really say on what happen on that part, the most I can do is guess on the reasons. I suspect they more of had in issue where they can't get the commits from CVS to line up to git (most likely the case) or more of space usage. |
|
Back to top |
|
|
miket Guru
Joined: 28 Apr 2007 Posts: 497 Location: Gainesville, FL, USA
|
Posted: Sat Sep 19, 2015 9:38 pm Post subject: |
|
|
ct85711 wrote: | I do know they did try to go through the effort to port the history over, when this was originally thought up. I couldn't really say on what happen on that part, the most I can do is guess on the reasons. I suspect they more of had in issue where they can't get the commits from CVS to line up to git (most likely the case) or more of space usage. |
A few years ago at work I wrote and ran a script to grab our most ancient CVS history and bring it into a Mercurial repository. One factor that made it more difficult is one to which you allude: lining up CVS commits in order to make Mercurial commits (Mercurial has been our main VCS since 2007). The problem, of course, is that CVS commits are simply per-file; there is no good way to find everything that happened for the commit of a large changeset. My solution was to look for all the CVS commits from a given author within a short time interval and stage them into a single Mercurial commit.
In general, this is a bit less of an issue with Portage than it was in our system: while we needed the changes to many files to appear coherently in changsets, in Portage the commits are often just for single ebuilds.
The process is certainly doable. With git it would be faster than with Mercurial and would take up less space. One thing that made the conversion easer for me was that I had direct access to the RCS files themselves.
I did this because our main Mercurial repository encompassed only those changes since we switched from CVS to Subversion in 2005. That switch involved a packaging change from a set of a couple of dozen separate repositories into a single repository. My work also coalesced those separate CVS repositories into a directory structure like the one we adopted at the time of the switch to Subversion. (Funny, we were on Subversion only 2 years before switching to Mercurial.)
There are many times I've found my reconstructed repository to be useful--even if it is for commits more than 10 years old.
The whole Portage CVS repository manages to sit on a Gentoo server. It certainly is of finite size. As to how large and how long it would take to transmit this if converted to git, I have no good guess.
Our main system at work has maybe 10,000 files in version control; the average size is around 10Kb. It does not take unmanagably long to get a checkout with that 10 years of history behind it (my reconstruction of the CVS history is not in that main repository). Portage has about 15 times that number of files, but the average size is more like 5Kb. (No wonder syncing takes so long!)
The whole of Portage history might be more than most users might want to slurp up, but at least it ought to be available. Would not have it been good to obtain the whole converted repository but be able to slice off part of it for distribution? At reasonable-enough intervals (say once a year), grab a patch of the last n years or months from the end of the big repository. For normal syncing, send that.
--Or-- (I think this would work but I haven't used git enough to be sure), make users' the initial clones be shallow clones at whatever convenient depth so that subsequent pull requests would pick up only newer revisions. That would involve no periodic splitting from the big repository. The pull requests at every normal sync would pick up only the newer changes.
There would still be a good basis for people who wanted to pull the whole authoritative history and use it (perhaps for routine uses like displaying good reconstructed-from-commit-history changelogs on sites like znurt.org).
After all, even those developers complaining it would take too long to transmit and store lots of really old Portage history and therefore took the expedient of making this unhappy August 2015 chop-point are going to be up against a new reality three or four years hence: the Portage repo will have accumulated lots of old history and those initial clones are going to take longer than maybe they want it to take.
I think that either of my proposals for using a grand authoritative repository is much more sustainable in the long term. It would be more honest, at least. |
|
Back to top |
|
|
desultory Bodhisattva
Joined: 04 Nov 2005 Posts: 9410
|
Posted: Mon Sep 21, 2015 4:03 am Post subject: |
|
|
hasufell wrote: | steveL wrote: | hasufell wrote: | If you don't care about git history and gpg signatures... |
Funny, just on the previous page, you were saying that the spiffing new git repo does not, and will not, have history: |
That is wrong, the repo does have history, it was just reset. Please refrain from spreading your negativity, hate and drama for no reason. You are neither productive, nor helpful. You should seriously reconsider your behavior.
PS: I'm going to unsubscribe from this topic... if anyone needs further help wrt the scripts, create an issue on github | It hardly seems to be "spreading your negativity, hate and drama for no reason" to cite what you yourself wrote . Granted, there have been some comments which reflect being somewhat less than fully informed on the git migration, but that also hardly seems to qualify, and certainly does not justify your claims made against steveL. If, by some chance, you do eventually lodge a valid complaint against steveL, it would be rather less likely to be dismissed out of hand if you would stop spuriously reporting him for doing what you are actually doing. |
|
Back to top |
|
|
alinefr Tux's lil' helper
Joined: 05 Jul 2009 Posts: 113 Location: São Paulo, Brasil
|
Posted: Thu Feb 18, 2016 1:29 am Post subject: |
|
|
Someone in this thread suggested to set:
CACHE_METHOD="/usr/portage/ parse|ebuild*"
... as eix-update was not working by default when 'sync-type = git'. But it was taking ages to build the database. So I did try different values for CACHE_METHOD, and in the end 'assign' is working as it should. A lot more faster than parse|ebuild:
So my CACHE_METHOD is set like this now:
CACHE_METHOD="/usr/portage/ assign" |
|
Back to top |
|
|
ExecutorElassus Veteran
Joined: 11 Mar 2004 Posts: 1464 Location: Berlin, Germany
|
Posted: Tue Oct 11, 2016 9:52 pm Post subject: |
|
|
I have /usr/portage/distfiles as a separate partition. When I try to sync, I get an error:
Code: | # emerge --sync
>>> Syncing repository 'gentoo' into '/usr/portage'...
/usr/bin/git clone --depth 1 https://github.com/gentoo-mirror/gentoo .
fatal: destination path '.' already exists and is not an empty directory.
!!! git clone error in /usr/portage
q: Updating ebuild cache in /usr/portage ...
q: Finished 0 entries in 0.000037 seconds |
Is there a way to use git without having to re-jigger all my partitions?
Cheers,
EE |
|
Back to top |
|
|
grumblebear Apprentice
Joined: 26 Feb 2008 Posts: 204
|
Posted: Wed Oct 12, 2016 5:24 pm Post subject: |
|
|
Your problem is the initial cloning? Just do:
umount /usr/portage/distfiles
emerge --sync
mount /usr/portage/distfiles |
|
Back to top |
|
|
ExecutorElassus Veteran
Joined: 11 Mar 2004 Posts: 1464 Location: Berlin, Germany
|
Posted: Wed Oct 12, 2016 5:30 pm Post subject: |
|
|
It was just the initial cloning, but then a dev told me to move what used to be /usr/portage to /var/portage/repos and distfiles to /var/portage/distfiles, so that it's no longer within the tree that git syncs to. I had to move mountpoints around and edit my fstab, but now it works.
But thanks for the tip. It's good to know I can also just unmount the partition.
Cheers,
EE |
|
Back to top |
|
|
|