| View previous topic :: View next topic |
| Author |
Message |
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
 Posted: Thu Jun 10, 2004 12:35 pm Post subject: Rsync problems... |
 |
|
Hi all,
I have a problem using rsync since 2-3 months ago. My problem is always something like this:
| Code: | rsync error: error in file IO (code 11) at receiver.c(305)
rsync: connection unexpectedly closed (120187 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(189) |
When I use rsync to update the portage tree (emerge sync) everything goes ok, it always finishes without any problem, but I always use rsyn through ssh to update a website tree at work and at home and I always get the same error, sometimes I get it really soon (3rd or 4th file being transfered) and sometimes I get the error later (like in the file 190th being transfered)...
I've been searching the forums and I've seen I'm not the only one having this problem, but I couldn't find a solution yet...
Anybody can help me with this?
Thanks a lot in advance. |
|
| Back to top |
|
 |
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 12:38 pm Post subject: Notes... |
|
|
Ah, I forgot, some adittional notes:
- The error is not always happening in the line 305 of receiver.c, sometimes it complains for other lines.
- I have the same problem (same error) using several servers with different OSs (Debian, FreeBSD, and the other one is, of course, Gentoo  ) |
|
| Back to top |
|
|
saccory
Apprentice
Joined: 18 Feb 2004
Posts: 176
Location: Göttingen, Germany
|
| Posted: Thu Jun 10, 2004 1:03 pm Post subject: |
|
|
| Maybe rsync -v -v -v ... tells you more. Is enough space left on the device you're syncing to? |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 1:10 pm Post subject: ey |
|
|
Thank you for your really quick answer 
1. Well, of course there is enough space in my disks when rsync'ing... (I'm never under 10-15 Gb of free space and the whole website filetree is like 300-400 Mb...)
2. I tryed using super-verbose mode  (-v -v) but the output is not telling much more than other times... here it is:
| Code: | rsync error: error in file IO (code 11) at receiver.c(305)
_exit_cleanup(code=11, file=receiver.c, line=305): about to call exit(11)
recv_generator(slalom/committee/Internet Madrid/calendar/pdf_file_2002/Muster Entree Prague WC 2002.pdf,1746)
generating and sending sums for 1746
count=454 rem=161 blength=700 s2length=2 flength=317261
recv_generator(slalom/committee/Internet Madrid/calendar/portugal Folha 2.DOC,1747)
generating and sending sums for 1747
count=384 rem=188 blength=700 s2length=2 flength=268288
rsync: connection unexpectedly closed (120412 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(189)
_exit_cleanup(code=12, file=io.c, line=189): about to call exit(12) |
Any idea?
thx again
Note: some ppl told me "It's for sure that the filenames contain spaces!!" but NO, the problem is not the filenames, 'cause 3-4 months ago, it was downloading ok exactly the same files with some spaces... |
|
| Back to top |
|
|
saccory
Apprentice
Joined: 18 Feb 2004
Posts: 176
Location: Göttingen, Germany
|
| Posted: Thu Jun 10, 2004 1:24 pm Post subject: |
|
|
Please try a third -v and I think the lines before the error are more interesting.
And just to be clear, my question was: is enough space on the target system? |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 1:41 pm Post subject: hi again |
|
|
really thank you for your speed answering...
Yes, I was also refering I have enough space in the target machines, in those where i want to copy the files to (I work in several machines), but anyway I checked "just in case" 2 mins ago and I have more than 20Gb free in every machine I use rsync (source and target machines....)
Also, my last post includes the output of the error using 3 -v switches, to be exact, here is the command I use:
| Code: | | rsync -v -v -v -zare ssh --delete --exclude=constants.php --exclude=phpmyadmin myuser@myhost.com:~/mywebfolder/ /mylocalfolder/ |
-v (x3) = I always use -v to see what's going on... here i include 3 following your advice to see more data in the error.
-z = compress the data before transfering (I also tried without compress and I have the same error)
-a = archive mode...
-r = be recursive!
-e ssh = use rsync over ssh protocol (I don't have rsync server installed in any machine, just the ssh daemon, it always worked very well using ssh)
I masked the ssh user, server address and local folders for security reasons
Thanks again for your support Saccory  |
|
| Back to top |
|
|
saccory
Apprentice
Joined: 18 Feb 2004
Posts: 176
Location: Göttingen, Germany
|
| Posted: Thu Jun 10, 2004 1:46 pm Post subject: |
|
|
So what does the line above
| Code: | | rsync error: error in file IO (code 11) at receiver.c(305) |
say? |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 6:18 pm Post subject: |
|
|
I unpacked the tarball and checked the file, here are the problematic lines...:
| Code: |
302 if (fd != -1 && write_file(fd,map,len) != (int) len) {
303 rprintf(FERROR, "write failed on %s: %s\n",
304 full_fname(fname), strerror(errno));
305 exit_cleanup(RERR_FILEIO);
306 }
307 offset += len;
|
Thx |
|
| Back to top |
|
|
saccory
Apprentice
Joined: 18 Feb 2004
Posts: 176
Location: Göttingen, Germany
|
| Posted: Thu Jun 10, 2004 6:32 pm Post subject: |
|
|
| No, I meant the output of rsync -v .... Isn't there any error message above the line I cited? The rsync error seems only to be a result of an IO error on the target system. |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 6:36 pm Post subject: |
|
|
Oh, one more thing I want to make clear:
I said in my first post that I use several machines with several OSs. Well, I mean I use several OSs in one side of the connection, but I always launch rsync from a Gentoo machine.
I have 2 Gentoo workstations (home and office) and another Gentoo system acting as router and samba/cups server in the office. I use these 3 machines to upload/download modifications to/from 3 webservers, one is Debian, another FreeBSD and the 3rd one is Gentoo, and I always use rsync. And I see the same error when I upload and when I download with rsync from any Gentoo machine.
I also have to tell, that I tried using rsync from a RedHat Enterprise server against the Gentoo webserver and same error... |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 6:45 pm Post subject: oops! |
|
|
ok lol sorry...
Now I'm in another computer (I'm at home...) and I get the same error, here it is with some lines above the error:
| Code: |
count=88 rem=562 blength=700 s2length=2 flength=61462
recv_generator(slalom/committee/worlds_ranking/2003/senior%20worlds%20augsburg/Results/medal_list.pdf,5618)
generating and sending sums for 5618
count=81 rem=443 blength=700 s2length=2 flength=56443
recv_generator(slalom/committee/worlds_ranking/2003/senior%20worlds%20augsburg/Results/medal_list_augsburg.pdf,5619)
generating and sending sums for 5619
count=83 rem=142 blength=700 s2length=2 flength=57542
recv_generator(slalom/committee/worlds_ranking/2003/senior%20worlds%20augsburg/Results/team_c1_final.pdf,5620)
generating and sending sums for 5620
count=95 rem=177 blength=700 s2length=2 flength=65977
recv_generator(slalom/committee/worlds_ranking/2003/senior%20worlds%20augsburg/Results/team_k1_men.pdf,5621)
generating and sending sums for 5621
count=87 rem=237 blength=700 s2length=2 flength=60437
recv_generator(slalom/committee/worlds_ranking/2003/senior%20worlds%20augsburg/Results/team_k1_women.pdf,5622)
generating and sending sums for 5622
count=92 rem=337 blength=700 s2length=2 flength=64037
write failed on "/mnt/webs/www.canoeicf.com/slalom/committee/news/ECA/SKOPJE/K1WHT2.PDF": Input/output error
rsync error: error in file IO (code 11) at receiver.c(305)
_exit_cleanup(code=11, file=receiver.c, line=305): about to call exit(11)
rsync: connection unexpectedly closed (127847 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(189)
_exit_cleanup(code=12, file=io.c, line=189): about to call exit(12)
|
As you say, it's an IO error, but it fails with different files everytime, also I tried to completely delete the whole filetree in target side when uplodaing and in the source side when downloading using rsyn... and same... |
|
| Back to top |
|
|
saccory
Apprentice
Joined: 18 Feb 2004
Posts: 176
Location: Göttingen, Germany
|
| Posted: Thu Jun 10, 2004 6:52 pm Post subject: |
|
|
Can you write manually to "/mnt/webs/www.canoeicf.com/slalom/committee/news/ECA/SKOPJE/K1WHT2.PDF"?
Maybe the logs on the target system can help you further. I'm out of ideas. |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Thu Jun 10, 2004 6:57 pm Post subject: yep |
|
|
Yes, I just tried this while you were writting your post hehe
Yeah it's a really strange problem man, I was searching over there in many forums and no luck yet...
Also I tried to directly scp the files that using rsync with ssh fail, and it works perfectly... It's only when using rsync...
You do what you can, and I really appreciate it, thx |
|
| Back to top |
|
|
Immortal Q
Apprentice
Joined: 14 Sep 2003
Posts: 241
Location: Silicone Valley
|
| Posted: Thu Jun 10, 2004 7:10 pm Post subject: |
|
|
For the "emerge sync" issues, try a different rsync mirror - one of the threads I was browsing mentioned that rsync is very lag-sensitive.
_________________
Osmos.org
Now with 20% fewer rabid primates. |
|
| Back to top |
|
|
Eamon Nerbonne
n00b
Joined: 16 Feb 2004
Posts: 45
|
| Posted: Sat Jul 03, 2004 5:33 pm Post subject: Similar problems here |
|
|
I'm also currently having rsync troubles (or rather; emerge sync - I've never used rsync for anything else). The symptoms seem to be identical:
| Code: |
ERROR: access denied to gentoo-portage from 34-239.surfsnel.dsl.internl.net (145.99.239.34)
rsync: connection unexpectedly closed (498 bytes read so far)
rsync error: error in rsync protocol data stream (code 12) at io.c(342)
|
I was wondering whether you had solved the rsync troubles in the mean while?
--Eamon
Edit: actually when looking at it like this they don't seem all that identical; however rsync dies all over the place during an emerge sync; not only like this... |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Sat Jul 03, 2004 10:55 pm Post subject: |
|
|
Hi Eamon,
Sorry, but I still couldn't fix this problem, I still have the same error when I try to rsync my website file-tree in different machines...
Anyway I realized that normally, when it's a io.c error, the common problem is when writting to the target file, I dunno why, but rsync cannot overwrite some files that you normally can do with cp or scp or any other tool...
I saw this strange behaviour because: if I delete the whole target file-tree, and then I run rsync (when before, several attempts failed), it goes perfectly... and after this perfect rsync operation, I repeat the same rsync command, and then I start again getting the same error once and again and again...
Anyway, that error that you mention like:
| Code: | | ERROR: access denied to gentoo-portage from 34-239.surfsnel.dsl.internl.net (145.99.239.34) |
Smells like a total different problem, did you try to use a different rsync public server for your portage tree updates? Try to use the command mirrorselect like the Gentoo-Install-HandBook tells:
| Code: | | mirrorselect -a -s4 -o | grep -ve '^Netselect' >> /mnt/gentoo/etc/make.conf |
(if you see that you don't have the mirrorselect tool, just emerge it like "emerge mirrorselect")
and after this, try to "emerge sync" again...
good luck |
|
| Back to top |
|
|
jruiz
Tux's lil' helper
Joined: 28 Apr 2004
Posts: 100
Location: Asturias
|
| Posted: Sat Jul 03, 2004 11:01 pm Post subject: |
|
|
Oh well, one important correction to my last post... 
When I tell you to use the command:
| Code: | mirrorselect -a -s4 -o | grep -ve '^Netselect' >> /mnt/gentoo/etc/make.conf
|
If you are not installing gentoo for the first time, just use this command instead:
| Code: | mirrorselect -a -s4 -o | grep -ve '^Netselect' >> /etc/make.conf
|
Sorry, I just copied/pasted the command and after sending my post, I realized my mistake  |
|
| Back to top |
|
|
Eamon Nerbonne
n00b
Joined: 16 Feb 2004
Posts: 45
|
| Posted: Sun Jul 04, 2004 9:31 am Post subject: |
|
|
Thanks a lot for such an extensive reply... wow!
The strange behaviour you referred to - namely that when you remove the portage tree and then re-sync it'll work... is behaviour I've seen too.
| Code: | total size is 71905617 speedup is 34.41
rsync error: some files could not be transferred (code 23) at main.c(1048)
>>> retry ... |
Another (very similar) machine (used as a router/fileserver) also running gentoo ~x86 with identical rsync mirrors doesn't have this problem - so unsurprisingly mirrorselect didn't help.
How can I safely remove my portage tree btw? Last time I removed my portage tree all kinds of things broke and I went through quite the hassle essentially reinstalling the portage - including wiping the list of installed packages which mean reemerging everything (fortunately I still had my world file and it all Just Worked).
Sometime this week I'll try debugging what's going on in more depth; if I find out what's happening I'll post it here. |
|
| Back to top |
|
|
|
Networking & Security
