![]() ![]() The source file that is affected by the above change (primarily) is While I’ve not done extensive testing, I’ve used it enough to believe it is working as designed/intended. socketpair(), which is enough for rsync to run using ssh, just as it does on UNIX. Not defining HAVE_SOCKETPAIR in the make configuration is enough to use pipe() vs. The ssh process never even gets as far as attempting network access and rsync The ssh process is detached, and apparently not diretly connected to rsync, as it is an orphan, owned by process 1. # rsync version compiled from hang is occuring when rsync is attempting to exchange protocol version numbers, it writes its version and then hangs waiting endlessly for a reply. # by cygwin setup.exe, we are still using the good ![]() # This way, even if /bin/rsync gets overwritten # (/usr/local/bin/rsync.exe) in your backup scripts. # at /bin/rsync use the compiled executable # Instead of overwriting the original executable # Now undefine HAVE_SOCKETPAIR in config.h You do not need to run rsync as a daemon.Ĭompile rsync from source and use pipes instead of socketpair. This entry was posted in Windows by Marc. Interestingly, neither the rsync FAQ nor the Cygwin FAQ mention this problem. So I ended up modifying the nf to use cygwin paths instead of Windows paths and then used the regular cygwin rsync.exe. I did run into a problem with that though because the rsync.exe in that package conflicted with my installed Cygwin ( cygwin1.dll versioning problems I suspect). I actually already had one because I had previously installed a modified Cygwin rsyncd package from the backuppc project. desc "Starts a rsync daemon for accepting incoming rsync connections" You can set up an rsync daemon as a Windows service using a command like this: cygrunsrv -install "rsyncd" -path /usr/bin/rsync -args "-daemon -no-detach" Well, I tried it and yes, running an rsync in daemon mode on the “server” completely solved the problem. Probably the most helpful page I came across was this page (which I actually found after I had fixed the problem, while looking for a good link for this post). Searching around on the Web, I found a lot of people complaining about Cygwin rsync hanging and not a ton of helpful information, but a few bits here and there that suggested that the problem was limited to using rsync over ssh and that rsync to a rsync daemon works better. Restarting the rsync would copy a few more files and then hang again. Invariably, the rsync would copy a handful of files and then hang. The rsync is running on a node with 128 cores and 256GB of RAM so I'm not resource limited.ĭoes anyone know if this is a problem with rsync? Are there better options I can use to get around this? I'm not seeing errors in the rsync log when it hangs - it sits there for ~7 hours not doing much of anything.I was using Cygwin rsync on our two Windows laptops to copy our files over from the old laptop to the new laptop. Note, both file systems are lustre running over ZFS. There is more than enough space on the target file system and the source file system doesn't have any faults. This is a problem as we are moving a large amount of data (100s of TB) from one data storage system to another. ![]() Then it starts up again, moves another 10k to 20k and hangs for another 7 hours. The process moves along pretty well until it has transferred around 100k files and then it hangs for around 7 hours. This file list corresponds to around 250k files from a single directory that has 500k files in it (I know, it's insane but that's how this genomics application spits out its results). Currently I'm using the following rsync -a -s -H -logfile=./rsync-logfile_1460 -files-from=./f.1460 /source/path /dest/path This option lets me pass rsync a pregenerated list of files I want to transfer. I'm running into a persistent problem with rsync when using the file-from option.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |