Re: [gutvol-d] ftp.archive.org
My understanding of rsync is that you had to mirror the entire PG archive. That was based on the PG FAQ and my attempts to read the help and man page. I couldn't figure out the command line options and experiments I tried gave me errors. I think the PG FAQ gives a sample command line but that's for everything which isn't what I want. Besides it's nice to manually look at and download each file. I often like to stop every few files and look at a book of interest. So, to answer your question, all of rsync confuses me since I never got it to work. Also, another problem might be that I'm primarily on Windows. I know rsync is common in Linux and I have it installed on the Debian server that I run but I'm not sure if it's available for Windows or not. I have Cygwin so I might have it, but again I have no idea how to get it to only get the files I want. That's nice about getting them manually, I can skip those I don't want as I see them in the newsletters. At 07:10 AM 6/28/06 -0400, you wrote:
I'm sorry... what? You can rsync exactly what files you wish, recursively or not, pick and choose, with rsync... using the right options. I mirror Gutenberg here with rsync, skipping the DVD files, .mp3 files, .rar files and a few others, getting only the useful copies of books.
What part of rsync's usage is confusing you?
-- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.1.394 / Virus Database: 268.9.7/379 - Release Date: 6/29/06
Rsync's available for Windows as part of the cygwin package. Just like FTP or wget you can tell rsync to get only the stuff you want. and unlike FTP or wget it will only download the files that need updating, without you having to wait several hours for it to skip over every file that hasn't changed. I admit it can be confusing since it's a very powerful too. I was talking about it with Aaron Cannon and he says it's a better way to make a "mirror" of PG (with or without specific files that you want. My suggestion for people who want to use rsync? Have someone write a more detailed FAQ on it, explain it in non-technical terms, and provide some examples (using the PG archive) of commands you can run with it, especially sample rsync lines like David has, explaining all the '-' tags and what they mean in context with the line and what they will make rsync do to the files you download/mirror. Jared Tony Baechler wrote on 30/06/2006, 12:58 AM:
My understanding of rsync is that you had to mirror the entire PG archive. That was based on the PG FAQ and my attempts to read the help and man page. I couldn't figure out the command line options and experiments I tried gave me errors. I think the PG FAQ gives a sample command line but that's for everything which isn't what I want. Besides it's nice to manually look at and download each file. I often like to stop every few files and look at a book of interest. So, to answer your question, all of rsync confuses me since I never got it to work.
Also, another problem might be that I'm primarily on Windows. I know rsync is common in Linux and I have it installed on the Debian server that I run but I'm not sure if it's available for Windows or not. I have Cygwin so I might have it, but again I have no idea how to get it to only get the files I want. That's nice about getting them manually, I can skip those I don't want as I see them in the newsletters.
At 07:10 AM 6/28/06 -0400, you wrote:
I'm sorry... what? You can rsync exactly what files you wish, recursively or not, pick and choose, with rsync... using the right options. I mirror Gutenberg here with rsync, skipping the DVD files, .mp3 files, .rar files and a few others, getting only the useful copies of books.
What part of rsync's usage is confusing you?
-- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.1.394 / Virus Database: 268.9.7/379 - Release Date: 6/29/06
_______________________________________________ gutvol-d mailing list gutvol-d@lists.pglaf.org http://lists.pglaf.org/listinfo.cgi/gutvol-d
-- . .:. .:::. .:::::. ***.:::::::.*** *******.:::::::::.******* Dmitri Yalovsky ********.:::::::::::.******** ********.:::::::::::::.******** USS Authority *******.::::::'***`::::.******* ******.::::'*********`::.****** Asst. Chief of Engineering ****.:::'*************`:.**** *.::'*****************`.* .:' *************** . .
My suggestion for people who want to use rsync? Have someone write a more detailed FAQ on it, explain it in non-technical terms, and provide some examples (using the PG archive) of commands you can run with it, especially sample rsync lines like David has, explaining all the '-' tags and what they mean in context with the line and what they will make rsync do to the files you download/mirror.
How about using Unison? http://www.cis.upenn.edu/~bcpierce/unison/ David A. Desrosiers desrod@gnu-designs.com http://gnu-designs.com
Hello list. I really appreciate the help with rsync, especially David's sample command. However, I don't think people understand what I want. I do not want a full mirror of the PG archive. I do have a partial mirror but it's very specialized. I don't follow the standard PG directory structure. I only download English books in plain text. I don't want html, non-English, or 8-bit. I have books divided up into 1,000 per directory. For example, my etext18 contains files 18000-18999 etc. For reposts, etext0\ contains books 0-999. This is completely different from the PG structure. Once I download a file, I don't have any reason to retrieve it again unless it gets updated. If rsync will grab all the files I need and put them in my customized structure, rsync will help. However, based on my reading of the help, it doesn't do that. My apologies for the Windows comment. I found it and it's part of cygwin as written below. The syntax actually isn't as bad as I thought. It reminds me of wget which I use frequently. It would be perfect for mirroring a primary server to a backup machine. Again, I'm not trying to mirror here so it doesn't do what I want based on my understanding of comments made on this list and the rsync help. At 04:37 PM 6/30/06 -0700, you wrote:
Rsync's available for Windows as part of the cygwin package. Just like FTP or wget you can tell rsync to get only the stuff you want. and unlike FTP or wget it will only download the files that need updating, without you having to wait several hours for it to skip over every file that hasn't changed.
I admit it can be confusing since it's a very powerful too. I was talking about it with Aaron Cannon and he says it's a better way to make a "mirror" of PG (with or without specific files that you want.
-- No virus found in this outgoing message. Checked by AVG Anti-Virus. Version: 7.1.394 / Virus Database: 268.9.8/380 - Release Date: 6/30/06
On Sun, 2006-07-02 at 23:58 -0700, Tony Baechler wrote:
If rsync will grab all the files I need and put them in my customized structure, rsync will help. However, based on my reading of the help, it doesn't do that.
In fact, it does exactly that, depending on how you tell it where to put the files its copying over. You may have to script a bit of it or run several rsync commands in a series to get what you want (fetch text first, indices next and so on). rsync -avSP --delete *[0-9].txt /my/custom/directory rsync -avSP --delete *.gz /other/place ...and so on. I've been using rsync for MANY years now, and tridge is one of the alumnus' of a previous company we both worked for (Linuxcare), so I can say with absolute-certainty, that if you're running into trouble with what rsync does for you, you're doing something wrong ;) If there's one thing rsync does well, its everything. There are even people out there who use rsync _exclusively_ as their MTA/MDA. Nutty, but true.
It would be perfect for mirroring a primary server to a backup machine. Again, I'm not trying to mirror here so it doesn't do what I want based on my understanding of comments made on this list and the rsync help.
It can do a lot of things, incremental backups, snapshots, mirroring, cloning of directories, complete transposition... pretty much anything you want. It just takes a remote file, block-copies it to some local place (or local to local, if you're cloning a drive for example. I've used rsync quite a bit to upgrade hard drives in laptops, works great). In any case, just define your schema and apply the rsync methodology to it. No need to get complicated or fancy. Oh, and lastly... rsync does NOT have to have the same version running on both ends. If that were true, it would break in thousands of situations. You simply have to have a version which understands the options you're passing it (i.e. rsync v1.x isn't future-compatible with v2.6.6). I can't speak for the Windows <-> Linux syncronization, but it should be moot. You don't need to run rsyncd to rsync files from machine to machine either, but you can if you wish. Good luck! -- David A. Desrosiers desrod gnu-designs com http://gnu-designs.com "Erosion of civil liberties... is a threat to national security."
participants (3)
-
David A. Desrosiers -
Jared Buck -
Tony Baechler