May 2006 - gutvol-d - lists.pglaf.org

re: accurately converted to any possible format
by Bowerbird＠aol.com 25 May '06

25 May '06

greg said: > So you're against storing (or creating) WAP versions, > Braille versions and MP3 versions on our server? is that a serious question? i doubt it. but just in case, my answer is no, i'm not "against" that. indeed, i'd like to support all of that. in my offline app. i already have the option of text-to-speech saved as mp3, but you might be talking about human audio versions... (if you do mean computerized text-to-speech, it seems to me that that could be done offline too, but if you want to create, store, and serve such files, i have no objection.) as for braille, i don't have the slightest idea how to do that. and i'm almost as clueless when it comes to wap, sorry... but i'm sure z.m.l. is simple and straightforward enough that anyone who does know about these formats could write a conversion program that would do the job well... and i would retain my preference for offline converters. but i think it'd be _great_ to have online converters too! it's just that you'll need a web-programmer, i can't help, at least for now, because i've just done a little perl stuff. still, even my brief experimentation has demonstrated perl can do such conversions from z.m.l. quite easily. and i'd assume that python and ruby are just as facile. > That's cutting out a whole lot of > potential readers I would like to support. i don't think you should cut anyone out. > Nobody's stopping anyone from running a program > on their own computer to do whatever conversion they like. i should hope not. besides, it's unlikely they could prevent it. but all that is beside the point, isn't it? the point is to _enable_ people to do these conversions. right? > Why are you trying to stop me from enabling various conversions on > the server, for people who want or need to get conversion done there? i'm not trying to stop you. i'm cheering you on. go greg! but if you are telling me .tei/.xml is the best way (let alone the only way!) to do what you want to do, then we really should have a long discussion. -bowerbird

1 0

re: [gutvol-d] re: accurately converted to any possible format
by Bowerbird＠aol.com 25 May '06

25 May '06

greg said: > If it's not online and on the fly, then it's > not what I've been talking about. Sorry. your model and my model differ. my model calls for online maintenance of one file per book -- the master version -- in z.m.l. format. once a person has downloaded that master version, a program running on their personal machine is used to convert that master version into auxiliary versions. i can certainly port my source-code over to perl, so it would run on the web. but that's not what my model is. my model is to put the power into the hands of the user. i don't like methods that _require_ access to the internet. > You seem to be saying that there is exactly > one application in the world that can change > a ZML-formatted eBook into HTML, PDF and > a variety of other formats. one so far, yep. but i don't see any reason why a multitude of other programmers couldn't build their own programs that would do the same thing. how many such programs do you think people need? it is also of importance to keep in mind my main orientation, which is to provide a reader-program that is so superior that nobody even _wants_ to do a conversion to another format... realistically, i can't see anyone ever using my app to make a .pdf, because why would they want to use acrobat as their viewer-app? and they won't convert to .html to read the e-book in a _browser_, that's for sure. the only conversion i can see them doing is .html so they can put it on a rocketbook or one of the other handhelds, and those machines will all be on the scrapheap before too long... > If/when there is such an application that can > run on our Unix/Linux servers, operate on the fly, > and integrate with the Web back end, > it will be great to provide access to PG readers. i don't program much in any of the scripting web-languages, so you'd have better luck trying those three d.p. programmers that i pointed you to -- thundergnat, donovan, and bill flis. as a said, their tools are already doing the vast majority of the work involved in ascii-to-html conversions for post-processors. and it would be _great_ to standardize your .html versions. the one-off nature of your current .html versions means that all of them will have to be replaced eventually, which is gonna break the hearts of the post-processors who worked so hard on them and expected that hard work to last many decades... i would be happy to consult with anyone who wants to do an open-source version of these converters. i could certainly offer pseudo-code (and even realbasic code, if it helps) that would serve as a guide in programming routines. for the most part, however, i think the translation of z.m.l. structures to (x).h.t.m.l. should be rather obvious and totally straightforward. i surely have encountered no difficulties in doing exactly what is needed. examples of some .zml files with .html conversions can be seen here: > http://www.greatamericannovel.com/ahmmw/ahmmw.zml > http://www.greatamericannovel.com/ahmmw/ahmmwc001.html > http://www.greatamericannovel.com/mabie/mabie.zml > http://www.greatamericannovel.com/mabie/mabiep001.html > http://www.greatamericannovel.com/myant/myant.zml > http://www.greatamericannovel.com/myant/myantc001.html > http://www.greatamericannovel.com/sgfhb/sgfhb.zml > http://www.greatamericannovel.com/sgfhb/sgfhbc001.html > Where can users who might be interested > download your viewer app from? they can get a beta version by signing on to the beta-test listserve: > zml_talk-subscribe(a)yahoogroups.com that beta-version is very old, though, and doesn't have the converter routines that we have been talking about here... i'll be bringing the program out of beta in the next month or two; people will be able to download a copy from the z.m.l. website (new!): > http://www.z-m-l.com i now have one brave linux alpha-tester, so watch for a linux version! -bowerbird

2 1

re: [gutvol-d] Kevin Kelly in NYT on future of digital libraries
by Bowerbird＠aol.com 25 May '06

25 May '06

jon said: > Of course, what I just said is patently obvious. of course it is. -bowerbird

1 0

re: [gutvol-d] Kevin Kelly in NYT on future of digital libraries
by Bowerbird＠aol.com 25 May '06

25 May '06

michael said: > It's so simple that even Mr. Bowerbird's rhetoric > cannot confuse the issue: it's even so simple that mr. hart's rhetoric cannot confuse it. :+) > A picture of the pages of a book, even if complete and OCRable, > is simply not a a full text eBook. a picture, by definition, is not text. whether or not a _scan-set_ qualifies as an _e-book_, however, is more of a semantic issue than anything else. > 1. It takes many times the drive space. there's no question about that. and no need to reiterate it. > 2. It takes much more download wire time. there's no question about that. and no need to reiterate it. > 3. You can't do ANY of the things you can do with full text, except... > EXCEPT THE MOST IMPORTANT. . .YOU CAN READ IT. and here we have the most important concession, finally -- that a person can indeed _read_ an e-book that is a scan-set. so, for the person who _only_ wants to _read_ a book, a scan-set of that book is all that that person needs... nobody, least of all me, is going to argue with the position that digital text is _better_ than a scan-set in a multitude of ways... so if that's what you think this is about, michael, you're wrong. nobody, least of all me, is saying that people should _settle_ for a scan-set instead of digital text, especially for our cyberlibrary. so if that's what you think this is about, michael, you're wrong. > But the expense in time and money is much larger, i can figure out some interpretations of this that make sense, thinking along the lines of file-storage and bandwidth costs. but both of those things are cheap now, and getting cheaper. and a look at the whole picture shows that _scanning_ incurs the biggest cost, in terms of human labor and machine-costs. so it's a good thing a rich company like google is doing _that_. as for the next step -- the o.c.r. followed by the proofing -- that incurs more cost, mostly in the form of human labor costs. so -- actually -- it would _cost_ less to just "settle" for the scans. that would be a false economy, though, because the extra time that it takes to convert a scan-set into digital-text is _worth_it_ (i.e., the benefits of having digital text _and_ the scan-set are significantly greater than those of having _only_ the scan-set, and the increase in benefits is greater than the digitization costs, and this will become increasingly so as we automate the proofing.) so there's no question that we should keep doing the digitization. so if that's what you think this is about, michael, you're wrong. it is important to keep in mind, though, that we have no funds for doing this digitization, so we are relying on _volunteers_, and the number of volunteers we have now, and can reasonably anticipate having in the near future, is not even _close_ to being enough to keep up with the rate at which google is now scanning. and when google kicks up their rate, and the new scanning projects get going as well, and the ones currently operating cumulate their results, the number of undigitized scan-sets will become _huge_... so it's counterproductive -- to say the least -- to continue to foster some kind of unrealistic attitude about these scan-sets as being totally without value. we need to see they are of _immense_ value. to continue to throw rocks at them because "they can't be searched" or "you can't copy-and-paste their text" is silly to the point of stupid. people lived with paper-books -- which cannot be searched and from which you cannot copy-and-paste text -- for 500 years, man! and maybe it's good training for me as an anarchist to have my hero say something silly to the point of stupid. but it sure is disillusioning. the task before us now is to find a way to make use of the millions of scan-sets that are out there, standing in line, waiting to be digitized. > and it's much harder to write research papers. i'll let the researchers worry about writing their papers. after all, that's why they get paid the big bucks, right? > By Mr. Bowerbird's logic, a pre-Gutenerg book > would be as useful as a post-Gutenberg book. only if you try and twist my logic to say things that i don't. -bowerbird

1 0

re: [gutvol-d] Kevin Kelly in NYT on future of digital libraries
by Bowerbird＠aol.com 25 May '06

25 May '06

michael said: > Then again, some people > think pictures of pages are > as good as full text those are probably the people who want to "just" _read_ the words of the book, and don't want to copy out its text. > but that would entail a different definition. yes, what a pity that their "definition" is so constrained. when black ink is splashed onto a white page of paper, the result is nothing more than a "picture" of the book. but somehow, for over 500 years, that has been enough. -bowerbird

3 2

Re: !@!re: [gutvol-d] Kevin Kelly in NYT on future of digital libraries
by Joshua Hutchinson 25 May '06

25 May '06

> ----- Original Message ----- > From: "Greg Newby" <gbnewby(a)pglaf.org> > > Woah there, cowboy. > > I've been waiting for DP to provide raw page scans for *years*. This is > something I discussed with Charles & Juliet years ago. The whitewashers > are ready. iBiblio is ready. > Have we ironed out HOW the images should be organized? I remember Marcello put forth a pretty good proposal for image naming/organization, but the response was luke-warm at best. I don't have nearly the backlog of images available that Jon does (the man is a scanning machine) ... but I'd be happy to start submitting images with my books from here on out if we've got the organization worked out. Josh

1 0

Subject: Kingkong was: [gweekly] PT1b Weekly Project Gutenberg Newsletter
by Dave Fawthrop 25 May '06

25 May '06

On Wed, 24 May 2006 09:50:50 -0700 (PDT), Michael Hart <hart(a)pglaf.org> wrote: |General Catalog of Old Books and Authors Including Dates of Death.!!!!! For the rest of the world on Life+70 or Life+50, i.e. non USA, date of death is crucial for copyright. |http://www.kingkong.demon.co.uk/ngcoba/ngcoba.htm | |which now indexes 24,000 books available free online, including all |PG(US) & PG(Aus)'s books, along with some basic date information |about them and their authors where you can find more. IME a great site. He missed a couple of ?my? authors and added them quickly when I informed him. Check that he has your favourite authors. -- Dave Fawthrop <dave hyphenologist co uk> "Intelligent Design?" my knees say *not*. "Intelligent Design?" my back says *not*. More like "Incompetent design". Sig (C) Copyright Public Domain

1 0

re: accurately converted to any possible format
by Bowerbird＠aol.com 25 May '06

25 May '06

greg said: > Not all input books or types can be reasonably > accurately converted to any possible format, especially > for the older titles with no well-formed & valid HTML version. as i've said for years now, with a small commitment from you to consistent formatting, i could take plain-ascii files as input, automatically apply the typographic niceties that are expected, and output the results to .pdf and to .html, such that the .html can be converted to a large number of other auxiliary formats. of course, i'm not unique. david moynihan has done it for years. david was willing to make a small commitment to edit the files himself so as to obtain that consistent formatting. i think it is more important to teach you how to fish than to give you fish. check with 3 tool-makers from distributed proofreaders -- thundergnat, donovan, and bill flis -- and they'll confirm that a clear path for ascii-to-(x)html conversion is quite workable -- due to the fact that d.p. now has the required consistency -- even with their current programs, and that if they worked on it a bit more, they could make it into a regular part of the workflow. there is no need for the more-complex switch to a .tei workflow. -bowerbird p.s. if you only would have accepted moynihan's offer of his files when he made it to you, you'd already _have_ a consistent library.

3 3

Re: [gutvol-d] re: accurately converted to any possible format
by Bowerbird＠aol.com 25 May '06

25 May '06

i said: > > as i've said for years now, with a small commitment from you > > to consistent formatting, i could take plain-ascii files as input, > > automatically apply the typographic niceties that are expected, > > and output the results to .pdf and to .html, such that the .html > > can be converted to a large number of other auxiliary formats. greg said: > Here's an eBook that should meet your requirements: > http://www.gutenberg.org/etext/18257 well yes, that one works just fine. already did it for that one. :+) > You already have server space, yes, thank you very much, snowy has been very kind to me... > to provide a conversion utility. > Looking forward the pudding... well, but first there is that minor matter of the small commitment to ensure that all future files will conform to consistency as well... you know, first you plug the leak, and only then clean up the mess. and that's not a new request. that was the original precondition. but perhaps there's another way out of this impasse. > To each their own pudding. If I'm not saying "no" to you, > why would I say, "no" to someone with a different approach? it would be silly of me to expect _you_ to tell them "no", as if they couldn't bloody well do it themselves anyway. no, what _i_ am doing is telling them that .tei would be a waste of their time. i'm giving them a friendly notice, a heads-up, saying "hey i got it covered, you go on", but they don't take it that way, they get all pissed off at me... so, ok, you want to be gruff, i can be gruff, no problem. oh yes, greg, _you_ are one of "them". so you would be telling yourself "no". and i don't expect you to do that, no. indeed, i don't even expect you to listen to me telling you that .tei is a waste of your time. that's fine. you'll learn. > I have two other servers (snowy.arsc.alaska.edu and readingroo.ms) > with complete copies of the PG collection for development. ok, now we're getting somewhere. this could break through. > If people don't like the way PG postprocesses & posts eBooks, > they can grab 'em and do their own postings. As long as there's > a reasonable adherance to the principle of unlimited distribution > etc. (http://www.gutenberg.org/about) we'll even link to 'em! i have no stealaway desires, i'm happy to do things under your wing, my intention is to try and show you -- project gutenberg -- how you can save yourself a lot of work, and increase your unlimited distribution. i want to help the best cyberlibrary get better, i don't want to tear it apart. i would definitely agree to demonstrate some automatic transformations of your e-texts on a library-wide basis that could show you some shit... at first this would just be for experimental purposes. no promises. however -- if the effort continues, and it should ever come to that -- my "shaping" of the files by progressive transformations would result in a substantial fork of the library, but once you have satisfied yourself that it had retained all important data so the integrity of the books was intact and that only inconsequential inconsistencies had been removed, you will be amenable to _considering_ a wholesale replacement in one fell swoop. the benefits will be quite obvious, though. it won't be hard for you to be the decider... > I don't recall turning down David, but might have on the grounds of > being unable to effectively ingest & manage the files he was producing. > Today, I'd offer him his own server space to help him do things his way. ok, now you're _really_ talking. because david already knows how to do this. unfortunately, as we all know, he's kinda busy right now. with any luck, though, he's bored, and restless because the website that has been his business has been his life has taken up a lot of his time in the last many years or so is shut down, with any luck he's itching for some diskspace to play with. he might be jonesing for an ftp-interaction... but you know, honestly, all _i_ would really need from him is his ascii files. he's already made all of them consistent... i've never asked him for them before. but maybe now's the time? and if you and i ask him together, so i can work on the files for you? i've never seen his ascii files for sale, but i'd be willing to pay some. after all, the reason i want those files is they'd save me a lot of time. it seems quite reasonable to reimburse him for a little of that time. especially because he's got lawyer bills, i'm sure. (actually, i'd hope a lawyer has taken this case pro bono for the great exposure, but you know there are always lawsuit-related bills that have to be paid. i consider him a trooper, and i do believe in supporting our troops!) anyway... with _consistent_ files, i can start turning neat tricks right soon now. i might have to reshape david-consistency into zml-consistency, but that'll be a lot easier than reshaping p.g.-inconsistency into anything. so even if it's not immediate, it would be soon. of course, i'm not taking david's files for granted, no sir, as we haven't even yet asked. and he _is_ busy these days. he might even say yes, but have no time to fill the request. but, to sum up, i would be most grateful for: 1. on snowy, a copy of the p.g. library that i can start "shaping". 2. also on snowy, a copy of moynihan's ascii files for experiments? 3. diskspace for david to play, now if he wants, or sometime later. the understanding is i'm just playing with your files. no promises, no expectations, no guarantees. ok? -bowerbird

2 1

Re: [gutvol-d] Kevin Kelly in NYT on future of digital libraries
by Joshua Hutchinson 24 May '06

24 May '06

> ----- Original Message ----- > From: "Michael Hart" <hart(a)pglaf.org> > On Wed, 24 May 2006, Michael Dyck wrote: > > > However, if it makes things easier, I'll gladly refuse to acknowlege > > that statement *now*. A linear growth curve *is* a growth curve, and a > > pretty ordinary one at that. It certainly doesn't make it impossible to > > create an ordinary growth curve. But that's just disagreeing with what > > you said, rather than what you meant. > > No, a line is not a curve. > Here we go again. 1 - A growth curve doesn't necessarily curve nor does it necessarily grow. A growth curve can be flat (no change), it can be negative (lose value like my stocks recently), it can be linear (grows at a constant rate) or it can be exponential (what we typically think of as a growth curve). 2 - Just because you don't WANT to include certain periods of PG history in your growth curve analysis doesn't mean no one else can or that they are wrong for doing so. If someone wants to plot growth from the start of PG to present (which certainly seems reasonable to me), then that is a valid growth curve plot. If you want to plot it in the "modern PG era" from circa 1993 to present, then that is valid, too. Both plots need to specify the time frame they are plotting. Since we argued this to death already (as Michael Dyck so nicely linked to previously), can we let it drop now? Josh

1 0