- gutvol-d - lists.pglaf.org

Re: !@!!@!!@!Re: Re: so what is so important about pagination?
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

michael, i wish you would've taken the time to read what i actually wrote, instead of just giving your kneejerk reaction. because your response doesn't address the point that i made. i am loathe to get into this argument, because it won't mean a thing in the long run... the future will have its own issues, and it will need to deal with them, and i laid it all out clearly. but let me just address a few things, to provide some clarity. > he fails to talk about the wide variety of Jane Austen's > p-books, and that paginations run rampant among them, > not to mention margination, spelling, etc. there are different editions of many books, to be sure... and i count each edition as a separate book. your e-book will have to mimic _one_ of the editions in a faithful manner, or it will be discarded... notice that when i say "mimic", i do _not_ mean that it has to match it _exactly_. so, for instance, if you wanna close up spacey contractions, or correct spelling, or make other kinds of changes, they might (or might not) be totally acceptable to any one specific end-user in the future... but you _will_ have to make it easy for that specific end-user to _compare_ your e-book with a p-book, in order to spot changes. i've shown how this comparison is done, by mounting a web-page which has the text on one side of the screen, the scan on the other. but if your e-book is a formless blob, that's not gonna cut it... and, for the record, i'm most certainly _not_ recommending that we create some "scholarly" version of our books. i laugh at that. the _only_ thing we know about the scholars of the future is that we do _not_ know what they will want and it'd be foolish to guess. put yourself in the shoes of the future. you have a dozen different e-book files, all purporting to be copies of "sense and sensibility". you know that some of them have been doctored, and others have been bowdlerized, and you _hope_ that some of them are accurate. you can, with some work, find the differences between them, but you'd prefer not to have to go through that exercise if you could, because you'd have to then do more work to find the _right_ copy. so, how do you proceed? well, i can tell you that if _one_ of those copies made it _simple_ for you to verify its accuracy by assuming the form of the p-book, that will be your obvious first choice. think about it. you'll agree. so, michael, if you want to respond to this, answer that question. -bowerbird

2 1

Re: so what is so important about pagination?
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

as i said earlier, there's no real need to provide "justification" for pagination. the fact that _some_ people want it is enough to make us decide that we shouldn't toss out that information. however, since nobody has mentioned the _best_ justification for including pagination information, i might as well tell you... here at the end of the paper-book half-millenium, we have roughly 10 million different books out there in the world... (this is according to my memory of recent figures, which may be off, perhaps even by a large amount, but that's immaterial.) if we figure there are an average of 1,000 copies of each book, that means we've got about 10 billion copies of paper-books... that's a lot of paper-books out there in the world. a whole lot. those paper copies are the _originals_, and they always will be. in the future -- even right now, thanks to google -- we have a virtually unlimited number of digital copies of those originals. but again, those digital versions will _always_ be "the copies"... and the paper-books will _always_ be "the originals"... forever. (even books that're "born digital" often become physical quickly, and that will continue into the far future with print-on-demand; and paper-books, due to their _physical_and_material_ nature, will always be the "real" books, while digital versions will always be the "copies", especially since they can be manipulated at will, while physical books have the virtue/liability of being "frozen".) "real" doesn't mean "more valuable" or "more important", it means _physical_ and _tangible_ and _visible_ and _made_out_of_atoms_. you really have to ground yourself in this thinking to understand -- _physical_ books are the "real" ones; digital books are "copies". that's our first important factor... and our second important factor is that e-books are manipulatable. and just as the frozen nature of p-books is both virtue and liability, so too is this manipulability. on the one hand, it's easy to fix errors, provide updates, and so on and so forth... but, on the other hand, it's also easy to alter the book in a way the author did not intend... and if you don't think people _will_ try to rewrite history, you're nuts. plus there's just sheer incompetence, which has already resulted in a number of very shoddy digitizations of books, full of inaccuracies. just try and find all the copies of "pride and prejudice" out there, and then do a determination on which ones are "accurate" and which not. you will find this task to be overwhelming, and nearly impossible, and that's just one book out of our 10 million books. that is the problem. so there's little question that people in the future will be _skeptical_ about each and every e-book which they are handed, and rightly so... for reasons from accidental to quite intentional, it might be inaccurate. so we have a state where there are some "known" p-book "originals", and a ton of digital "copies" that might or might not be "trustworthy". (i believe jon noring has been absent from here for long enough that it's once again safe to use that word without all his derogatory spin.) now, there's only one solution to this state. any specific digital copy will have to be able to _prove_ its correspondence to a paper-copy... the easiest way to provide such proof is to assume the same form as the paper-copy; that is, it must adopt the linebreaks and pagination, so that each and every page can be subjected to visual confirmation... of course, in order to have value as a digital book, the file must be able to drop the linebreaks/pagination, and assume another form, one that reflows to the current set of desires of the end-user, _but_ it _must_ be able to mimic the look-and-feel of the paper-book too. if it cannot, it's simply going to be discarded as being untrustworthy. your e-book cannot afford to be nothing more than a formless blob. it _must_ be able to "snap to" a form that exactly imitates the p-book. and for it to be able to do that, you must keep linebreaks/pagination. it's really that simple. -bowerbird

2 1

hallelujah
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

hallelujah! some people are finally talking some sense into rfrank's head. one person suggested some reg-ex tests be shown to proofers. all by itself, this is an improvement, but not that big, because these tests should be done _before_ the text goes to proofers. however, what it did was it jolted roger out of his thinking that such tests are done in _postprocessing_, a huge ideological shift. roger admitted as much. thank you lord! another person pointed out that a global search-and-replace would be a real asset. d'uh, who's been saying that for _years_? one person said: > I've noticed "Pem" scanned as "Pern" a few times. roger responded with: > Done. Fifty-seven replacements. yes! see how easy this can be? so roger said, "tell me what kind of global changes you'd make". so one person came back and said, "how about things like these:" > change did n't to didn't > change could n't to couldn't kinda hard to believe that those fixes weren't already being made in preprocessing, isn't it? but hey, let's be glad for the progress... sure enough, roger got the hint, and changed all of the floating contractions, and then promised he'd do that in _preprocessing_ next time. hallelujah! now is the time to give roger that list of 30 tests that i highlighted back in that month-long series i did... now the _next_ thing would be for someone to _volunteer_ to do all of this preprocessing for roger, using the tool dkretz coded... hallelujah! -bowerbird

1 0

Re: "The Inheritance" by Susan Edmondstone Ferrier
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

al said: > Hmmm... maybe bowerbird is up for them? what's my motivation, al? the one edition i looked at -- in english, i speak no german -- is a simple book, quite straightforward, so wouldn't prove anything. i mean, i'm totally willing to run through the exercise-wheel, but what do i get when i come out the other side? how about this, a win-win-win-win situation for everyone... for a while, the p.g. website has been directing newbies over to distributed proofreaders if they want to help out with the cause. and sure enough, d.p. gets a ton of volunteers as a result of that. unfortunately, d.p. doesn't appreciate the newbies, because they just contribute more stuff to the plethora of p1-proofed backlog. so how about, if i were to do this book for you, p.g. would start sending all the new volunteers to rfrank's roundless site instead? d.p. happy, rfrank happy, the bird happy, al happy, and p.g. happy. so, do we have a deal? -bowerbird

3 2

Re: so what is so important about pagination?
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

keith said: > I am not talking markdown or Restructured. > I am talking about a true markup langauge. it appears you don't really know what you're talking about, as both markdown and restructured _are_ "true markup languages". you want to invent a new one. fine. go ahead. i did it myself... -bowerbird p.s. don, restructured text is older than markdown, and textile too, as far as i know... it's a reworking of "structured text", which is the granddaddy of all the light markup languages. and -- by the way -- z.m.l. is older than markdown. indeed, one of the few appearances of charlz on this listserve was when he came to announce markdown as a z.m.l. clone...

2 1

[SPAM] re: Re: so what is so important about pagination?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

jim said: > I dislike anything that prevents reflow, > which I think is necessary for the enjoyment of most users. there is absolutely nothing about retaining pagination (or linebreaks, or end-line hyphenates) that "prevents reflow", jim, and i wish you would stop repeating that nonsense. you need to pay better attention. -bowerbird

5 8

[SPAM] re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

michael said: > Let's just forget the whole idea of error free texts. . . . well, that's going a bit overboard. better, let's try to achieve perfection, but let's not let that high goal get in the way of making work available even if it's not yet perfect... > I would prefer just to get these materials in circulation-- > then worry about approaching perfection along with Xeno. yes, except i don't see any part of project gutenberg that is doing very much at all in the way of "approaching perfection". once a text is posted, people seem to forget it completely... even when the whitewashers "correct" an e-text, they aren't doing nearly what they could be doing in order to improve it. it seems like there is a constant mad dash to do new books, but almost nothing is being done to fix up any older books. i asked at 10,000 books for a review in terms of quality control, and again at 15,000, and again at 20,000, and again at 25,000. i didn't bother to ask again at 30,000, because what's the use? but at some point, some hard questions will need to be asked... > Does anybody have a serious objection to putting the 8,000, > or so, books that were listed earlier as being in limbo, in > something like our "PrePrints" section, where we put eBooks > that are admittedly not ready for prime time??? well, i certainly don't... but many of the volunteers over at distributed proofreaders do. indeed, according to a poll (which has now received one of the highest number of votes on any poll that has been done there), there are split evenly -- right down the middle -- on this issue. i don't know what to make of that. but that's the way it is. it's also worth mentioning that those 8,000 books are _not_ "almost done". some of them really aren't even very close... some are full of typos, still. most contain pseudo-markup, which really should be converted to something more useful before the books are ever put in front of the general public. lots contain "proofer's notes", which would confuse people. it should also be noted that many of them are not in english, which might (or might not) have bearing on the question, but since i only speak english, i wouldn't have any idea what it is. considering all this, it would _not_ be a simple procedure to free up this matter. it could use up a lot of time and energy, and for very little benefit in return. (does any use "preprints"?) what _would_ be useful is for this material to be put on a wiki, in order to test notions of public postprocessing collaboration. instead of saying "here, take this unfinished work", we _should_ instead be saying "here, come help finish this unfinished work". at one point, i was tempted to build such a postprocessing system. but then i realized i didn't want to help d.p. get over their backlog; d.p. deserves to suffer the consequences of their terrible workflow, or they'll _never_ be motivated to fix it... so i decided to let it be... -bowerbird

2 1

Re: ok, let's take a look at gardner's book, just for the exercise
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

i've merged gardner's corrections into the archive.org o.c.r., and posted the results on my website. here's a sample url: > http://z-m-l.com/go/gardn/gardnp123.html gardner dehyphenated end-line hyphenates, so i rejoined (some of) them. i'll write another routine to do the rest... out of a file that contains about 4,000 lines, there are only 800 (at this point) which differ in the two versions. so even early in the merge, 80% of the o.c.r. lines were right, and that number will increase with more aggressive cleaning. the (presumably incorrect) lines from the o.c.r. are in red, while the lines from gardner's proofed copy are in blue... if you prefer to view all of the edits on one web-page, see: > http://z-m-l.com/go/gardn/gardn-hybrid.html -bowerbird

1 0

Re: so what is so important about pagination?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

gardner said: > I'm going to take this as a jumping off point > for a more general question about > whether pagination of a published edition, > is worth saving. Obviously there is a range of opinion. > I'll give you mine. yes, there is a range of opinion. i can give arguments -- even good arguments -- on both sides. which obviously means that _some_ people have good arguments for retaining pagination. and you disenfranchise those people entirely when you throw out the pagination, no matter how good your intentions might be for doing so. i'd rather not disenfranchise those people... so i think it's necessary to include the pagination, and the original linebreaks, with end-line hyphenates. now, i think it's _imperative_ that we give people tools that enable them to discard that pagination, and unwrap those original linebreaks, and rejoin end-line hyphenates. to do otherwise would be to disenfranchise _those_ people, and i'd rather not disenfranchise them either. so, for me, the answer to the question is extremely simple. > What I believe, philosophically, I am shooting for > is to capture the core content, and reject the details > that have mainly to do with the medium of publication. i can understand that perspective. i can also understand the other perspective. and i see no reason that anyone has to be unhappy here. it's very important to understand that this does _not_ have to be an either/or question. we _can_ do _both_... > Maybe there are possible future uses of my text > that would want the things that I left out. > I tend to doubt that this could ever be very important. well, then, your imagination is starting to lag behind... :+) because we are now right in the middle of a situation here where "the things that you left out" _are_ "very important", namely a reproofing of your book, to test your accuracy... it's an order of magnitude more difficult to proof a book when the text has lost all of its linebreaks and pagination. are you of the opinion that the future will simply _accept_ that you did a perfect job in the digitization of your books? or do you think they will want to verify the quality of them? if you make it too difficult for them to undertake that job, they will just toss out your text and start anew. your loss... > As an individual contributor I do not feel that my time > is best spent capturing and encoding that, and so I don't. except the info was already captured. then you threw it away. > And I am happy that PG finds my efforts acceptable > despite this deficiency. except the future will throw out all the d.p. works because your deficiency is shared by the entire d.p. corpus, sadly... (even the d.p. people who save pagination toss linebreaks.) > I haven't done any sort of real research, but > a quick look shows me that > not many texts attempt to preserve line endings in any way. > Preserving line endings seems quite unpopular. the future needs to future-proof tens of millions of books... they can afford to throw out everything done up to this point, if they feel they need to, and they will, they most certainly will. (and advances in o.c.r. and o.c.r. correction will make it easy.) (well, as i pointed out, they might use some of the current texts to proof the new o.c.r. that they do, but then they'll toss them.) > Bowerbird wants to keep both actually, i don't need to take a personal stand on the issue, not as an end-user. and that's a good thing, because often i don't have a need for the original linebreaks or pagination. so i definitely want the option of discarding that information. what i am saying is that, as a "best practice" for digitization, the discarding of such information is clearly a terrible mistake. and if you're doing it simply because _you_ don't see a need for that information, then you're being selfish and shortsighted. plus you're giving an ultimatum to people who want that info: they're forced to toss your text as failing to meet their needs. and i am positive that you're going to lose that bet, gardner... -bowerbird p.s. by the way, i found 23 discrepancies in the paragraphing between your version of "the advocate" and archive.org's o.c.r. however, 21 were errors in the o.c.r., and only 2 in your book. the two errors in your book, both of them missed paragraphs: > http://z-m-l.com/go/gardn/gardnp087.html > "What, take the bird back to the bush where we > http://z-m-l.com/go/gardn/gardnp101.html > Yet who shall blame the sun and moon for that?

2 1

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

postprocessing at distributed proofreaders is difficult only because the proofers are instructed to throw out meaningful data, which later then needs to be replaced, and the formatters insert obtrusive pseudo-markup, much of which later needs to be reworked or deleted. if the proofers used nonobtrusive zen markup instead, it wouldn't interfere with their proofing task, and there wouldn't need to be a separate formatting task, even if (in reality) people decided to specialize on that aspect. also, the conversion of proofed/formatted pages into a full-on electronic-book should be an automatic process. i've already demonstrated this many times, but would be happy to do it once again, on any book of your choice... this point is particularly important in a roundless system, where the object is to move a page to a "finished" status as quickly as possible. -bowerbird

2 1