- gutvol-d - lists.pglaf.org

Re: "The Inheritance" by Susan Edmondstone Ferrier
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

al said: > Hmmm... maybe bowerbird is up for them? what's my motivation, al? the one edition i looked at -- in english, i speak no german -- is a simple book, quite straightforward, so wouldn't prove anything. i mean, i'm totally willing to run through the exercise-wheel, but what do i get when i come out the other side? how about this, a win-win-win-win situation for everyone... for a while, the p.g. website has been directing newbies over to distributed proofreaders if they want to help out with the cause. and sure enough, d.p. gets a ton of volunteers as a result of that. unfortunately, d.p. doesn't appreciate the newbies, because they just contribute more stuff to the plethora of p1-proofed backlog. so how about, if i were to do this book for you, p.g. would start sending all the new volunteers to rfrank's roundless site instead? d.p. happy, rfrank happy, the bird happy, al happy, and p.g. happy. so, do we have a deal? -bowerbird

3 2

Re: so what is so important about pagination?
by Bowerbird＠aol.com 22 Feb '10

22 Feb '10

keith said: > I am not talking markdown or Restructured. > I am talking about a true markup langauge. it appears you don't really know what you're talking about, as both markdown and restructured _are_ "true markup languages". you want to invent a new one. fine. go ahead. i did it myself... -bowerbird p.s. don, restructured text is older than markdown, and textile too, as far as i know... it's a reworking of "structured text", which is the granddaddy of all the light markup languages. and -- by the way -- z.m.l. is older than markdown. indeed, one of the few appearances of charlz on this listserve was when he came to announce markdown as a z.m.l. clone...

2 1

[SPAM] re: Re: so what is so important about pagination?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

jim said: > I dislike anything that prevents reflow, > which I think is necessary for the enjoyment of most users. there is absolutely nothing about retaining pagination (or linebreaks, or end-line hyphenates) that "prevents reflow", jim, and i wish you would stop repeating that nonsense. you need to pay better attention. -bowerbird

5 8

[SPAM] re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

michael said: > Let's just forget the whole idea of error free texts. . . . well, that's going a bit overboard. better, let's try to achieve perfection, but let's not let that high goal get in the way of making work available even if it's not yet perfect... > I would prefer just to get these materials in circulation-- > then worry about approaching perfection along with Xeno. yes, except i don't see any part of project gutenberg that is doing very much at all in the way of "approaching perfection". once a text is posted, people seem to forget it completely... even when the whitewashers "correct" an e-text, they aren't doing nearly what they could be doing in order to improve it. it seems like there is a constant mad dash to do new books, but almost nothing is being done to fix up any older books. i asked at 10,000 books for a review in terms of quality control, and again at 15,000, and again at 20,000, and again at 25,000. i didn't bother to ask again at 30,000, because what's the use? but at some point, some hard questions will need to be asked... > Does anybody have a serious objection to putting the 8,000, > or so, books that were listed earlier as being in limbo, in > something like our "PrePrints" section, where we put eBooks > that are admittedly not ready for prime time??? well, i certainly don't... but many of the volunteers over at distributed proofreaders do. indeed, according to a poll (which has now received one of the highest number of votes on any poll that has been done there), there are split evenly -- right down the middle -- on this issue. i don't know what to make of that. but that's the way it is. it's also worth mentioning that those 8,000 books are _not_ "almost done". some of them really aren't even very close... some are full of typos, still. most contain pseudo-markup, which really should be converted to something more useful before the books are ever put in front of the general public. lots contain "proofer's notes", which would confuse people. it should also be noted that many of them are not in english, which might (or might not) have bearing on the question, but since i only speak english, i wouldn't have any idea what it is. considering all this, it would _not_ be a simple procedure to free up this matter. it could use up a lot of time and energy, and for very little benefit in return. (does any use "preprints"?) what _would_ be useful is for this material to be put on a wiki, in order to test notions of public postprocessing collaboration. instead of saying "here, take this unfinished work", we _should_ instead be saying "here, come help finish this unfinished work". at one point, i was tempted to build such a postprocessing system. but then i realized i didn't want to help d.p. get over their backlog; d.p. deserves to suffer the consequences of their terrible workflow, or they'll _never_ be motivated to fix it... so i decided to let it be... -bowerbird

2 1

Re: ok, let's take a look at gardner's book, just for the exercise
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

i've merged gardner's corrections into the archive.org o.c.r., and posted the results on my website. here's a sample url: > http://z-m-l.com/go/gardn/gardnp123.html gardner dehyphenated end-line hyphenates, so i rejoined (some of) them. i'll write another routine to do the rest... out of a file that contains about 4,000 lines, there are only 800 (at this point) which differ in the two versions. so even early in the merge, 80% of the o.c.r. lines were right, and that number will increase with more aggressive cleaning. the (presumably incorrect) lines from the o.c.r. are in red, while the lines from gardner's proofed copy are in blue... if you prefer to view all of the edits on one web-page, see: > http://z-m-l.com/go/gardn/gardn-hybrid.html -bowerbird

1 0

Re: so what is so important about pagination?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

gardner said: > I'm going to take this as a jumping off point > for a more general question about > whether pagination of a published edition, > is worth saving. Obviously there is a range of opinion. > I'll give you mine. yes, there is a range of opinion. i can give arguments -- even good arguments -- on both sides. which obviously means that _some_ people have good arguments for retaining pagination. and you disenfranchise those people entirely when you throw out the pagination, no matter how good your intentions might be for doing so. i'd rather not disenfranchise those people... so i think it's necessary to include the pagination, and the original linebreaks, with end-line hyphenates. now, i think it's _imperative_ that we give people tools that enable them to discard that pagination, and unwrap those original linebreaks, and rejoin end-line hyphenates. to do otherwise would be to disenfranchise _those_ people, and i'd rather not disenfranchise them either. so, for me, the answer to the question is extremely simple. > What I believe, philosophically, I am shooting for > is to capture the core content, and reject the details > that have mainly to do with the medium of publication. i can understand that perspective. i can also understand the other perspective. and i see no reason that anyone has to be unhappy here. it's very important to understand that this does _not_ have to be an either/or question. we _can_ do _both_... > Maybe there are possible future uses of my text > that would want the things that I left out. > I tend to doubt that this could ever be very important. well, then, your imagination is starting to lag behind... :+) because we are now right in the middle of a situation here where "the things that you left out" _are_ "very important", namely a reproofing of your book, to test your accuracy... it's an order of magnitude more difficult to proof a book when the text has lost all of its linebreaks and pagination. are you of the opinion that the future will simply _accept_ that you did a perfect job in the digitization of your books? or do you think they will want to verify the quality of them? if you make it too difficult for them to undertake that job, they will just toss out your text and start anew. your loss... > As an individual contributor I do not feel that my time > is best spent capturing and encoding that, and so I don't. except the info was already captured. then you threw it away. > And I am happy that PG finds my efforts acceptable > despite this deficiency. except the future will throw out all the d.p. works because your deficiency is shared by the entire d.p. corpus, sadly... (even the d.p. people who save pagination toss linebreaks.) > I haven't done any sort of real research, but > a quick look shows me that > not many texts attempt to preserve line endings in any way. > Preserving line endings seems quite unpopular. the future needs to future-proof tens of millions of books... they can afford to throw out everything done up to this point, if they feel they need to, and they will, they most certainly will. (and advances in o.c.r. and o.c.r. correction will make it easy.) (well, as i pointed out, they might use some of the current texts to proof the new o.c.r. that they do, but then they'll toss them.) > Bowerbird wants to keep both actually, i don't need to take a personal stand on the issue, not as an end-user. and that's a good thing, because often i don't have a need for the original linebreaks or pagination. so i definitely want the option of discarding that information. what i am saying is that, as a "best practice" for digitization, the discarding of such information is clearly a terrible mistake. and if you're doing it simply because _you_ don't see a need for that information, then you're being selfish and shortsighted. plus you're giving an ultimatum to people who want that info: they're forced to toss your text as failing to meet their needs. and i am positive that you're going to lose that bet, gardner... -bowerbird p.s. by the way, i found 23 discrepancies in the paragraphing between your version of "the advocate" and archive.org's o.c.r. however, 21 were errors in the o.c.r., and only 2 in your book. the two errors in your book, both of them missed paragraphs: > http://z-m-l.com/go/gardn/gardnp087.html > "What, take the bird back to the bush where we > http://z-m-l.com/go/gardn/gardnp101.html > Yet who shall blame the sun and moon for that?

2 1

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 21 Feb '10

21 Feb '10

postprocessing at distributed proofreaders is difficult only because the proofers are instructed to throw out meaningful data, which later then needs to be replaced, and the formatters insert obtrusive pseudo-markup, much of which later needs to be reworked or deleted. if the proofers used nonobtrusive zen markup instead, it wouldn't interfere with their proofing task, and there wouldn't need to be a separate formatting task, even if (in reality) people decided to specialize on that aspect. also, the conversion of proofed/formatted pages into a full-on electronic-book should be an automatic process. i've already demonstrated this many times, but would be happy to do it once again, on any book of your choice... this point is particularly important in a roundless system, where the object is to move a page to a "finished" status as quickly as possible. -bowerbird

2 1

rfrank reports in
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

ok, rfrank has made a report over in the d.p, forums on the latest set of results from his "roundless" experiment. so let's see what he says, and what i think in reaction... *** rfrank said: > So far in testing the roundless system as it stands, > I've left it to the proofer to say when they thought a page > was done. Turns out, that is reliable for only a very few proofers. > Those who wish to say "I told you so" can chime in now, rightfully. ok. > the post-processing clearable errors were > caused mostly by four proofers, and > each of those made several different kinds of mistakes. > These mistakes were found almost entirely > on pages that were one-and-done, that is, proofed once. > So what is to be done? inform those proofers they are making mistakes, and how, and that they not doing nearly as well as they think they are. and to put this into perspective, there are about a dozen committed proofreaders taking part in this experiment, with another 5 dozen people contributing fewer pages... so four "bad" proofers constitutes about 1/3 of the lot... in other words, even though "4 proofers" sounds _rare_, the actuality is the percentage of "bad" proofers is high. this fact should _not_ be surprising. when you fail to give people any feedback on their performance, many will think they're doing a fine job, even if they're doing a terrible job. (this is a big problem over at the d.p. mothership, but we probably shouldn't be getting into that can of worms now.) after all, if they didn't think they were doing fine, they would change what they were doing, so they _could_ be doing fine. so you absolutely need to give them good and fast feedback. > One solution is to have every page > looked at by at least two proofers. that seems straightforward, but it has some gotchas. > That seems straightforward but it has some gotchas. right. :+) > If every proofer knows that every page > is going to be looked at by someone else, > will they proof that page differently > than if they intended it to be one-and-done? it's likely. so you'd need to assume it, and work from there. > I think they might. Knowing the underlying mechanism > can undermine the process. well, you must assume people "know the underlying mechanism", because you want to be open and transparent about it with them. there's really no other option when you're working with volunteers. > Also, what if the second proofer is one of the four mentioned earlier? or what if they both were? > There is a good chance that many of the errors would slip through. right. > It's easy for me to change the site code to force two looks at every page, > and I'll probably do that, perhaps even with a project in progress. doesn't matter. even after two forced looks, some errors will remain. > A down side to the "every page looked at by at least two proofers" > approach is specific to fadedpage: that there are only a dozen or so > active proofers of the 60 or so registered users. The double-look > algorithm adds about 35% to the number of page looks on a project. doesn't matter. there's no need for any haste on the books coming out... > A better solution that just a double-look is > to actually instantiate Confidence in Proofer (CiP). i was afraid you were gonna say that. and it's absolutely the wrong approach. > For these four proofers, the system could schedule a second look at > their pages even if they check the "this is done" box when done proofing. > It would give them plenty of diffs to look at, and they would be > expected to look at those diffs that show some correction was made. well, it'd be better just to inform them and educate them in the first place, rather than impose an "expectation" on them that informs them (indirectly) and forces them to educate themselves (again, in a very indirect fashion)... > If diffs were not checked, then their access to new pages > would be reduced. The kind of proofer who checks diffs, > learns, and continues to contribute is exactly what is needed. well, yeah, maybe... but you're assuming a real luxury of an overabundance of volunteers, and a willingness to throw a good number of them away as "not exactly what is needed". it's better to figure out how to find a use for _all_ volunteers. > I believe for a roundless system to work, there has to be > a reliable mechanism for stopping a page as done. d'oh. there has been complete agreement that that is the issue from day 1. > I also believe that to have a reliable way to make that determination, > some form of Confidence in Proofer needs to be in place. some people have held that belief, yes. i think it's unobtainable, and wrongheaded, and basically a dead end. even if you get a rudimentary version, it won't turn out to be useful... > Therefore, CiP, which is important, and page tweets, which are > useful and fun, are currently my main coding efforts at fadedpage. yeah, well, you'll be coming back sometime down the line and saying "those who wish to say 'i told you so' can chime in now, rightfully"... the thread has more, on confidence-in-proofer, but i'm not gonna waste any more of my time dealing with that flawed concept... -bowerbird

13 41

Re: Bowerbird's software projects
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

i've uploaded the archive.org o.c.r. for "the advocate" -- the book that gardner suggested that i look at -- into the skeleton that was previously being used, at: > http://z-m-l.com/go/gardn/gardnp123.html the last page of the book shows the changes i made: > http://z-m-l.com/go/gardn/gardnp126.html the .zml that made these .html pages, as usual, is at: > http://z-m-l.com/go/gardn/gardn.zml if you compare the .zml file to the original o.c.r., you can see that it is very similar. it doesn't take much work to massage typical o.c.r. output to .zml. as no proofing has been done yet, the text is raw... (although this book came from the internet archive, it is a copy of a google book, which means the o.c.r. is very shoddy, since google puts out low-res scans. when archive.org does o.c.r. on its own scan-sets, the o.c.r. is fairly good, since they're using abbyy.) ordinarily at this point, in order to clean the o.c.r., i'd restore the linebreaks to gardner's p.g. e-text by using the linebreaks from the o.c.r. as a guide... however, gardner sent me a copy of the text as it was _before_ he rewrapped the original linebreaks, so i won't need to go through that boring exercise. i decided to post this o.c.r. text anyway, just so you could see what it would look like as it is "in process". at this point, the structure of the thing is pretty solid, in the sense that all of the paragraphing is correct, and the chapter-heads are in place, and all of that, it's just the scanning errors make the thing awful... but if you look past those scanning errors, you can see why this version is superior to the p.g. e-text: it is obviously self-validating against the p-book from which the text claims to have been generated, since it is easily compared to scans of that p-book. (or even against an actual hard-copy, if you prefer.) even if the p.g. text _is_ accurate, it can't be _verified_ as accurate, not quickly and easily, like this text can... -bowerbird

1 0

ok, let's take a look at gardner's book, just for the exercise
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

i said: > no, in such situations, i'll download the o.c.r. from archive.org, > which _does_ still contain pagebreak info, and unwrapped text, > and end-line hyphenates. and then i will use your proofed text > to make the corrections to the archive.org o.c.r. and then i will > throw your text away, and keep the corrected, unwrapped and > page-marked text with the original end-line hyphenates in it... it occurs to me that it would be quite instructive to demo this exercise. i'll be using gardner's book to show how i'd go through this process. to prep, i downloaded the scans for his book from canadia.com, and mounted them on my website, along with a skeleton copy of the text. here's a sample url: > http://z-m-l.com/go/gardn/gardnp123.html as you can see, the prefix for this book is "gardn", so if you put a copy of the banana-cream program in a folder of its own, and name it "gardn.exe", it will download the .zml file and the scans from the website, and you'll be able to see how to start to use it. -bowerbird p.s. mac users should name the app "gardn.app", of course... (or, since your .app extensions are likely hidden, just "gardn".)

2 1