February 2010 - gutvol-d - lists.pglaf.org

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 16 Feb '10

16 Feb '10

walter said: > In that vein, how flexible is the DP software? it depends. in general it's not all that flexible... but sometimes a little creativity goes a long way. even more inflexible than the software, however, is the willingness of its coders to change anything. and even when there is someone like dakretz who is willing to roll up his sleeves and do some work, the administrators won't let him. so there you go. > I've been wondering to what extent > parallel P1 rounds might be helpful. ... > Aforementioned suggestions may be silly, > feel free to point out their silliness. it's _good_ to "wonder" about things, walter. it means your mind is working on a solution. so that part isn't silly at all. and the part about parallel p1 rounds is not silly either. to the contrary, it's a good idea; might not _work_, but it's still a good _idea_. so that's not silly at all. what _is_ silly, however, is that -- in spite of the fact that people have had this good idea for a very long time now -- d.p. has _never_ actually _tested_ it directly to see if it works. oh, they've run some research, and tried out some things, but they've never actually done a full-on _experiment_ to test the hypothesis. so, for years and years a parallel-proof idea has been around, but we're still "wondering" whether it might work or not. _that_ is silly... for the record, once again, i've reassembled some data from various d.p. "experiments" (i'm using the term extremely loosely here) and i've even written the software that helps you reconcile two iterations of parallel proofs, so i can give you some conclusions on all that, namely that it doesn't give you better accuracy, and thus it certainly doesn't outweigh the cost of doing the reconciliation (which is rather high, even given a good tool), so i don't recommend it. however, a focused experiment on this matter would be good, so as to validate my findings... having said all that, though, there's a "variant" on parallel proofing that you might find interesting... taking o.c.r. results from 2 different sources and comparing them to find their differences and then resolving those differences and calling it "finished" _does_ happen to be an extremely effective strategy, since it avoids all the word-by-word proofing rounds. i documented all this on a thread on the d.p. forums, entitled "a revolutionary methodology for proofing", or something to that effect... you could look it up... > I find P2 proofing exceedingly boring > because of the small number of errors > that are left to be fixed in texts that are > well-scanned and well-proofed in P1. well, there's a lot that could be said about this, walter. perhaps first and foremost is that proofing _is_ boring. especially a word-by-word proofing on an accurate text. > I can't imagine how mind-numbing P3 will be > if I ever become eligible for that 'status'. since most of the o.c.r. errors are gone by the time of p3, most p3 proofers have resorted to trying to find errors in the book itself, errors that the publisher/typesetter made. this lets them leave a comment, so they can do something. for instance, in the book that i'm now examining which rfrank used in his "roundless" experiment, there were 50 comments left in a 240-page book, or 20% of the pages. of course, addressing all these comments is a task that is done by the postprocessor, which is one of many reasons why that job has become more taxing in the current era... > I can imagine that only having to look at the differences > between redundant P1 proofed texts might be helpful > since it would take two independent P1 proofers > to overlook the same error to have it slip through. well, yes, and that's the main argument for parallel proofing. but it ends up that yes, indeed, "two independent p1 proofers" often _do_ "overlook the same error" and it then slips through. and in the same manner, sometimes an independent p1 and p2 proofer "overlook the same error" and it then slips through to p3. now it would be great if we had some solid _data_ on the numbers, so we could decide how much energy we want to spend on catching these errors that slip through. we've found that _some_ errors can go as many as 7 or 8 or 9 rounds without being caught, but no one is suggesting we spend that many proofing rounds on every page... so we have to decide how many rounds we will expend our energy, in order to catch what percentage of errors. it's really that simple. and to make that decision, it would be great if we had some data. and it's silly -- ridiculous! -- that we have not collected that data. > Another potential improvement might be to make > texts available to the next round on a per page basis > instead of having to wait for all pages to be finished > in the previous round. well, now you're suggesting a "roundless" system, walter. which is also not a silly suggestion. unfortunately, it's not a _new_ suggestion either, so you're not advancing the art. what you _are_ doing is showing we have no data on _this_ particular wrinkle either, even though it's a very old idea... and again, this failure to collect data and test hypotheses is extremely silly, especially since we debate matters endlessly. like clara peller bellowing "where's the beef?", we should now make it a community slogan to demand "where's your data?" meanwhile, i keep myself busy by collecting what data i can, and writing the software tools that we need to do these jobs. and i talk and talk, but most people here are too busy being silly to listen to me. which i find to be endlessly amusing. :+) -bowerbird

1 0

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 16 Feb '10

16 Feb '10

al said: > In short, DP's current processes produce error-free texts; > its old processes, from what I've seen of the results, didn't. oh my, this is just too rich. a full-on admission that an "old" d.p. e-text was full of bugs. too rich. because d.p. cheerleaders -- like karen "zora" lofstrom -- have _always_ maintained that d.p. output was super-clean; it's the stuff from the _individual_ producers that is shoddy, _not_ the material from d.p. you can see this same attitude expressed _to_this_very_day_ over on the d.p. forum boards. of course, it was _easy_ to prove them wrong in the old days; all you had to do was make a laundry-list of errors in a text. (i provided such a list of errors to this very listserve for a book that was postprocessed by zora herself -- #13603 -- and the _hundreds_ of errors i located have _still_ not been repaired, even though the book was posted way back in october of 2004. so much for zora's stance of superiority. her work is flawed.) anyway, after enough laundry-lists of errors had been made, d.p. people finally had to admit their quality-control was faulty. sadly, they didn't know how to fix their system, so they just piled on more rounds, and built a flawed "certification system" to promote some proofers to "final-round" status, which only had the effect of stagnating their workflow with huge queues, as a boatload of books (thousands!) plugged up the system... and they've clung to this hierarchical model in the face of clear evidence (from their own experiments!) that _proved_ that the p3 proofers aren't any better than the p1 proofers... and even though it's perfectly clear that you can get good pages without subjecting every page in every book to 3 proofing rounds and 2 formatting rounds, following by postprocessing, and then maybe smoothreading, and then maybe postprocessing verification, nonetheless that's what their workflow system calls for them to do. so they experiment with ways to circumvent that workflow system, instead of just fixing it. it is a comedy of errors, in slow-motion... but hey, as long as you get "error-free texts", then who cares if you're wasting tons of time and energy donated by the volunteers? -bowerbird

1 0

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 16 Feb '10

16 Feb '10

andrew said: > An easy text that has come through DP > can be prepared and submitted in one day; > a more difficult one can take a week or two; > a really hard one might take months > working on it on and off. > > Now take those same texts without the DP preparation, where > an individual starts working himself from the ocr output. > The easy text could take perhaps three to six weeks; > the more difficult one five to eight months or longer; > and the hardest texts that have been done through DP > could never have been attempted by an individual. you just made up all of those numbers. take an easy text, the kind that can be "prepared and submitted in one day" after having gone through d.p., but which -- according to you -- "could take perhaps three to six weeks" were it to be done by a solo person. your figures are just ridiculous... it takes an hour, perhaps two or three, to spellcheck a typical easy book and get it formatted into shape... for a more difficult book, the spellchecking time is dwarfed by the formatting task, which is not really significantly lessened by having gone through d.p. and no text is so difficult that it "could never have been attempted by an individual", so that's just balderdash... there might not be any individuals who _are_ motivated to take on big projects, but given the rate at which these big projects get finished at d.p., the gap isn't all that big. > One other very significant aspect is that DP > has been set up to encourage a sense of community. i'm not sure charlz "set up" d.p. for that specific purpose. it's true that a sense of community _has_ developed there. but that can happen just about anywhere. it's also the case that the d.p. community indulges itself often in groupthink, which is one down side of "community". i'm not arguing that the down side offsets the good, because i don't think it does, but if we are going to mention one side, let's mention both... > And you have ready access to people with > specialized knowledge about many languages, > musical notation, obscure unicode characters, > obselete typesetting conventions, etc. that's true. but it's also the case that that "ready access" _could_ have developed right here, on this listserve, and been available to everyone, including the "solo" producers. so having it exist only within the d.p. silo is a bit regrettable. > At no point in the history of DP was well, carlo has already pointed out that andrew's memory is a bit foggy on this particular point. and that happens with individuals as we grow older, so there's no shame in that... but there's a tendency among d.p. people to rewrite history, almost always in a way that's favorable to their interpretation, so it's always refreshing when that tendency gets a fact-check. -bowerbird

1 0

Re: When DP started, here was the basic process
by Bowerbird＠aol.com 15 Feb '10

15 Feb '10

don said: > So it should now be obvious to you > why the thread was not hijacked um, no. gardner's thread was most certainly hijacked. he was looking for other solo producers, and someone who might be interested in that topic now has to plow through a bunch of posts that talk about something completely different... -bowerbird

1 0

Many solo projects out there in gutvol-d land?
by Gardner Buchanan 15 Feb '10

15 Feb '10

I've done a few books for PG. I've used DP -- back in the day, but mostly I've been doing solo projects. I don't hear a lot about folks doing projects solo these days. Are there many of us out there? ============================================================ Gardner Buchanan <gbuchana(a)teksavvy.com> Ottawa, ON FreeBSD: Where you want to go. Today.

15 34

Re: When DP started, here was the basic process
by Bowerbird＠aol.com 15 Feb '10

15 Feb '10

see how easy it is to change the subject-header? *** don said: > When DP started, here was the basic process the irony was, back in those olden days, it was actually much more difficult to digitize a text, because the o.c.r. was horrific, and thus it was a pure pain to proof. nowadays, even though o.c.r. is vastly improved, it seems to take forever for a book to transit d.p. here's an illustrative datapoint i just churned... in one of the books that rfrank is using for his roundless experiment, even tepid preprocessing (which is what he practices) combined with o.c.r. to produce 20% of the book's 240 pages perfectly. another 30% of the pages had only 1 error on 'em. and most of the errors failed spellcheck, meaning they could've been isolated and fixed immediately, without need of a word-by-word proofing modality. d.p. uses dozens of volunteers, taking hours of time, to do something that one person can do in one hour. which would, you know, ordinarily be a very sad thing. except what makes it funny, in this particular case, is that the people at d.p. think they're being "efficient"... -bowerbird

2 1

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 15 Feb '10

15 Feb '10

why do you guys insist on hijacking threads? it's rude. -bowerbird

1 0

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 14 Feb '10

14 Feb '10

karl said: > I think we can change this. It would be much better > to do this mysterious PP'ing in a collaborative manner. > To experience this, I created an SVN repository and > started with TEI tagging. I'll add more of > the PGTEI framework soon: that's right. to simplify the job of postprocessing, throw in a dose of s.v.n. and then add some t.e.i., and all the complexities will waft away on a breeze. -bowerbird

3 2

Re: rfrank reports in
by Bowerbird＠aol.com 13 Feb '10

13 Feb '10

what a convoy of clowns... -bowerbird

1 0

Re: Revisting DP
by Bowerbird＠aol.com 12 Feb '10

12 Feb '10

keith said: > It seems obvious to all that > the DP system has severe deficiencies. > The question is how to help. you need to discuss that over at d.p. they don't listen to anything over here. the only reason i discuss things here is because they banned me from there -bowerbird

2 1