March 2010 - gutvol-d - lists.pglaf.org

Re: any arguments against "free-range" proofing?
by Bowerbird＠aol.com 16 Mar '10

16 Mar '10

juliet said: > The Australian Newspaper project allows a volunteer to > proof any article from ~100 yrs of lots of newspapers. well, that's quite different from what i'm talking about, which is to allow people to proof any page of a _book_, one that is being actively worked on at the present time. > Also, since there's no information as to whether or not > anyone has looked at (proofed) this article yet, there's > no way to know if one is duplicating work already done. again, quite different. i will explicitly inform people of the exact status of every page. however, if they _want_ to "duplicate" work that is "already done" -- by proofing a page that's already "finished" -- they can certainly do so. indeed, up through the 3rd "confirmation" a page is "done", the person would continue to receive "points" for doing so... the main reason a person would go the "free-range" route, i would think, would be so they could actually read the book in the process of proofing. i think that's a useful perspective. another reason might be to do a "specialized" look at the book. for instance, i think it'd be great for a person to look through the entire book just to find cases of _italics_ and _formatting_. even a pass checking the paragraph-starts at page-tops will be a useful quality-control mechanism i'd want to encourage. > All of this ties in to a sense of progress. indeed. > In a free-range system, there is no such assurance that > anyone else will want to help finish that book. i think it's just the opposite. if i inform people which pages haven't yet been proofed, many people would choose them. if i show people which pages need to be confirmed, i think some people will want to get their "points" that way instead. and other people will want to read straight through the book, without any regard for the state of any one particular page... by letting them choose whatever they like, rather than just _assigning_ them a page, with a choice to "take it or leave it", i think they're going to do a good job of progressing a book. > The only way I can see to achieve that > in a free-range environment is by > limiting the number of books that are currently available. which is, i think, a perfectly good way to achieve that goal. the idea that d.p. seems to have settled upon is that anyone can put a book in the system, without anybody knowing who -- if anyone -- will be there at the end to pick up the pieces. as a consequence, you now have two tons of half-done books. and this has nothing to do with "free-range", as evidenced by the fact that rfrank is using a "limited-number" approach in his system, in order to make sure that books don't get beached. > If no one is likely to look again at whatever page I've just done, > there is nothing to keep me from changing what it says. ok, i guess you're talking about the australian newspaper thing. which, again, has no applicability to what i'm talking about, so i won't bother to address it here, except to say that my system is _expressly_and_extensively_ geared to checking changes made. there won't be any "graffiti" that won't be painted over very quickly. -bowerbird

1 0

Re: Co-operative proofreading
by Bowerbird＠aol.com 16 Mar '10

16 Mar '10

dkretz said: > Nice one! > I can see we're going through the classic learning curve! > Let a thousand flowers bloom! you seem like a nice-enough fellow, don. so why are all those people over at d.p. throwing rocks at you? ;+) *** lee said: > Inspired by Mr. Frank, Mr. Adcock, Ms. Miske and Mr. Morasch who are all these people? hey lee, you left out dkretz. who has done more than anyone, including rfrank. don was instrumental in producing dp-canada. i have not talked about dp-canada because i was banned from it before it even started by one of the crazy people involved with it. but it seems to be limping along just fine, as far as i know, so if anyone wants to start a site, i'd advise you to look at dp-canada. or talk to don. no one has come nearly as close to me in inspiring the d.p. rock-throwers as don; he must be doing something right. > I decided to try to implement > my own vision of a co-operative proofreading process. good luck! i'd give you some feedback, but since you're in my spam folder and all, dialog would be stilted, so you're on your own, passey. it would be nice if a thousand flowers bloomed, but my bet is 999 will die on the vine. which might also be fine, i don't know. i just know i won't bother to keep count until there's a shakeout... but geez louise, even some people from d.p. itself now seem to be newly motivated to _do_something_. which would be a good sign, except they're planning a facelift to a system with structural damage. it might _look_ a little better, but it won't really _work_ any better. but -- largely due to rfrank's competition -- they're now _trying_... -bowerbird

2 1

Re: [SPAM] re: any arguments against "free-range" proofing?
by Greg M. Johnson 16 Mar '10

16 Mar '10

From: "Keith J. Schultz" <schultzk(a)uni-trier.de> > > More interesting would be how you track everything. I > believe you will need some form of a database for > the points system, Could points be *the* problem? I don't think "points" works well in performance metrics for either high-level professionals or in nonprofits (say, the parents of a Cub Scout den expected to contribute so much volunteer effort per year). For one, it creates the expectation that the reason one contributes to humanity is to get recognition for their effort on a piecemeal basis. That the person who puts in 40 hours a week of effort needs more praise than he or she who put in 35, 30, or 10 hours. Secondly, it creates false hierarchies where you don't allow someone into leadership until they've "reached a level". It's just my philosophical bias that the best nonprofits are not those with the best volunteer awards dinners. -- Greg M. Johnson http://pterandon.blogspot.com

1 0

[SPAM] re: any arguments against "free-range" proofing?
by Bowerbird＠aol.com 15 Mar '10

15 Mar '10

nobody came up with many objections to "free-range" proofing... *** keith said: > I do not see anything truely speaking against such a system. > The only problems are the administrative tasks involved. thanks for the feedback keith... > 1) you have to track all this. i think that's pretty easy. when they hit the "save/confirm" button, if there were changes made, then they have _saved_ a new version, which will then set up a "diff appointment" with any prior proofers. or, if there were no changes made, it's registered as a "certification". once a page has two consecutive certifications, it's marked as "done". (any free-range proofers can still proof the page, of course, but once all the pages are marked as "done", the book is ready to be finished.) the "diff appointment" can be resolved in one of three different ways. the first proofer can say "i goofed", or the second one can say "oops!" either of these actions mean one person loses points, and one gains... the third resolution -- when they can't come to a mutual agreement -- comes via a referee, who decides a winner and rewrites documentation so the issue doesn't come up again. points might or might not be lost. (or deducted points might be doubled, if a ref was called unnecessarily.) the purpose of diff review is simply to train correct proofing and coding. people who continue making bad changes might be asked to leave, but i don't anticipate it happening very often. people like to do a good job. > 2) keep everything store somewhere each subsequent saved-text will be stored, for subsequent diff tests. upon saving, it will be compared to all of the earlier saved versions, to see if it is a revert to an earlier save. if it is, it will be dealt with appropriately, depending on the resolution of that earlier version... any proofer will be able to step through all the versions of each file, viewing which changes were made. i expect that some proofers will specialize in this particular tactic, making sure every change is good. > 3) keep everything in sync i've had my share of sync problems in the past, coding e-book authoring-tools, so i think i know where all the pitfalls are now. :+) which is not to say i won't fall in some of 'em again sometimes. but i can usually figure out pretty quickly now what i did wrong. > you will need an authority/ies that finally certify that > a page satisfies your criteria as being done. that's easy. if the text is .zml that creates .html which looks like the page-scan, then that satisfies the criteria. the proofers view .html output, so can see for themselves. > Some may call it a administrative nightmare, > but it should be workable. yes, i think i can make it work. again, thanks for the feedback. -bowerbird

2 1

Re: roundlessness -- 013
by Bowerbird＠aol.com 14 Mar '10

14 Mar '10

i said: > even then, i just don't think "confidence in proofer" will actually work. > > or, to be more accurate, i think it'll work just well enough that rfrank will > put lots of energy into it before he finds out it doesn't work well enough. > > or, worst case scenario, he'll convince himself that it really _is_ working, > and other people believe him, and we all end up with non-perfect pages. good thing i posted that yesterday. because today rfrank posted his first "informal analysis". and it looks like i was right... rfrank did his analysis on 32 pages that were marked as "done" but then subsequently proofed again, as is done for a random sample... he admits this is a small number of pages, and that there are also "many factors at play", but then goes on to draw conclusions anyway. of the 32 pages, two had added proofer notes, and 1 error was fixed. he doesn't tell us if either (or both) of the proofer notes were good, in the sense that they pointed out something of value, so we'll have to assume that they were meaningless and just added noise to the text. but even then, we have 1 error missed in 32 pages. on the face of it, that means that 3% of the "done" pages had an error. so, for a 200-page book, that would cumulate to a total of 6 mistakes. again, by my 1-error-every-10-pages criterion, that's fully acceptable. but by the (unrealistic) standards of _most_ of the volunteers, it's not. rfrank concludes that "this seems to say that making sure every page is seen by two proofers is not warranted"... so that's his take on this. *** partly the decision rests on the abundance of proofers. if you have lots and lots of proofers, like d.p., then you can afford to send a page through them 2 times or 3 times, even 4 or 5 times. but if your proofers are scarce, like they are over at fadedpage.com, then you might be reluctant to have them view a page even twice... i think i'm pretty good about making sure proofers are used _wisely_. i don't think i abuse their contribution, or that i take 'em for granted; neither am i afraid to use their resources if it is responsible to do so. and i think having 2 people verify a page as clean is responsible use. *** the other thing, though, in evaluating all these experiments, is that you need to know how many errors there _really_ were on each page. only _then_ can you accurately access the accuracy of the proofers... remember that there are lots of pages in these books that have _no_ errors on them, none at all. is it any surprise, then, that they were _actually_ "done" when they were _marked_ as "done"? not hardly... likewise, it isn't really a surprise when a page with _one_ error on it has that error fixed, is then marked as "done", and is _really_ done. what you have to pay attention to, in such cases, are the pages where an error is _not_ found by the first person, who marks it "done", but is then found by the second person. rfrank isn't making nearly enough information available that we can analyze the results in a reasonable way. so i guess we just have to "trust" him. i just wish i had more faith in his reasoning. -bowerbird

7 8

Re: New Tool "pgdiff"
by Bowerbird＠aol.com 14 Mar '10

14 Mar '10

jim said: > Sorry – I guess I assumed people know how to use a regex editor! um, you just shot yourself in the foot, jim. you assume that people know how to use a reg-ex editor, but you also assume they need to have wdiff wrapped in an .exe? not much logic there, i'm afraid... but hey, for the sake of completeness of the thread, how about you quickly run through how i'd "use a reg-ex editor" for this? because i honestly don't know. (and i _do_ know how to wdiff.) -bowerbird

2 1

[SPAM] re: New Tool "pgdiff"
by Bowerbird＠aol.com 13 Mar '10

13 Mar '10

jim said: > To help clarify what I am talking about > I enclose below an except of the output of this tool personally, i find that output to be obtuse and hard to read. and editing it would be very problematic, and error-ridden. giving people a tool that would present the choices to them, and let them click a button for the correct one, would make this far more easier to work with. *** jim said: > I have created a new command line tool “pgdiff” good for you, jim! i assume you wrapped the "wdiff" routine in an .exe? that'll make it easier to use for normal windows users. > In this regards it is similar to “worddiff”, as opposed > to “diff” which is the approach BB has been > talking about, which compares on a per-line basis. well, i use "diff" as a generic term. whether you use "diff" or "wdiff" depends largely on whether the lines are broken in a similar way, or have been rewrapped. i usually find it's worthwhile to fix the linebreaks so they are identical in the files, and match the p-book. that's because to resolve many of these differences, you have to look at the actual page, and that job is infinitely easier if your linebreaks match the page... > But my new tool has several tricks > that haven’t been seen before: um, ok... > It can be used with two different versions or editions of > the text as long as there are not really long differences ok, but that's something that's been "seen before"... > This means it can also be used for “versioning” – > for example using a copy of a PG text from one version > or edition of a text to help fix and create a text > from a different version or edition of the text. i'm not sure i understand what you're talking about here. if there are differences, how do you know if the differences are edition differences or o.c.r. differences? you'd have to refer to the page-scans for one version or the other, right? > It can also be used to recover linebreak information, > where linebreak information has been lost, for example > to take an older PG text and recover linebreak information > in order to allow, for example, the resubmission of that > PG text back to DP for a clean-up pass. again, not something that hasn't been seen before... but i'd love to see this in action. carlo has _posted_ that people could use wdiff to do this chore automatically, but when asked to explain the procedure, he failed to follow up. > In normal mode when if finds an mismatch it outputs > the mismatch like this { it’ll | it’11 } within the body of > the text so that given a regex compatible editor it is > very quick to search for and fix the errors found. i'd really like to learn the reg-ex that makes this "very quick". i assume you'd search for the first half of the pair, and erase it if it's incorrect. then you'd do the same for the second half. then you'd go back and globally remove the excess characters. but i'd sure like to see that in action. and i don't think it would be very fast. or feel very easy. especially when -- for an error like '11 -- a global change within each of the files would end up being more efficient. it's also the case that, as i mentioned up above, you _need_ to have the scan available for viewing to resolve some diffs, so the ability of the tool to present those scans is _crucial_. > I find that finding differences on a word basis rather than > a line basis makes it quicker and easier to fix the errors if you've looked at the diffs i've presented, the _indicator_line_ narrows your focus down to a single word (if that's the diff), or even a single _character_ (like a comma, if that's the diff). it's just showing you the entire line so you have the _context_, and so you can _find_that_line_ more easily on the page-scan. > Source and a compiled windows version at i'll take a look, as soon as i happen to be around a windows box. in the meantime, congratulations for programming a tool! :+) -bowerbird

3 2

any arguments against automatic good-word listing?
by Bowerbird＠aol.com 12 Mar '10

12 Mar '10

i recently praised rfrank for simplifying the procedure of adding a word to the "good-words list", compared to d.p. when a word which _should_ be on that list is missing, proofers have to struggle through the false flagging of that word, and that decreases the efficiency of flagging. for instance, when the name of a character in the book is flagged every time it appears, it can cause "flag fatigue", by virtue of its frequent nature. moreover, it can cause you to miss the cases where the name was misrecognized, because you've grown accustomed to skipping that flag... (if the name is in the good-words list, only an _incorrect_ version of the name gets flagged, which is what we want.) and once your good-words list is _complete_, you can do spellcheck on the book and have it come out _clean_. this is extremely valuable, because it means that you can repeat that spellcheck after any major editing operation (or at any milestone that you decide during the workflow) to make sure that your processing didn't introduce errors. so it's in everyone's best interest to have a good-words list which actually contains all "good words" in the book. which is why rfrank's simple-and-immediate procedure is far superior to the d.p. way, which is hard and slow... but there are methods even better than rfrank's... one of the most useful tools in my arsenal is one that takes text as its input -- up to the entirety of a book -- and quickly spits out a list of words not in its dictionary. thus it gives me a list of words that i'll need to check... but _many_ of these words, primarily _names_, but also words that appear a relatively large number of times, are ones that will go onto the good-words list for the book. so this tool can be used in _preprocessing_ to create a good-words list that is actually compellingly complete. but let's say you did not use this tool in preprocessing, and your good-words list is still missing lots of words... now let's look at a case where a proofer has just finished a page that had a number of words flagged on it, because they were not in the dictionary or on the good-words list. even if the proofer didn't take the time to add the flagged words to the good-words list, should they be auto-added? because, if they're ok on this page, they should be added! in other words, why make the proofer go to _any_ trouble to add a word to the good-words list? just analyze the page they've saved, as "good", finding all words that are not on the good-words list, and adding them automatically? as far as i can see, there are 2 problems that might result. the first is that the proofer made an error, and failed to catch a flagged word that was incorrect. in such a case, that would mean that any other occurrences of that word would not be flagged. that's unfortunate, of course, but is it a great tragedy? i think not. proofers need to know that an unflagged word _might_ be incorrect, n'est pas? of course they do. that's the essence of a stealth scanno. the second problem is that the word might be correct on _this_ page, but is _incorrect_ if it appeared elsewhere, and would need to be flagged there. again, i do not take a failure to flag every bad word as being necessarily bad. but directly to the point of this second possible problem, i simply don't think this situation occurs all that often... so i would like to issue a challenge, to the people who look at more books-in-progress than i do, to _locate_ this situation, where a non-dictionary word is _correct_ on _one_ page, yet _incorrect_ on some _other_ page... after all, this is the raison d'etre for the d.p. procedure, which insists that a word nominated for the good-words list must be inspected/approved by the project manager. so -- if this situation is more common than i believe -- project managers should have _lots_ of examples for me. so let's hear them. anyway, that's the suggestion, that when a page is proofed, all the words which had been flagged are _automatically_ added to the good-words list. and yes, i will also add that i believe such additions should be _screened_ by someone, but that's in keeping with my overall plan that any and all changes that are made will be doublechecked by someone. -bowerbird p.s. if you were quick enough to grok a flip-side suggestion that any word on the good-words list which was _changed_ on any page should automatically be _removed_ from the good-words list, give yourself a gold star for a sharp mind.

1 0

Re: a question for dkretz about twisted
by Bowerbird＠aol.com 12 Mar '10

12 Mar '10

gardner said: > an AIR application does not translate easily into > a "just a browser" server-hosted application. so, gardner, i think you're telling me that "it can't be done", in regard to running "twister" in a browser. if i'm mistaken, and it can be done -- i.e., _you_ think that _you_ can do it -- do please let me know... thanks. -bowerbird

1 0

a question for dkretz about twisted
by Bowerbird＠aol.com 11 Mar '10

11 Mar '10

ok, so i'm coding a proofing system. and wow, i'm impressed with myself and how far i've gotten in just 2 days. i've got a solid engine going already... in programming, the saying goes that the first 90% of a project takes 90% of the time, and the remaining 10% takes the other 90% of the time. and it's true. but still, to have a solid engine after just 2 days means i think i can have a pretty smooth system in 2 weeks... but before i go reinvent the wheel... a question for dkretz on "twisted"... it was coded in "air", so in _theory_ anyway, it will run on a web-server. so, don, can you make it do that? is there any place where it _is_ running on a web-server now? if someone (like me) wanted to run it on their server, would you make the app available to them? i'd love to see it run in a browser. -bowerbird

3 3