- gutvol-d - lists.pglaf.org

rfrank reports in
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

ok, rfrank has made a report over in the d.p, forums on the latest set of results from his "roundless" experiment. so let's see what he says, and what i think in reaction... *** rfrank said: > So far in testing the roundless system as it stands, > I've left it to the proofer to say when they thought a page > was done. Turns out, that is reliable for only a very few proofers. > Those who wish to say "I told you so" can chime in now, rightfully. ok. > the post-processing clearable errors were > caused mostly by four proofers, and > each of those made several different kinds of mistakes. > These mistakes were found almost entirely > on pages that were one-and-done, that is, proofed once. > So what is to be done? inform those proofers they are making mistakes, and how, and that they not doing nearly as well as they think they are. and to put this into perspective, there are about a dozen committed proofreaders taking part in this experiment, with another 5 dozen people contributing fewer pages... so four "bad" proofers constitutes about 1/3 of the lot... in other words, even though "4 proofers" sounds _rare_, the actuality is the percentage of "bad" proofers is high. this fact should _not_ be surprising. when you fail to give people any feedback on their performance, many will think they're doing a fine job, even if they're doing a terrible job. (this is a big problem over at the d.p. mothership, but we probably shouldn't be getting into that can of worms now.) after all, if they didn't think they were doing fine, they would change what they were doing, so they _could_ be doing fine. so you absolutely need to give them good and fast feedback. > One solution is to have every page > looked at by at least two proofers. that seems straightforward, but it has some gotchas. > That seems straightforward but it has some gotchas. right. :+) > If every proofer knows that every page > is going to be looked at by someone else, > will they proof that page differently > than if they intended it to be one-and-done? it's likely. so you'd need to assume it, and work from there. > I think they might. Knowing the underlying mechanism > can undermine the process. well, you must assume people "know the underlying mechanism", because you want to be open and transparent about it with them. there's really no other option when you're working with volunteers. > Also, what if the second proofer is one of the four mentioned earlier? or what if they both were? > There is a good chance that many of the errors would slip through. right. > It's easy for me to change the site code to force two looks at every page, > and I'll probably do that, perhaps even with a project in progress. doesn't matter. even after two forced looks, some errors will remain. > A down side to the "every page looked at by at least two proofers" > approach is specific to fadedpage: that there are only a dozen or so > active proofers of the 60 or so registered users. The double-look > algorithm adds about 35% to the number of page looks on a project. doesn't matter. there's no need for any haste on the books coming out... > A better solution that just a double-look is > to actually instantiate Confidence in Proofer (CiP). i was afraid you were gonna say that. and it's absolutely the wrong approach. > For these four proofers, the system could schedule a second look at > their pages even if they check the "this is done" box when done proofing. > It would give them plenty of diffs to look at, and they would be > expected to look at those diffs that show some correction was made. well, it'd be better just to inform them and educate them in the first place, rather than impose an "expectation" on them that informs them (indirectly) and forces them to educate themselves (again, in a very indirect fashion)... > If diffs were not checked, then their access to new pages > would be reduced. The kind of proofer who checks diffs, > learns, and continues to contribute is exactly what is needed. well, yeah, maybe... but you're assuming a real luxury of an overabundance of volunteers, and a willingness to throw a good number of them away as "not exactly what is needed". it's better to figure out how to find a use for _all_ volunteers. > I believe for a roundless system to work, there has to be > a reliable mechanism for stopping a page as done. d'oh. there has been complete agreement that that is the issue from day 1. > I also believe that to have a reliable way to make that determination, > some form of Confidence in Proofer needs to be in place. some people have held that belief, yes. i think it's unobtainable, and wrongheaded, and basically a dead end. even if you get a rudimentary version, it won't turn out to be useful... > Therefore, CiP, which is important, and page tweets, which are > useful and fun, are currently my main coding efforts at fadedpage. yeah, well, you'll be coming back sometime down the line and saying "those who wish to say 'i told you so' can chime in now, rightfully"... the thread has more, on confidence-in-proofer, but i'm not gonna waste any more of my time dealing with that flawed concept... -bowerbird

13 41

Re: Bowerbird's software projects
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

i've uploaded the archive.org o.c.r. for "the advocate" -- the book that gardner suggested that i look at -- into the skeleton that was previously being used, at: > http://z-m-l.com/go/gardn/gardnp123.html the last page of the book shows the changes i made: > http://z-m-l.com/go/gardn/gardnp126.html the .zml that made these .html pages, as usual, is at: > http://z-m-l.com/go/gardn/gardn.zml if you compare the .zml file to the original o.c.r., you can see that it is very similar. it doesn't take much work to massage typical o.c.r. output to .zml. as no proofing has been done yet, the text is raw... (although this book came from the internet archive, it is a copy of a google book, which means the o.c.r. is very shoddy, since google puts out low-res scans. when archive.org does o.c.r. on its own scan-sets, the o.c.r. is fairly good, since they're using abbyy.) ordinarily at this point, in order to clean the o.c.r., i'd restore the linebreaks to gardner's p.g. e-text by using the linebreaks from the o.c.r. as a guide... however, gardner sent me a copy of the text as it was _before_ he rewrapped the original linebreaks, so i won't need to go through that boring exercise. i decided to post this o.c.r. text anyway, just so you could see what it would look like as it is "in process". at this point, the structure of the thing is pretty solid, in the sense that all of the paragraphing is correct, and the chapter-heads are in place, and all of that, it's just the scanning errors make the thing awful... but if you look past those scanning errors, you can see why this version is superior to the p.g. e-text: it is obviously self-validating against the p-book from which the text claims to have been generated, since it is easily compared to scans of that p-book. (or even against an actual hard-copy, if you prefer.) even if the p.g. text _is_ accurate, it can't be _verified_ as accurate, not quickly and easily, like this text can... -bowerbird

1 0

ok, let's take a look at gardner's book, just for the exercise
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

i said: > no, in such situations, i'll download the o.c.r. from archive.org, > which _does_ still contain pagebreak info, and unwrapped text, > and end-line hyphenates. and then i will use your proofed text > to make the corrections to the archive.org o.c.r. and then i will > throw your text away, and keep the corrected, unwrapped and > page-marked text with the original end-line hyphenates in it... it occurs to me that it would be quite instructive to demo this exercise. i'll be using gardner's book to show how i'd go through this process. to prep, i downloaded the scans for his book from canadia.com, and mounted them on my website, along with a skeleton copy of the text. here's a sample url: > http://z-m-l.com/go/gardn/gardnp123.html as you can see, the prefix for this book is "gardn", so if you put a copy of the banana-cream program in a folder of its own, and name it "gardn.exe", it will download the .zml file and the scans from the website, and you'll be able to see how to start to use it. -bowerbird p.s. mac users should name the app "gardn.app", of course... (or, since your .app extensions are likely hidden, just "gardn".)

2 1

Re: Bowerbird's software projects
by Bowerbird＠aol.com 20 Feb '10

20 Feb '10

gardner said: > That's kind of my experience, I guess. > Several fixes will suggest themselves, > in the context of a given specific text. > The next one might need different fixes. right. a rule i always follow is that when i find an error, i always search the rest of the text for other occurrences. > But that doesn't mean a long list of fixups > might be tried when there's no cost > to just adding tests/fixes to the list. well, except for what i mentioned about false alarms. it's a rare test that doesn't turn up any false alarms, but when one turns up too many of them, it becomes a liability instead of an asset. the question is always, "how many false alarms is too many?", and the next question is always, "how can i weed out false alarms?" > Better stick to Windoze, if it's a GUI. actually, they are both generated from the same code, so they should act identically. whether they really do... > Text in one file -- check. > I favour marking page boundaries with "===00123" > these days, but a global search/replace can fix that. my app is looking for separator lines that look like this: {{myantp123.png}} || the_runhead || (note that there is a space in the first column.) that .png filename there is the name of the page-scan, and the program assumes you name your files wisely... so, for instance, if you want to jump to a certain page, you simply type the page number and press enter, and the program automatically jumps to that page. nifty... > Yes. Looking at that. I am not 100% sure > I want to mess with Twister exactly, but > the list of regular expressions looks interesting. > I'm picturing building a perl script that > applies all of these fixes, then creates a patch set > based on the the differences it has introduced. > I could then edit the patch set as a file, > nuking changes that are wrong, and > finally apply the patches for the changes I like. i would be very surprised if you can make that workflow more efficient than simply editing text in the interface... the beauty of my app, and twister too, is that you can view the page-scan to help you make the edit decision. i'm well aware that you don't _need_ to view the scan in order to resolve the vast majority of questions, but the inefficiency in handling that thin minority is huge if the bureaucracy of viewing the scan is too convoluted. > Jeebies and gutcheck reference specific line numbers. try twister. seriously. the ability to jump right to the page where the question occurs, and view the scan in context, is a major boost to efficiency. i bet you will be surprised... > I find it takes a good couple of passes before > I am satisfied I have all the genuine hits covered. > Invariably the WW finds things I've missed anyhow. that's a sign of an inefficient workflow. you want to accomplish things in one pass, and you want to make sure you got all of it. > Got those. Like I say -- I will turn it into > a perl script and see where that takes me. a perl script is operating blind. get a seeing-eye dog. > Lots of choices there. > http://www.canadiana.org/ECO/ItemRecord/48293?id=16c79d4f15394e51 > http://www.archive.org/details/advocateanovel00heavgoog > http://books.google.com/books?id=ot4OAAAAYAAJ&oe=UTF-8 none of those options are all that useful to me, however... what i need is to have the individual scans available online, each of them individually addressed with their own address. for instance, that "myantp123.png" file i referenced above, the one that reflects the scan of page 123 in "my antonia"? you can find that right here, in sequence with all the rest: > http://z-m-l.com/go/myant/myantp123.png this is the way the library of the future will be organized... if you want your work in it, mount your files appropriately. yes, i can download the .zip file of the scans from archive.org, or pull 'em from the google .pdf, and then mount them myself, but that's too much work for me to do, when you could have mounted them correctly in the first place. > There are no page numbers in the Gutenberg text though. then you threw away some very crucial information, didn't you? probably rewrapped the text too, am i right? and dehyphenated? all these actions make any kind of reproofing an impossible task. which is not to say that your proofing work was a waste of time. no, in such situations, i'll download the o.c.r. from archive.org, which _does_ still contain pagebreak info, and unwrapped text, and end-line hyphenates. and then i will use your proofed text to make the corrections to the archive.org o.c.r. and then i will throw your text away, and keep the corrected, unwrapped and page-marked text with the original end-line hyphenates in it... and when i throw away your text, i throw away your credit-line. had you kept all that valuable information which i need to have, instead of tossing it out, i probably would keep your credit-line. you know, just so you know... -bowerbird

2 1

oops
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

crap. i meant to send that only to gardner and joey, not to the list. i don't care who gets the app, but i _would_ like to know who. so if you download it, please send me a backchannel saying so. (and if you download it more than once, let me know that too.) if there are unaccounted downloads, i'll have to delete the file, because i don't want too many copies out in the wild right now, beings that this is not meant as a finished copy in any respect... and since the cat is out of the bag, here's the mac version: > http://z-m-l.com/go/bananacream/bc2008f.app.zip -bowerbird

1 0

banana cream
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

hey guys- here's where you can get a windows version of banana cream: > http://z-m-l.com/go/bananacream/banana-cream2008.exe you'll also want to download this sample text: > http://z-m-l.com/go/bananacream/tjbus.zml put these two files in a folder of their own, and then rename the program to "tjbus.exe"... (by default, it loads the .zml file with the same name.) the program will then download the scans for that book ("the jungle", by upton sinclair) automatically from my site. (you can control the download method under one of the menus.) if you have any questions, let me know. i'd prefer to have the chance to address any complaints before you go public, but i have no desire for you to muzzle your truth however you see it. as stated earlier, i'd prefer you not distribute the app. thanks. -bowerbird p.s. the app follows my naming conventions, which require a 5-letter prefix at the start of each filename (e.g., "tjbus"), followed by a single letter declaring the type of page (either "c" for a cover, or "f" for forward matter, or "p" for a page), followed by the page-number (padded out to three places). if the page was unnumbered, you use the last page number, and append an "a", "b", "c", "d", etc., respectively, as needed. it's also the case that the .zml file needs a certain structure. i'll write something up for that, and send it along to you later.

1 0

banana cream for mac and windows is available
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

banana cream for mac and windows is available. the linux version seems to have a plug-in conflict, which i do not feel like debugging right now, so linux people can use the windows version, or wait. (note: i also have a version for the classic mac o.s.) if you want to be a tester for this nice little app, let me know and i'll tell you how you can get it... this is the version from the fall of 2008, which is the last time i worked on it in earnest, and i can't remember what kind of state i left it in, so there might well be some rough unfinished edges in it; but for the most part, it should be pretty smooth. like i said, though, it ain't pretty. ain't pretty at all. ;+) -bowerbird p.s. if people want to use it, i'd be happy to work on it again, to include incorporating all of those reg-ex tests that gardner was requesting. but... as only gardner and joey (yes joey, i did hear you) have asked for a copy so far, that seems unlikely...

1 0

[SPAM] re: Re: Bowerbird's software projects
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

gardner said: > Perhaps not, but over time you have described > checks that your tools can do and > fixes that you can automatically make > that sound a little to me like a super-duper gutcheck. yes, except that those checks and fixes are most often programmed only into one-off versions of programs... it's usually the case that making those checks and fixes useful in the general case, against any random book, is a more difficult matter. this isn't an apology of any sort; it's just that my intentions (for the most part) are to show that a particular check can be accomplished, and is useful. so far i haven't concentrated on building them into my app, because nobody's really expressed much interest in the app. the app has a general spellcheck ability, and that captures a very high percentage of the errors that occur within a text. > Also the workflow I picture is a little like gutcheck -- > I am thinking of text-in text-out command line tools, > not something that needs to look at image scans or > makes me talk to it in a fancy U/I. i'm not sure what you mean by the workflow you "picture". i was asking about your _actual_ workflow, the one whereby you currently digitize books. are you saying that you now do your digitizations without ever looking at image scans? because i have a hard time imagining how you can do that. you should also know i am a mac person, for good reason. for me, the interface is prime. if you're looking for tools that work on a command-line, in a text-in-text-out way, i'm the wrong tree for you to be barking up, that's for sure. i certainly wouldn't call my interface "fancy". to the contrary, it's extremely utilitarian, and not very pretty, not pretty at all. but it _is_ an interface, with buttons and menus and all that nice stuff that makes the program a lot easier to work with... > a Linux build would, I think. Windows would be fine too. i'll send you both. > This is still fairly accurate: ok, that was very useful... my tool assumes that the page-scans are in the same folder as the app, which is easy enough to satisfy. the tool also assumes that your text is all in one file, and that the page-boundary is of a certain type. i'd assume that your vi skills will enable you to satisfy this assumption in a fairly simple manner. other than that, i'd say you'll be good to go. > The last couple of books I've done instead by scanning, > bulk OCR and then proof from the scans and raw OCR text > which I can do on the road with my laptop or > anywhere I can mount a USB key for a couple of hours. that's how you'll want to operate with my software, yes. > After OCR I have a few basic things that I do > via regular expressions in vi: you can continue to do those things in vi if you like. global changes in vi are much quicker than going through one-by-one changes in the interface. > The thing is that I do not have > a specific set of checks and fixes that I consistently do. that's something you'll want to remedy. i did a series here a couple years back where i collected a list of checks that was necessary for the book i tested, and somebody turned that list into a set of reg-ex tests. you can find that set on the download page for don's app: > http://code.google.com/p/dp50/downloads/list indeed, since you are already using reg-ex, you'll probably find that you prefer don's tool over mine, since his program lets you actually _build_in_ your own list of reg-ex checks... > I rely a lot on jeebies and gutcheck. so, when you get a report from them on the possible errors, you enter vi and use search to locate each one of the errors? > I would like something perhaps with > a wider range of things that it can find > so I don't have to know all the things to look for. well, yes. and you can find some very extensive lists of reg-ex checks, right on d.p. the problem is that many of those checks have a low signal-to-noise ratio, in that they create far too many false-alarms. this is a problem even with gutcheck and heebe-jeebe, if i'm not mistaken. so you really have to fine-tune your list of checks to the particular corpus on which you are working, to be useful. this is why don's app is so useful, because you can build in the list of checks you want to do, and modify it at will, and even enter in a specific reg-ex to see if it returns any hits. > Over the years you have mentioned > several automated checks and fixes > that sounded sensible enough to me. sounds like you really want to use that reg-ex list that was based on the month-long series that i did. > I'm not keen enough to go back through the archives, > find them and implement them -- but I am > nevertheless interested in trying a tool like this out > on a project to see if it adds value for what I do. having heard all this, i'd guess don's app is the one for you. > http://code.google.com/p/dp50/downloads/list i'll send mine to you too, but his is based on reg-ex checks... > http://www.gutenberg.org/dirs/3/1/2/1/31212/31212-8.txt > and just tell me what you find. I have no doubt there is lots if the scans are online too, or can be, i'll certainly take a look at it... without looking at them, i can't know if something is an error or not. -bowerbird

4 3

Re: Bowerbird's software projects
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

gardner said: > Where was that Sourceforge project again? there is no sourceforge project. my source code has never been open-source. my enemies will need to write their own code. > I know you've talked about tools that > do more/better checking than Gutcheck actually, i don't think i've ever compared my stuff with any other software directly, because any tool is better than no tool. gutcheck has some charms. my tools do different things, and do things differently, but whether they do "more" or "better" is not an issue. > and have automated fixing and such. some have, yes. it's also important to remember that i do lots of experimentation, with quick-and-dirty code that serves to test the usefulness of a particular feature, but which might never be implemented further, perhaps because it doesn't prove to be worthy, or because the generalized code would take more time than i can give, or simply because that task just hasn't been done yet... > I would like to try them out. > Where can get get my hands on this stuff? i'll be happy to send you a copy, gardner, since you are an independent producer -- that's my target sweetspot. until i release the program generally, which might be very soon but also might not, you'll have to agree not to distribute the app any further, since i want to know who has it so that i can engage them in dialog about it, but that's the only restriction at this point. you'll also need to tell me what version you want -- mac or p.c. or linux. your signature-block screams out linux, which is fine, but you'd be one of my first linux users, so if you want the more-well-tested windows version, say so. finally, please give a short description -- frontchannel -- of your _current_ workflow. how do you do your books? do you use an editor, or some other tool? use gutcheck? what kind of preprocessing do you do on the raw o.c.r.? if you need to view a scan to check the text on some page, how do you do that? how do you find errors, with reg-ex?, or via a word-by-word proof of every page? anything else? if anyone else wants to get a copy of my program, say so, either frontchannel or back. the same conditions apply... also, if you're interested, you should check out don's app: > http://code.google.com/p/dp50/downloads/list his tool is similar in many ways, and you might like it too. -bowerbird

3 2

roundlessness -- 005
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

we're looking at rfrank's "roundless" experiment at fadedpage.com... *** i've been looking at the data for one of the books rfrank used... the book is titled "eagles of the sky", and it runs about 240 pages. in total, there were about 500 lines that were changed by proofers. here's a list of roughly 300 of those lines: > http://z-m-l.com/go/frabf/frabf300diffs.html the list shows the original line, from o.c.r., and the edited line. there's also a link to each page-scan, if you want to view that... i could've included all 500 changed lines in this list, except that the 200 lines i've excluded contained errors that _should_have_ (most definitely) been fixed in preprocessing, and it burns me up to be a witness to such a tremendous waste of proofer resources. it's just a crime against the generous contribution of the proofers to put such shoddy text in front of them and expect them to fix it. roger frank may think i'm talking shit about him again, but i tell you, i'll talk shit about _any_ producer who gives shoddy text to proofers, and, on top of that, i will feel _justified_ and _moral_ about doing it. i mean, it's not exactly a _mystery_ how to do good preprocessing. i have spent a lot of time writing posts here documenting _exactly_ how to do good preprocessing, and i did a heckuva lot of research to learn and test those preprocessing methods to prove their utility. so when someone just _ignores_ what i've done, and continues to burn out proofers by wasting their time and energy with material that should've had the obvious-and-easy-to-detect-automatically errors fixed before it went in front of 'em, i have a right to be mad. someone needs to talk some sense into roger's head, and do it fast. not to say that roger is the only one, or the worst one. not by far. i don't even bother looking at what the p.g. content producers are giving to their proofers these days, to protect my blood pressure. but since the odds that anything has changed over there are none and none, someone should talk some sense into _their_ heads too. it's a _shame_ to be wasting proofer time, and you producers who fail to do good preprocessing should be _ashamed_ of yourselves. *** at any rate, if that list of 300 changes is too dense for your brain, you can also thumb through the pages and see the changes made: > http://z-m-l.com/go/frabf/frabfp123.html all changed lines are marked in red (o.c.r.) and blue (edits), so if any particular page (like page 123, linked to above) is all-in-black, it means that none of the lines on that page had any changes made. you will find this "stepping through the edits" to be very friendly... -bowerbird

1 0