February 2010 - gutvol-d - lists.pglaf.org

banana cream for mac and windows is available
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

banana cream for mac and windows is available. the linux version seems to have a plug-in conflict, which i do not feel like debugging right now, so linux people can use the windows version, or wait. (note: i also have a version for the classic mac o.s.) if you want to be a tester for this nice little app, let me know and i'll tell you how you can get it... this is the version from the fall of 2008, which is the last time i worked on it in earnest, and i can't remember what kind of state i left it in, so there might well be some rough unfinished edges in it; but for the most part, it should be pretty smooth. like i said, though, it ain't pretty. ain't pretty at all. ;+) -bowerbird p.s. if people want to use it, i'd be happy to work on it again, to include incorporating all of those reg-ex tests that gardner was requesting. but... as only gardner and joey (yes joey, i did hear you) have asked for a copy so far, that seems unlikely...

1 0

[SPAM] re: Re: Bowerbird's software projects
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

gardner said: > Perhaps not, but over time you have described > checks that your tools can do and > fixes that you can automatically make > that sound a little to me like a super-duper gutcheck. yes, except that those checks and fixes are most often programmed only into one-off versions of programs... it's usually the case that making those checks and fixes useful in the general case, against any random book, is a more difficult matter. this isn't an apology of any sort; it's just that my intentions (for the most part) are to show that a particular check can be accomplished, and is useful. so far i haven't concentrated on building them into my app, because nobody's really expressed much interest in the app. the app has a general spellcheck ability, and that captures a very high percentage of the errors that occur within a text. > Also the workflow I picture is a little like gutcheck -- > I am thinking of text-in text-out command line tools, > not something that needs to look at image scans or > makes me talk to it in a fancy U/I. i'm not sure what you mean by the workflow you "picture". i was asking about your _actual_ workflow, the one whereby you currently digitize books. are you saying that you now do your digitizations without ever looking at image scans? because i have a hard time imagining how you can do that. you should also know i am a mac person, for good reason. for me, the interface is prime. if you're looking for tools that work on a command-line, in a text-in-text-out way, i'm the wrong tree for you to be barking up, that's for sure. i certainly wouldn't call my interface "fancy". to the contrary, it's extremely utilitarian, and not very pretty, not pretty at all. but it _is_ an interface, with buttons and menus and all that nice stuff that makes the program a lot easier to work with... > a Linux build would, I think. Windows would be fine too. i'll send you both. > This is still fairly accurate: ok, that was very useful... my tool assumes that the page-scans are in the same folder as the app, which is easy enough to satisfy. the tool also assumes that your text is all in one file, and that the page-boundary is of a certain type. i'd assume that your vi skills will enable you to satisfy this assumption in a fairly simple manner. other than that, i'd say you'll be good to go. > The last couple of books I've done instead by scanning, > bulk OCR and then proof from the scans and raw OCR text > which I can do on the road with my laptop or > anywhere I can mount a USB key for a couple of hours. that's how you'll want to operate with my software, yes. > After OCR I have a few basic things that I do > via regular expressions in vi: you can continue to do those things in vi if you like. global changes in vi are much quicker than going through one-by-one changes in the interface. > The thing is that I do not have > a specific set of checks and fixes that I consistently do. that's something you'll want to remedy. i did a series here a couple years back where i collected a list of checks that was necessary for the book i tested, and somebody turned that list into a set of reg-ex tests. you can find that set on the download page for don's app: > http://code.google.com/p/dp50/downloads/list indeed, since you are already using reg-ex, you'll probably find that you prefer don's tool over mine, since his program lets you actually _build_in_ your own list of reg-ex checks... > I rely a lot on jeebies and gutcheck. so, when you get a report from them on the possible errors, you enter vi and use search to locate each one of the errors? > I would like something perhaps with > a wider range of things that it can find > so I don't have to know all the things to look for. well, yes. and you can find some very extensive lists of reg-ex checks, right on d.p. the problem is that many of those checks have a low signal-to-noise ratio, in that they create far too many false-alarms. this is a problem even with gutcheck and heebe-jeebe, if i'm not mistaken. so you really have to fine-tune your list of checks to the particular corpus on which you are working, to be useful. this is why don's app is so useful, because you can build in the list of checks you want to do, and modify it at will, and even enter in a specific reg-ex to see if it returns any hits. > Over the years you have mentioned > several automated checks and fixes > that sounded sensible enough to me. sounds like you really want to use that reg-ex list that was based on the month-long series that i did. > I'm not keen enough to go back through the archives, > find them and implement them -- but I am > nevertheless interested in trying a tool like this out > on a project to see if it adds value for what I do. having heard all this, i'd guess don's app is the one for you. > http://code.google.com/p/dp50/downloads/list i'll send mine to you too, but his is based on reg-ex checks... > http://www.gutenberg.org/dirs/3/1/2/1/31212/31212-8.txt > and just tell me what you find. I have no doubt there is lots if the scans are online too, or can be, i'll certainly take a look at it... without looking at them, i can't know if something is an error or not. -bowerbird

4 3

Re: Bowerbird's software projects
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

gardner said: > Where was that Sourceforge project again? there is no sourceforge project. my source code has never been open-source. my enemies will need to write their own code. > I know you've talked about tools that > do more/better checking than Gutcheck actually, i don't think i've ever compared my stuff with any other software directly, because any tool is better than no tool. gutcheck has some charms. my tools do different things, and do things differently, but whether they do "more" or "better" is not an issue. > and have automated fixing and such. some have, yes. it's also important to remember that i do lots of experimentation, with quick-and-dirty code that serves to test the usefulness of a particular feature, but which might never be implemented further, perhaps because it doesn't prove to be worthy, or because the generalized code would take more time than i can give, or simply because that task just hasn't been done yet... > I would like to try them out. > Where can get get my hands on this stuff? i'll be happy to send you a copy, gardner, since you are an independent producer -- that's my target sweetspot. until i release the program generally, which might be very soon but also might not, you'll have to agree not to distribute the app any further, since i want to know who has it so that i can engage them in dialog about it, but that's the only restriction at this point. you'll also need to tell me what version you want -- mac or p.c. or linux. your signature-block screams out linux, which is fine, but you'd be one of my first linux users, so if you want the more-well-tested windows version, say so. finally, please give a short description -- frontchannel -- of your _current_ workflow. how do you do your books? do you use an editor, or some other tool? use gutcheck? what kind of preprocessing do you do on the raw o.c.r.? if you need to view a scan to check the text on some page, how do you do that? how do you find errors, with reg-ex?, or via a word-by-word proof of every page? anything else? if anyone else wants to get a copy of my program, say so, either frontchannel or back. the same conditions apply... also, if you're interested, you should check out don's app: > http://code.google.com/p/dp50/downloads/list his tool is similar in many ways, and you might like it too. -bowerbird

3 2

roundlessness -- 005
by Bowerbird＠aol.com 19 Feb '10

19 Feb '10

we're looking at rfrank's "roundless" experiment at fadedpage.com... *** i've been looking at the data for one of the books rfrank used... the book is titled "eagles of the sky", and it runs about 240 pages. in total, there were about 500 lines that were changed by proofers. here's a list of roughly 300 of those lines: > http://z-m-l.com/go/frabf/frabf300diffs.html the list shows the original line, from o.c.r., and the edited line. there's also a link to each page-scan, if you want to view that... i could've included all 500 changed lines in this list, except that the 200 lines i've excluded contained errors that _should_have_ (most definitely) been fixed in preprocessing, and it burns me up to be a witness to such a tremendous waste of proofer resources. it's just a crime against the generous contribution of the proofers to put such shoddy text in front of them and expect them to fix it. roger frank may think i'm talking shit about him again, but i tell you, i'll talk shit about _any_ producer who gives shoddy text to proofers, and, on top of that, i will feel _justified_ and _moral_ about doing it. i mean, it's not exactly a _mystery_ how to do good preprocessing. i have spent a lot of time writing posts here documenting _exactly_ how to do good preprocessing, and i did a heckuva lot of research to learn and test those preprocessing methods to prove their utility. so when someone just _ignores_ what i've done, and continues to burn out proofers by wasting their time and energy with material that should've had the obvious-and-easy-to-detect-automatically errors fixed before it went in front of 'em, i have a right to be mad. someone needs to talk some sense into roger's head, and do it fast. not to say that roger is the only one, or the worst one. not by far. i don't even bother looking at what the p.g. content producers are giving to their proofers these days, to protect my blood pressure. but since the odds that anything has changed over there are none and none, someone should talk some sense into _their_ heads too. it's a _shame_ to be wasting proofer time, and you producers who fail to do good preprocessing should be _ashamed_ of yourselves. *** at any rate, if that list of 300 changes is too dense for your brain, you can also thumb through the pages and see the changes made: > http://z-m-l.com/go/frabf/frabfp123.html all changed lines are marked in red (o.c.r.) and blue (edits), so if any particular page (like page 123, linked to above) is all-in-black, it means that none of the lines on that page had any changes made. you will find this "stepping through the edits" to be very friendly... -bowerbird

1 0

Re: "The Inheritance" by Susan Edmondstone Ferrier
by Bowerbird＠aol.com 18 Feb '10

18 Feb '10

al said: > I'm not aware of any general PG practice to send newcomers to DP. i'm not surprised to learn that you're not paying any attention, al. > Your motivation? yeah, you know, the reason why i would spend time and energy on this. > What do I care? why did you suggest i do this book? > Be altruistic, and do a book. i'm altruistic just by being here, sharing my analyses with people, al. i'm altruistic when i _do_ the research that _leads_ to those analyses. i'm altruistic when i design and program tools that do what's required, because that proves that such tools can indeed be designed and coded, and it lets me be precise and experienced when i assess their value... i'm altruistic when i work up my suggestions for an improved workflow. i'm altruistic in a number of ways, al. to "do a book" seems unnecessary, unless you're intentionally suggesting that i should lower my sights a lot. i clean text for the fun of it, al, just like my girlfriend does her sudoku. so i don't need a _lot_ of motivation... but i certainly do need _some_... you've given me no good reason to work on the book you've suggested, al, so i'm not sure why you even bothered to mention my name at all... > If you want something challenging in the exact same way that _i_ will decide how i will be altruistic (or even _if_ i will be altruistic), i will also be the one who decides what is "challenging" to me. but, like, thanks for your suggestion, and have a nice day, ok? -bowerbird

2 1

obsoleting etext #1250?
by Peter Saint-Andre 18 Feb '10

18 Feb '10

[resending to gutvol-d after mail to subs(a)pglaf.org bounced] Project Gutenberg etext #1250 (Ayn Rand's novel Anthem) contains a fair number of errors, which I realize is not unusual for early PG texts. Last year I completed a thorough check of all published editions of Anthem in an effort to produce a definitive text. Although I have not yet had an opportunity to check the galley proofs of the 1946 edition housed in the Library of Congress (which I have reason to believe will answer several nagging questions about the author's intentions), as far as I know the text I have produced is the most accurate version on the Internet. Would Project Gutenberg be interested in accepting the text I have produced? You can find an XHTML version here: http://www.monadnock.net/rand/anthem.html For your convenience I have moved my textual notes to a separate page: http://www.monadnock.net/rand/anthem-notes.html Thanks for Project Gutenberg! Peter -- Peter Saint-Andre https://stpeter.im/

1 0

"The Inheritance" by Susan Edmondstone Ferrier
by Al Haines (shaw) 17 Feb '10

17 Feb '10

If anyone's looking for a project, look no further than the above. Internet Archive has assorted editions, none of which are projects in DP. All editions appear to be clearable under Rule1 (pre-1923). Hmmm... maybe bowerbird is up for them? Al

1 0

Re: Many solo projects out there in gutvol-d land?
by Bowerbird＠aol.com 17 Feb '10

17 Feb '10

yeah, i know, a call for actual _data_. what a bummer, man. ruins _all_ your fun, and brings the dialog to an abrupt stop. -bowerbird

1 0

roundlessness -- 004
by Bowerbird＠aol.com 16 Feb '10

16 Feb '10

we're looking at rfrank's "roundless" experiment at fadedpage.com... *** i'm going over the data for one of the books rfrank used in his test, and once again the results i observe are striking and unequivocal... the proofers made hundreds of changes in this 240-page book, but most of 'em could've been detected and fixed during preprocessing, which would've made the workflow both more smooth and efficient. sure, there is the occasional stealth scanno -- "array" for "army", and "riot" for "not" -- which (one could argue) would seem to require the word-by-word proofing that is expected at distributed proofreaders. but they are few and far between, and in almost all cases innocuous. and certainly one round of such close proofing will be all that would be needed if the obvious-and-easy-to-automatically-detect errors were found and fixed in preprocessing. once these obvious glitches have been fixed, the proofer is essentially doing _smooth-reading_... this is of the utmost importance if you really want (as rfrank claims) to have each page be "one and out", (i.e., be finished by one proofer). otherwise once is simply not enough, not for a good many pages... it's also the case that -- with the right tool -- doing preprocessing is fun and exhilarating. it's really a kick in the pants to be able to improve a book so quickly and efficiently, and move it to "the finish". compared to the boring nature of proofing, there is no comparison... i have demonstrated this same finding on book after book after book, with no exceptions, so i am quite confident that it is extremely robust. all you have to do is look for it, and i assure you that you will find it... i wonder why so many of you are so resistant to learning the truth... -bowerbird

1 0

in search of proofer(s)
by Marc D'Hooghe 16 Feb '10

16 Feb '10

Most people on this list will be busy (and silly or not), amongst other things, with PG... Nevertheless ... to speed up some of the things I'm doing I would welcome some help in "first round" proofing for complete books in English, French or Dutch. Anyone interested, or if you would know of somenone? Let me know. Marc.

1 0