- gutvol-d - lists.pglaf.org

Re: !@! I Take That Bet! Re: Re: DP/PG vs. Google
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

keith said: > One big problem, > You dio not stil a a PG or DP text ebook. there seems to be a transcription difficulty there... that's ok, it happens even to the human among us. > You do have any markup what so even! who needs markup? o.c.r. can manage italics, not so well, agreed, but still. as for the structural aspects of a text, like chapter-heads and block-quotes and stuff like that, google is already showing that they are capable of handling such things... -bowerbird

1 0

Re: the d.p. opinion on "prerelease" of e-texts
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

jim said: > Although I don't see why pointing out that the system > doesn't work as designed should be considered "bellyaching" ...also said tongue in cheek... ;+) -bowerbird

1 0

let them eat cake
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

michael said: > Sorry, but lots of libraries are doing JUST that!!! > Selling the books after digitizing. . . . > I bought several volumes of the NY Herald when this was done. > I will probably buy more. i'm afraid this situation is already bad, and will certainly get worse. universities nationwide are coming into a huge money crunch and it costs a lot of money to house books. and the benefits of doing so are becoming ever less clear, since usage has dropped precipitously. (ends up if the kids can't get it online, they won't trudge to a library, they'll just do without. and not just the kids, but the _faculty_ too!) so yes, universities are building book warehouses for collections. (the u.c. southern regional library facility is on-campus at u.c.l.a.) but there's even more to the story. some google library partners like _michigan_ are forming co-ops (e.g., the hathi trust) that will offer scansets to other institutions. right now, obviously, they're aiming at colleges and universities, but it's fairly clear they will soon target research institutions and private schools, public schools in big cities, and big city libraries. every entity that is now funding a library (which is probably quite limited in scope and fairly expensive to maintain) will soon find they can instead get access to a much bigger corpus of material for a much cheaper price by subscribing to these rent-a-libraries. so they will all get rid of their paper-books. (can't afford both!) so the problem is not just the libraries where scans are made, but every single library across the entire country. now, in an ideal world, that would be great!, because we would all agree that everyone should have unlimited access to this library, just like they have unlimited access to their neighborhood library. but that's not what the moneychangers have in mind, no siree... this isn't a chance for society to save money. to the contrary, it's a way for the moneychangers to rob society. in addition to the fee they'll extract from each overall institution, they'll likely want to charge a fee to each individual user as well, perhaps even a per-page fee for every page every user views... some of you might think that that would be totally reasonable. you're mistaken. you're badly mistaken. you're very badly mistaken. the reason you're mistaken is that sharing these scans is a process that has very little variable cost. most of the costs were fixed costs. scanning, for instance, was a fixed cost, and a one-time cost at that. by the time these scans have been pushed out a dozen times, they will have paid their fixed costs... everything after that, and there'll be much usage after that, will be profit, pure profit, _excess_ profit. the moneychangers want to get paid over and over and over again. when you consider that these books were purchased and housed at _public_ expense -- some of 'em for well over a century -- this profiteering against the public's pocket is totally unconscionable. still, librarian bureaucrats have proven time and time and time again that they're complete idiots who will cut their own throats long-term to get even a questionable good in the short-term. so they will play along into this little con-game being played by the moneychangers, and the public will once again be left holding the bag of bills to pay... and the final upshot? we'll pay even more for books than we pay now, and the poor among us will find that their access is sharply curtailed... but, hey, what do poor people need books for anyway? let 'em eat cake. -bowerbird

1 0

roundlessness -- 010
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

by the way, i just thought i would reiterate an old screenshot... perhaps some of the people planting bugs in rfrank's ears will consider pointing him in the directions suggested here: > http://z-m-l.com/3column-zml.jpg that's a big screenshot, because i've got a big screen, but the idea is that the proofer does the word-by-word scan _not_ against a web-page's textfield version of the page, but rather against an .html-realized version of the page... (if proofers wanted, you could even use the d.p. font on it.) the main benefit is that you free 'em from having to look at the markup, because that's an unnecessary distraction. they see actual rendered italics, not the markup for italics. it's also possible this way to red-flag any possible scannos, as well as capitalization and punctuation improbabilities... you can also colorize quotations, which helps locate any missing or incorrect quotemarks. like i said, i have a big screen, so i can put up 3 pages -- the textfield, the original scan, and the .html version, but for a smaller screen, you'd put up the scan and the .html. then, only if there are changes to be made will the proofer summon the textfield for editing. i also show lots of buttons on the screen. some are there to add words to the book's custom dictionary, so that you don't have to have them flagged the next time they appear. the others are just marked with numbers, to indicate things that they could be used for, such as italicizing selected text. again, proofing the .html version is easier than the textfield. -bowerbird

2 1

Re: !@! I Take That Bet! Re: Re: DP/PG vs. Google
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

michael said: > The bet is that a Xerox machine type of scanning and OCR > will produce a 95% accurate copy of certain pages selected > from an average set of books, magazines, etc. > Just go to a library and ask for samples. that's not the bet at all. the bet is whether google can increase accuracy to 96% or 97%. we're not talking about the limits of scanning and o.c.r., people. we're talking about what a company with virtually unlimited funds and lots and lots and lots and lots of expertise with handling text can do _after_ they've scanned books and done o.c.r. on the scans, in order to improve the accuracy of that text. folks, we're talking about how well they can clean up their o.c.r. and i'm conservative by saying 96% or 97%... quite conservative. i've shown how useful it can be to compare two book digitizations. but for some editions of some books, google will have _many_ different digitizations, involving different physical copies taken from different physical libraries throughout the country, scanned by different machines, and perhaps processed using different o.c.r. they will certainly experiment with despeckling and resolution, and other variables, and should hit on a comparison combination which -- for their particular scans -- works remarkably effectively. they will also have tons of data on the types of errors that are made by their equipment, and knowing that _will_ help them fix the errors. but mostly just having _multiple_digitizations_ of the same edition of a book gives them the chance to raise accuracy through the roof. you guys want to tie google's hands in the same way yours are tied. but google's money and expertise mean they are _miles_ ahead... and eventually probably even light-years ahead... -bowerbird p.s. and the limitation on the bet that google can't use humans? why not? they have billions of pageviews every single day, not? why do you think they bought recaptcha and hired luis von ahn? they're not limited by the shackles that you want them to wear...

1 0

Re: Processing eTexts
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

michael said: > For those who worry about BB and "perfect" eBooks, > the following should ease that worry greatly!!! i must admit i can no longer tell when you are serious, when you are misreading me, and when you are joking. but i've been _perfectly_ clear, and consistent, all along. i say a book that has 1-error-or-less-every-10-pages is _perfectly_ ready for release to the general public, with the explicit understanding that we do all we can to encourage and make it easy for that general public to help us in moving the books toward _perfection_... i hear lots of chestbeating about quality -- both by those who argue for it, and those who argue otherwise -- but i see very little activity productively engaged in attaining it. i've also done scads of research on how to develop tools and processes that will help us improve on our accuracy, and means by which we can have the general public help. precious little of my progress has been utilized by anyone. i'm not hung up on perfection -- it's nigh unattainable -- but neither have i ever been willing to have any other goal. i consider my position to be a fully reasonable one, and i've been perfectly clear on it, and preached it consistently, since the start. if you heard anything else, you misheard. -bowerbird

1 0

Re: Processing eTexts
by Bowerbird＠aol.com 04 Mar '10

04 Mar '10

carel said: > Yes, we have been doing a semantic dance ok, that's kinda what i thought. > I just feel that a final 'proofing' stage before release > would assist in locating errors that were either > missed or introduced by the processing. well, having "one more proofing" is _always_ a great thing. provided that somebody else is willing to _do_ it, that is... the acid test is whether you deem it to be so necessary that you will do it yourself. that's a nice way to help you decide whether the _cost_ of that additional proofing is _worth_it_, whether it will provide enough _benefit_ in the text accuracy. once you gain enough trust in your tool and its performance, believe me that you'll decide that it performs "well enough"... but yes, it is important to gauge the accuracy of your tool... my goal is less-than-1-error-every-10-pages, and my tool and workflow consistently delivers better results than that. > A human will do the processing and humans > can make mistakes and some of the mistakes > that could be made in what would be both > error and formatting processes could be quite grand. i don't worry about errors that are "quite grand"... they're easy to spot, and obvious to debug and fix. my experience is small errors are more troubling... > I feel that a second set of eyes can never be > a bad thing when it comes to something like this. that's easy to say until we ask you to be "the second set" on a million e-texts, all of which are almost perfect now. you -- and anyone else we ask -- will say "good enough". at some point, the benefit of greater accuracy isn't worth it. and then we say, "if the people reading this book because they _want_ to read it cannot find any errors in it, then that's their problem, but we cannot spend any more time having innocent people re-proof this book _once_again_ simply because there _might_ still be an error in the thing." again, i draw the line quite specifically. if a page has been looked at by 2 people in a row who could not find an error, then i certify that page as "good enough for the public" and stop looking at it. you can make it 1 person, or 3 people, or 4 people or 8 people or 22 people, whatever you like, but nobody would ever suggest we keep proofing a book forever. now, let me be clear that i understand that you only said "a second set of eyes" and not 22 sets of them. i agree... and that's specifically why i use the comparison method, because it gives us two sets of eyes on a book, essentially. > And, those with less experience (or no experience) > in shaping the output of a text may feel more confident > about doing the process if they know someone else will > provide a checksum for their work before it goes public. except that the _public_ provides that checksum for them. they constitute your "second set of eyes", your 3rd, your 23rd. > the text would be released to PG and then should be > placed in some environment that allows for editing it > to 'perfection.' it would be nice if p.g. did this. or d.p. but neither does. -bowerbird

2 1

Re: the d.p. opinion on "prerelease" of e-texts
by Bowerbird＠aol.com 03 Mar '10

03 Mar '10

bob said, to jim- > I'm not going to argue this any further with you, though. truth be told, you've haven't provided any argumentation anyway. you ignored jim's main point, to argue some legalistic crap which jim knows quite well and was never in dispute. indeed, it is precisely the troubling fact that material which _is_ "in the public domain" in a _legal_ sense, but is only _available_ for sale, because d.p. can't get it out the door, that's the point... and if you have nothing to say in regard to that point, then it's probably a good thing that you stop posting replies of any type. (except that you've illustrated jim's point about d.p. apologists.) -bowerbird

4 6

Re: do not listen, pray
by Bowerbird＠aol.com 03 Mar '10

03 Mar '10

i said: > thousands/hundreds/dozens and just to remind everybody, the solution is very clear, and has been for a very long time, ever since we learned -- by my careful analysis of the many d.p. experiments -- that p1 proofers find as many errors in subsequent proofings as p2 proofers and even p3 proofers do. (indeed, of the 3, the p2 proofers were the least good at locating the errors.) so it's obvious that we can move text to perfection by simply running it through p1 repeatedly. one problem with that -- as we've already found -- is that sometimes p1 proofers will change correct text to incorrect text. that problem can be eliminated easily with a policy to review and reconcile diffs. (this policy is easy to implement roundlessly, and will also serve to train up your low-quality proofers, so it's win-win.) the other problem currently with repeated p1 is that d.p. hasn't created an unambiguous set of proofing instructions -- i know, you'd think the need for that would be obvious -- and thus sometimes proofers "cycle through" corrections... (e.g., a first proofer dehyphenates, a second rehyphenates, a third asterisks the hyphen, a fourth dehyphenates, etc.) i haven't discussed the f1/f2 problem, because it's a mirror of the p1/p2/p3 problem. a quick-and-easy confirmation of the f1 by a subsequent f1 view, and we're off to the races. likewise, i have not discussed the postprocessing problem, not explicitly for the most part, because it's the microcosm of the thousands/hundreds/dozens problem, it certainly is. and once again, the problem is the workflow. by the time all of its pages have been proofed and formatted, the book should fall in place more or less naturally and automatically. the fact that it does not, in the d.p. workflow, is a shortfall... it indicates that the workflow is deficient in some major way. but correcting the postprocessing problems is relatively easy. you simply need to analyze each page that needs "finishing" and determine how the proofers could've provided that for it, and you modify the proofing instructions appropriately. easy. -bowerbird

1 0

Re: do not listen, pray
by cmiske＠ashzfall.com 03 Mar '10

03 Mar '10

1 0