- gutvol-d - lists.pglaf.org

improve scanning on color docs?
by Kimo Crossman 19 May '10

19 May '10

love comments from anyone who knows about this: Improve OCR Accuracy on Color Documents This white paper confirms that industry-standard practices to clean color document images can be improved to produce higher OCR accuracy. Image Detergent™ from Accusoft Pegasus improves OCR accuracy by 5-10% more than a standard Smoothing filter. This white paper leads the reader through the testing that proves it.

3 2

the problem with the e-books from the internet archive -- 08 of 32
by Bowerbird＠aol.com 18 May '10

18 May '10

for 32 days, i am showing samples of the problems with the text in e-books from the internet archive... *** today's sample is from the ever-popular "the art of war". here is the page-scan for page 54: > http://www.archive.org/stream/artofwaroldestmi00suntuoft#page/54 here's the o.c.r. for the entire book: > http://ia331333.us.archive.org/3/items/artofwaroldestmi00suntuoft/artofwaro… here is the o.c.r. for page 54: > > 54 > > the four seasons make ^ay for each other in turn. > Literally, "have no invariabl, seat>>> > > There are short days a^ long; the moon has its periods > of waning and waxing. > > Cf. V. 6. The purport of the passage j s s i mp iy to illustrate the want > of fixity in war by the chang s constant i y taking place in Nature. The > comparison is not very happ.^ however, because the regularity of the > phenomena which Sun Tzu m n ti ons i s by no means paralleled in war. this particular edition mixed ideographs in with the text, so i could've picked virtually any page and presented some hairy-looking o.c.r., but i decided to focus instead on a page with no ideographs whatsoever... -bowerbird

1 0

the problem with the e-books from the internet archive -- 07 of 32
by Bowerbird＠aol.com 18 May '10

18 May '10

for 32 days, i am showing samples of the problems with the text in e-books from the internet archive... *** today's sample comes from "gods and fighting men". here's the page-scan: > http://www.archive.org/stream/godsfightingmens00greguoft#page/17 and here's the o.c.r. for the entire book: > http://ia341017.us.archive.org/0/items/godsfightingmens00greguoft/godsfight… and here's the o.c.r.: > > '^f// THE COMING OF LUGH 17 > > ,^ Vf X W r^J^^*" ""* brass." " We have a worker in brass > oui's I\ s, that is Credne Card." > > Then Lugh said : " Go and ask the king if he has any > one man that can do all these things, and if he has, I > will not ask to come into Teamhair." The door-keeper > went into the king's house then and told him all that. > "There is a young, man at the door," he said, "and > his name should ue the Ildanach, the Master of all > Arts, for all the things the people of your house can > do, he himself is able to do every one of them." " Try > him with the chess-boards," said Nuada. So the chess- > boards were brought out, and every game that was > played, Lugh won it. And when Nuada was told > that, he said : " Let him in, for the like of him never > came into Teamhair before." > > Then the door-keeper let him pass, and he came into > the king's house and sat down in the seat of knowledge. > And there was a great flag-stone there that could hardly > be moved by four times twenty yoke of oxen, and > Ogma took it up and hurled it out through the house, > so that it lay on the outside of Teamhair, as a challenge > to Lugh. But Lugh hurled it back again that it lay > in the middle of the king's house. He played the harp > for them then, and he had them laughing and crying, > till he put them asleep at the end with a sleepy tune. > And when Nuada saw all the things Lugh could do, he > began to think that by his help the country might get > free of the taxes and the tyranny put on it by the > Fomor. And it is what he did, he came down from > his throne, and he put Lugh on it in his place, for the > length of thirteen days, the way they might all listen to > the advice he would give. > > This now is the story of the birth of Lugh. The > time the Fomor used to be coming to Ireland, Balor > > B we have several problems in the first few lines, which are bad on the scan as well, so we cannot fault the o.c.r. here, but whatever the cause, the text is botched nonetheless... -bowerbird

1 0

Re: Internet Archive makes more than 1 milllion books available online to blind, dyslexic
by Bowerbird＠aol.com 18 May '10

18 May '10

jim said: > How about if you make your point by "fixing" some of the bad OCR's at TIA, > resubmit them to TIA -- and then use those examples to make your point? how about if i decide how i will make my point? -bowerbird

1 0

adding e-books to itunes
by Bowerbird＠aol.com 17 May '10

17 May '10

here's more on this subject: > http://answers.oreilly.com/topic/1505-how-to-quickly-add-your-oreilly-ebook… -bowerbird

2 1

Re: Internet Archive makes more than 1 milllion books available online to blind, dyslexic
by Bowerbird＠aol.com 17 May '10

17 May '10

aaron said: > You did not fully understand the announcement, > nor do you understand what Archive.org is doing. yes, i do fully understand it. i didn't care to talk too much about the in-copyright stuff, because i believed that the less said about that, the better. i can talk about that if you want to delve into it deeper, but remember i do support the overall mission of archive.org... so, do you want me to talk about that? first, though, answer me this: do you believe archive.org's o.c.r. of in-copyright material will be significantly better? i'm using the public-domain material to show the problem, simply because it's all i can see, and conveniently point to. but do you believe in-copyright o.c.r. will be problem-free? there's a chance that newer books will o.c.r. more cleanly, but if not, do you think archive.org will make corrections? will they bring their e-books up to a standard of _quality?_ i don't think so. so i, too, stand behind the point which i have been making. -bowerbird

4 3

Re: Internet Archive makes more than 1 milllion books available online to blind, dyslexic
by Bowerbird＠aol.com 17 May '10

17 May '10

aaron said: > LOL. So much for > "_all_ these e-books are also available as text, > and have always been available in that format." aaron, i don't have experience with the daisy format. are you telling us that one can't get text out of that? and are you claiming archive.org doesn't _start_ with the o.c.r. output when it creates the daisy-format file? if they give you daisy, and not text, then that must be part of the d.r.m. they wrap around the file to prevent the books from being "pirated" out to the seeing world. what i said _is_ true about the public-domain material. perhaps none of the public-domain material is counted in their announcement of "more than 1 million books"... > I suspect that the quality of the public domain books > will be roughly the same as the copyrighted texts. so we agree on that. > As to whether or not they will spend the time and > money to improve the quality of their books, I doubt it. so we agree on that. > The cost would otherwise be prohibitive, unless > the improvement could be made via software. well, we cannot know that unless we categorize the errors. which is precisely why i am undergoing such categorization. but yes, i've done more than enough research along the way to say unequivocally that _many_ improvements can be made, via software, with -- at most -- minimal human input needed. which pinpoints some of the ridiculousness around this issue. on the one hand, there are apologists who try to tell us that "even bad o.c.r. is better than no o.c.r. at all." well, that's true. but if we're really willing to settle for that, then why do those people over at distributed proofreaders even bother working? obviously, some of us put a value on correcting bad o.c.r. on the other side of the tightrope, we have those same people at distributed proofreaders who try to tell us that their job of correcting o.c.r. is horrendously difficult and time-consuming. and this is equally ludicrous. it takes them a long time to do it because they're doing it in a way that takes a long time to do it. i'm trying to walk the middle-ground, which seems to me like it should be a very broad path that's obviously the best to take, where we spend a sensible amount of time on a book (an hour), and take it to a place where it meets almost all of our needs... we can decide to ignore quality and focus on quantity alone, and we can have millions of books; and all of 'em are flawed. we can decide to ignore quantity and focus on quality alone, and we can have 23,456 books, like project gutenberg does... we can even do _both_, and have 23,456 high-quality books and millions of flawed ones. so, is that what we want to do? or we can spend one hour per book, and have half-a-million pretty-darn-good books, with all their obvious flaws corrected. i don't know why it's so hard to see this middle path is the best. -bowerbird

1 0

Re: Internet Archive makes more than 1 milllion books available online to blind, dyslexic
by Bowerbird＠aol.com 16 May '10

16 May '10

aaron said: > I just love it when people criticize something > without fully understanding it first. um, aaron, just exactly what is it that you think i fail to "fully understand"? because i do fully understand that a book with poor o.c.r. is better than no book with no o.c.r. there's no question about that. but that's not what archive.org's press release was all about. they were trumpeting the release of e-books in the daisy format, for the blind and dyslexic. _all_ these e-books are also available as text, and have always been available in that format. the reason this press release is cynical is that the o.c.r. can be awful, as i've been showing, meaning the text is bad and the daisy is bad. a sighted reader can refer to the page-scan if a passage is unclear, and resolve the issue. but a blind reader has no such recourse... so, if anyone here doesn't "fully understand" the issue, i'm afraid it is you, aaron, not me. -bowerbird

2 1

the problem with the e-books from the internet archive -- 06 of 32
by Bowerbird＠aol.com 16 May '10

16 May '10

for 32 days, i am showing samples of the problems with the text in e-books from the internet archive... *** nobody bothered to comment on yesterday's example, which likely means that nobody's even looking at these. which is fine. i'm a self-starter. yesterday's scan was a page that had been "annotated"... that is, a library patron had underlined one sentence and bracketed a few more, and "corrected" a few printer errors. such marks are not that unusual if we remember that these are books that are in a library, and have been checked out by who-knows-how-many patrons along their long journey. it might be "unfair" to judge uncorrected o.c.r. on such a page, because we cannot expect o.c.r. to overcome marks on a page. but that's the _point_, you see. these _are_ library books, and we can _expect_ that they might be marked on, and we cannot expect o.c.r. to compensate, which is just one more reason why we _need_ to make a good effort to actually _correct_ the o.c.r. *** nonetheless, it's not difficult to find examples of terrible o.c.r. even if we rule out any pages with markings made by patrons... so, for today's example, i'll again use verne's "around the world", and focus on a page that has no such patron-markings on it... *** here's the scan of page 69: > http://www.archive.org/stream/tourofworldineig00vern#page/69 and here's the o.c.r. of the entire book: > http://ia310838.us.archive.org/0/items/tourofworldineig00vern/tourofworldin… and here's the o.c.r. of page 69: > > Thii Tour of the World in Eiyldy Days. 69 > > " Be so good, landlord, as not to swear, but > remember this : cats were formerly considered, in > India, as sacred animals. That was a good time." > *' For the cats, mv lord 1 " > *' Perhaps for the travellers as well ! " > After which Mr. Fogg quietly continued his > dinner. Fix had gone on shore fchortly after ^Ir. > Fogg, and his first destination was the head- > quarters of the Bombay police. He made himself > known as a London detective, told his business > at Bombay, and the position of affairs relative to > the supposed robber, and nervously asked if a > warrant had arrived from London. It had not > reached the office; indeed, there had not yet been > time for it to arrive. Fix w^as sorely disajDpointed, > and tried to obtain an order of arrest from the > director of the Bombay police. This the director > refused, as the matter concerned the London of- > fice, which alone could legally deliver the warrant. > Fix did not insist, and was fain to resign himself > to aw^ait the an-ival of the important document ; > but he was determined not to lose sight of the > mysterious rogue as long as he stayed in Bombay. > He did not doubt for a moment, any more than > Passepartout, that Phileas Fogg would remain > there, at least until it was time for the warrant > to arrive. > > Passeparto\it, however, had no sooner heard his > master's orders on leaving the Mongolia, than he > saw at once that they were to leave Bombay as > they had done Suez and Paris, and that the journey > would be extended at least as far as Calcutta, and > perhaps beyond that place. He began to ask him- i count at least a half-dozen misrecognized words there, and another half-dozen other o.c.r. errors of various kinds. -bowerbird

1 0

Re: Internet Archive makes more than 1 milllion books available online to blind, dyslexic
by Bowerbird＠aol.com 16 May '10

16 May '10

when are people going to start calling "bullshit" on the press releases from the internet archive? that's what i want to know. look at their text. it's raw o.c.r., and it's _awful_. and unlike google, they're doing nothing to fix it. but they're putting it out for the "blind and dyslexic"? is this some kind of mean trick they're trying to play? do they think blind people won't notice the bad o.c.r.? i don't know who's calling the shots over there, since they all _seem_ like nice people... but my goodness, the cynicism of these press releases is overwhelming. and all this "good publicity" that they are getting will soon exercise its karmic force, and bite 'em in the butt. it's past the point of embarrassment; it's _dangerous_. even if the press will print your lies, it's wrong to lie... -bowerbird

3 2