January 2012 - gutvol-d - lists.pglaf.org

Re: [gutvol-d] Amazon releases new KF8 and MOBI tools
by Jim Adcock 14 Jan '12

14 Jan '12

>Not sure what "we all know." Are you saying Kindle Fire is only internally HTML 3.2? Are you saying KF8 is only internally HTML 3.2? Also, reading the Amazon documentation on the KF8 release they seem to be saying that they are planning new releases soon of the software in the "current generation" Amazon eInk devices to make them KF8 compatible. How compatible remains to be seen.

1 0

Re: [gutvol-d] book of james -- 010
by Bowerbird＠aol.com 14 Jan '12

14 Jan '12

james said: > I concede your point about ABBYY Fine Reader. well, first of all, there's no need to "concede", because i'm not trying to "win" an argument... i'm trying to help you figure out what you are doing wrong, to make you feel like every book is the last one that you're ever gonna digitize... even as you lay out plans to do a dozen more... :+) so i will have much more to say, later, but i want this file off my plate, today, if i can find a way to make some time, so i will have to delay the temptation... james, you're taking all this really great. -bowerbird

3 2

Amazon "Send to Kindle"
by James Adcock 13 Jan '12

13 Jan '12

Amazon has created a new app for your desktop or laptop which makes it easier to send an (unencrypted) book file from there to your Kindle. Free if you use wifi. You do not have to save what you send in Amazon's cloud. Useful during book dev, or if you download a book from PG via your computer, and then want an easy way to transfer it to your Kindle. You don't have to do it this way of course. You can use a USB, or a local wifi server, or use the Kindle web browser to download directly from the web, or .. Just right click on a file and send. This solution is not useful for people who do not use Amazon/Kindle. http://www.the-digital-reader.com/2012/01/12/amazon-releases-sendtokindle/#m ore-28952

1 0

Re: [gutvol-d] book of james -- 010
by Bowerbird＠aol.com 12 Jan '12

12 Jan '12

james said: > I did some work for One Laptop Per Child. yes, i saw that. > http://www.archive.org/details/EBookEnlightenment haven't seen that yet, but i'll look at it when i get a chance. > The goal of this book is to get children and teachers > in poor countries to digitize their own books. right on. right arm. > Again, cheap and time-consuming > is better than expensive and efficient. but we can agree that cheap and efficient is _best_, yes? > Kid's labor is worthless i'm sure you didn't mean that the way it sounds... ;+) > and doing it the hard way > might even be educational. if you emphasize the "might"", i might agree. but it's beside the point, right? the point is that we do things that are educational, and whether they are easy or hard is immaterial. the important thing is that _we're_learning_. there's a time and place for _difficulty_ too -- when we want to instill capabilities for discipline and tenacity and persistence and self-confidence and asceticism and resolve and dedication and a sturdy work ethic -- so i have full appreciation for its qualities, but a love for difficulty _per_se_ is twisted. > (I'm assuming that the kid is > doing this because he *wants* to do it. > I don't advocate child labor). and thus you touch on what might well be _the_ most important issue -- motivation. i have found -- most especially for kids -- _a_sense_of_mastery_ is one of the crucial -- even _vital_ -- sources for motivation... if you feel like you're good at something, you are much more likely to want to do it. likewise, when you feel you're not good at something, you're unlikely to want to do it. so, do you feel you're a master of digitizing? or more like you've been beaten down by it? and another thing lurks right behind mastery -- so close it may be its twin, or its shadow -- namely _a_sense_of_fun_... it almost sounds redundant to say that people are much more likely to want to do things they think are "fun". the linkage is so tight it's almost tautological. similarly, if you're good at doing something, you're probably gonna come to think it's fun. and if you find something fun, you will do it, and the experience will lead to more mastery. so the intertwine of these three factors is quite compelling and creates a fine braid. and "difficulty" is not present in the picture. *** so, are there difficult parts to book-digitization? yes, without any question. but _on_the_whole_, if book-digitizing is hard, you're doing it wrong. and people who tell you otherwise... are wrong. -bowerbird

1 0

Re: [gutvol-d] book of james -- 010
by Bowerbird＠aol.com 12 Jan '12

12 Jan '12

james said: > I'm not exactly sure what > "start from scratch and do the job correctly" > would mean in my case. well, if it's any consolation, i didn't mean to imply that anything done "incorrectly" was _your_ fault... mostly this is a botched job by the internet archive. they did a lousy job of scanning the book, and then their stupid workflow went and sabotaged the o.c.r. so i consider you to be an innocent victim of them... the one thing you did do wrong was the rewrapping. but again, your guide in that was a misguided policy from project gutenberg, which _might__ have made some kind of sense back when it was established, but has outlived all of its usefulness by now, to the point it's a bad practice which can only be preached against. so, you know, once again, you are the victim here... having said all of that, though, my advice to you has not changed since i first gave it to you a while back, which is that you would be better off starting over... you could spend a little time cleaning up the scans, and then ensure that the o.c.r. gets done correctly, with the options specified intelligently, and output saved properly, and _still_ finish the thing _faster_ than continuing to work with the file as it now exists. (and that's _after_ we've both put in a bunch of work!) i'm just about done doing my part, so i no longer have a dog in this fight, so my advice is for _your_ benefit... nobody else here will care. they aren't being affected. but that's my advice. for you. and i feel some kind of urgency and obligation to tell everyone down the line to avoid a situation like this... because it simply doesn't need to be _this_ difficult... you're willing to take on the burden, and that's fine, but even then, it's my opinion that your time would be better spent doing four books the easy way than one the hard way. > I do not own and don't intend to buy ABBYY Fine Reader. i guess mr. haines didn't convince you with his argument that -- at a value of $50 an hour for your time -- it would _pay_ for itself very quickly... :+) truth is, even at $1/hour, it will pay for itself soon enough, if you think you're gonna keep doing more digitizations... you can also pick up a "used" copy at a really low price... and it's not just the money; it's your _sanity_ you're saving... but that's beside the point, too. there are enough people here that _do_ own finereader that you should be able to find one of them to do you the favor of doing the o.c.r. for you. you could do them a favor back... heck, if you woulda asked me, in the first place, i would have done it with my mac copy of finereader (which is way behind, yet would still probably give you better results than you have). but you wanted to pursue the file you had, so i went that way. if i had it to do over, however, i would reverse that decision... > I did attempt to make my own page-at-a-time OCR > with Tesseract yeah, tesseract is an even bigger waste of time and energy. > Obviously room for improvement, but > much better than what I started with. there might be different classes of trash, but it's all garbage. don't start comparing one pile of garbage to another one, or you're just gonna convince yourself to end up with garbage... > Putting in em-dashes and accents is painful, > but realistically what alternative do I have? re-doing the o.c.r., so those things are _retained_, instead of being thrown away. that is exactly what i am saying, james... you _did_ have an alternative. but you choose not to use it... because you didn't want to throw away the time you'd spent. so, instead, you will end up throwing away even more time, working on a decidedly-inferior file. i told you that goin' in, and now that i've spent a bunch of time with the file myself, i can confirm that my original advice was extremely sound... and i shoulda stuck with it, until i eventually convinced you. i took up the challenge because i knew i could learn from it, and i did, so it wasn't a waste of _my_ time. but i can tell you that you'll be wasting your time, doing unnecessary hard work. and let me reiterate: you _still_have_ that better alternative! you can still find someone to do the o.c.r. for you, and you can clean up the scans first, so they give you better results. and then, because you will not do the stupid stuff that was done by internet archive, you won't have to waste your time undoing the damage that they did. you still have a choice... and it's not just you, james. there are a whole lot of people who're beating their heads against the wall to digitize books, because they think they have "no alternative" to head-beating. > My understanding of what you're currently trying to do > is get my partially corrected text to match line-by-line > and page-by-page with the page images. i'm not "trying" to do it. i'm actually doing it. and i'm also doing _some_ cleaning of the text, and a lot of formatting. but you're going to have to do a whole lot _more_ cleaning... much of the work will be painful, and some of it unnecessary. oh, and just for the record, i have spent a good many hours getting those pagebreaks and linebreaks back like they were. it might not have taken you more than a minute to rewrap it, but it's taking me hours and hours and hours to undo that... it's not a job that i _like_ doing... in fact, i totally detest it... and the reason i hate it so much is that it is _unnecessary_... you should have never rewrapped that text before proofing. but let me tell you why i did that job. i did it for you, james. i did it because i knew that every hour i spent "fixing" that would save you 3-to-7 hours of proofing time, and _sanity_. so i _spent_ my own time, james, so you could save _yours_. (i wouldn't have done it if it would've been a 1-to-1 payoff; but with a 3-to-1 payoff, maybe even 7-1, i decided to do it.) so i think i've "earned" the right to give you some good advice. (as if i really had to earn something like that in the first place.) the important thing is that you learn that you should _never_ do a book this way ever again. it's simply not worth the work. > Once you have that I'm supposed to go through > page by page and change the text to look > exactly like the page images. i will have done most of that by the time i hand it over to you. once the original linebreaks are restored, it's a piece of cake. that's why i was able to use the text from the internet archive to show well-formatted model-pages right from the outset... > I add underscores for italics, right, and that's something you shouldn't have had to do, since abbyy does a good job of catching most of the italics. but then the internet archive went and threw them away, along with the em-dashes, which you must also restore... in the future, i'll make available an app that will go and grab that information -- it's sitting there, in one of the data-files at archive.org -- so people don't have to suffer because of a bad decision made outside of their control, and i could've helped _you_ to avoid that unnecessary work as well, but... one thing you will not have to do is to worry about diacritics. at least for the names you've done thus far, i can capture 'em, and make a global search-and-replace to put in the diacritics. but that's another thing you shouldn't have had to worry about, because i'm pretty sure abbyy can recognize those characters... > but I do not re-wrap the text, remove page numbers, > join paragraphs split between pages, or anything else > I've been doing so far. that's right. don't do any of that. the machine can do that. don't waste any of your time doing what the machine can do. > I just correct spelling and accents. no accents either. but it's more than just "spelling". indeed, the "spelling" part is the easy part, because the spellcheck routine flags most of the bad words. but where the bulk of the work exists for this text is in the correction of the punctuation. the scans were bad, so that means the o.c.r. of punctuation is bad... but the problem is exacerbated because the book was typeset very poorly, with lots of punctuation glitches... and the exacerbation continues, because many of the punctuation glitches _can't_ be found with algorithms. and, just as another example of a task that took hours, the doublequote checking was a bear, because the o.c.r. was bad, but -- even worse -- because the typographer made lots and lots and lots of doublequote mistakes... this includes a failure to mark continuing dialog with an open-quote at the start of subsequent paragraphs. so i installed most of them, to the best of my abilities, but it's very difficult sometimes to make that judgment when you're not actually absorbing the book's content. and this means i also introduced lots of doublequotes that were _not_ in the p-book itself, and you might be uncomfortable with that, and want to go delete them... (which is fine, by the way; i have no attachment to 'em; if you decide that's what you want to do, i can probably write a routine that could locate them rapidly for you.) but most of your work will revolve around punctuation. and whether you'll want to standardize inconsistencies in the book's typography, some of which are... weird... and this is picayune stuff. it's... _irritating_. not fun. now, this punctuation work would have been necessary even if you would have had reasonably good o.c.r. text. but after having done all the unnecessary work required, i'm not sure that i would have the stomach to be able to face the stresses of handling those types of judgments... so i suggest that you gird yourself... > After I've done all that I go back and put in > the ZML stuff like extra blank lines for page headings. again, most of that will have already been done. > Then the magic happens and we get several formats > of the book from one source. I'm not entirely clear > on that part. you don't need to be clear. you just need to click a button. -bowerbird

2 1

book of james -- 010
by Bowerbird＠aol.com 11 Jan '12

11 Jan '12

james- work is coming along fine on the file, but i'm sorry that i haven't gotten it turned back to you sooner than this... i got caught up in some other things, plus the clean-up is taking longer than i'd hoped it would, mostly because the typography of the book really wasn't quite up to snuff. in the meantime, you can always work on the family-trees. *** to make this post more widely applicable, i can probably reiterate the point i usually describe at the very beginning: take care to select a well-done book which will scan nicely. or, if you're using pre-existing o.c.r., make sure it's good. otherwise, you might well have to spend a lot more time. for instance, on this book which james is digitizing now, he faces the triple whammy -- poor p-book typography, bad scans, and internet archive's careless text handling (where the book is missing em-dashes, utf8, and italics). so even after all the time he and i have spent cleaning it, my guess is that he would still be finished with it sooner if he were to start from scratch and do the job correctly... -bowerbird

2 1

so, is anybody gonna tell him?
by Bowerbird＠aol.com 11 Jan '12

11 Jan '12

so, it's been a week now... is anybody gonna tell him? or not? -bowerbird

1 0

Re: [gutvol-d] book of james -- 009
by Bowerbird＠aol.com 10 Jan '12

10 Jan '12

james said: > I assumed that you were writing > software that could be used by anyone. > I didn't think that you were > putting together something that > would only be useful to one person > working on one specific book. > Because who does that, really? well, you've raised some good philosophical questions which i might wanna tackle later... but for right now, to answer quite exactly... yes, james, as crazy as it might seem, i _am_ -- indeed -- writing this editor-app for you, one person, to use on this one specific book. :+) and, believe it or not, that was fun and easy, compared to the mess of cleaning this text... so i'll go do that, so i can hand it back to you. :+) -bowerbird

1 0

Re: [gutvol-d] book of james -- 009
by Bowerbird＠aol.com 10 Jan '12

10 Jan '12

james said: > I've been reading the PDF of the book on a > second computer when I've worked on editing the pages james, james, james, james, james, james, james... i've been jumpin' through these hoops to try to figure out how to fit this on one screen for you, and you just forgot to mention that you've been using two screens all along? aarrgghh... you're reminding me why i hate supporting users! :+) (which, by the way, is a _great_ thing for me to remember, really an excellent thing, so i _thank_you_ -- sincerely! -- for bringing these lessons back home where i live, because i was honestly thinking i would release some software soon, and now i will have to engage in a reconsideration of that.) ok, so anyway, _problem_solved_! keep using that .pdf on the second screen, when you need. now, if you'll excuse me a bit, i need to go off and mutter... -bowerbird

2 1

book of james -- 009
by Bowerbird＠aol.com 10 Jan '12

10 Jan '12

now for today's fun and games with james... :+) doing the spellcheck and having all of the names pop up in front of my face gave me a great idea... those diacritics are a pain in the wazoo, and i have been trying to figure out how to work around them, when it occurred to me that i could eliminate them. not entirely, mind you, as james wouldn't like that! but i could certainly global-change the names from the diacritic versions into friendly 7-bit equivalents, and then global-change them back when i was done! same with _any_ word with a diacritic in it, actually... so i pulled all of 'em out, and then did the big switch. oh boy, that worked nicely... plus, as a side bonus, of course, i got rid of that pesky validation warning. but i did the sleuthing to puzzle it out anyway, and it's being caused by a couple specific names, james, but just a couple of them, so it's easy enough to fix. (thank you for your detective work at stack overflow; i had solved the problem, but that is good to know.) anyway... reducing all of the text to a 7-bit version will make it _much_ easier to rewrap the linebreaks to their original state, so i'll tackle that tomorrow... now the file validates nicely. > http://zenmagiclove.com/bhaga/globalchange.html that also helped me to locate some unbalanced italics, which are now fixed. *** james said: > a screen grab I made following your instructions. > It is actual size. except that "actual size" doesn't mean anything when someone is looking at that image on another monitor. > As you can see, it is much too small to work with. no, that's the point -- i _can't_ make such a judgment. and sending me a screenshot of a view that _doesn't_ work doesn't give me any information that i can use... here's the problem, described in words... the screen needs to be split, between text and scan. for now, i'm only doing the left/right type of split, so i need to know where to draw the line between them. the text must be a big enough fontsize to be workable -- a judgment that only you can make for yourself -- and if the text-side is too narrow, the lines will wrap, and destroy your capability to compare text with scan. so those are the constraints on that side of the screen. on the other side, your willingness to scroll the scan means that we will make it as wide as we possibly can, and then make it as high as the aspect-ratio requires. is all of that clear? so, when you adjust the size of the text, by clicking the "smaller" or "bigger" button until it's acceptable -- you want it to be _big_enough_ so you can see it well enough to do your editing on it, but not too big, since horizontal screen-space is extremely valuable -- what number is in the title-bar on your last click? (your title-bar will say something like "smaller=88"; that number will also be in a box at the bottom-left.) and then, after your text is the right size, click the "widenscan" button, repeatedly, to upsize the scan, until it "disappears". (it really drops under the text. and you need to go find it there to find the buttons.) then click the "narrowscan" button until it "reappears" at the top of the screen directly across from the text. now tell me what is the number in the title-bar then? (the title-bar will be something like "narrowscan=49".) sorry this question seems to be so hard. had i known, i woulda tried to devise a better way to get an answer. *** > It will take me awhile to get to that word list. no need now... i'll get you a much briefer list of some possible scannos, and that's all you'll need to review... -bowerbird

2 1