- gutvol-d - lists.pglaf.org

Re: [gutvol-d] A novel suggestion!
by Bowerbird＠aol.com 21 Nov '11

21 Nov '11

keith- that "suggestion" isn't "novel" -- it's downright bizarre! but hey, since "tools for doing the initial proofing" are _simple_ to make -- as i've demonstrated many times -- how about _you_ make some, and then get together with those other three individuals to make your neato system? -bowerbird p.s. i'm tempted to give your post one of my $2 prizes for "a positive contribution" by being such a stirring example of being bizarre, but i'd be afraid of setting a precedent...

1 0

Re: [gutvol-d] a review of some digitization tools -- 003
by Bowerbird＠aol.com 21 Nov '11

21 Nov '11

ok, you'll remember our three overarching requirements: 1. obtain the text (via scanners, and o.c.r. software) 2. clean the text (the first set of tasks for our tools) 3. turn the text into e-books (the second set of tasks) under the second of those requirements, we covered: 2a. do a spellcheck 2b. fix spacey punctuation 2c. restore styling, e.g., italics now we'll work on the list of tasks for the third one: 3a. tagging the structural aspects of the text 3b. converting the text into .html (intermediate and final) 3c. converting intermediate .html into e-book output-files our biggest task here is 3a, tagging the structural aspects... so let's talk about what it takes to "tag" the elements of a book, where the tags describe the _structure_ of the specific element. the first thing you need to know is that -- for many books -- the number of types of tags needed is embarrassingly small... indeed, here are the big 5... 1. paragraphs you need a paragraph tag -- [p] -- at the beginning of paragraphs. (i'll use square brackets in this post, rather than angle-brackets, just so nobody gets confused, as i'm speaking in general terms about marking the elements, and not about .html coding per se.) depending on the flavor of the markup, you might need to have _closing_ tags, which might be a [/p] at the end of a paragraph. similar in function is the [br] break command, to force a linebreak. 2. blocks which should not be rewrapped some blocks -- such as poetry -- should not be rewrapped, so you need to mark them, so they'll retain their formatting. a quick-and-dirty way is a [pre] tag before, and [/pre] after. 3. headers you need to mark headers, usually indicating the _level_ as well. so [h1] might start a level-one header, with [/h1] ending it, and [h2] and [/h2] surrounding a level-two header, and continuing... 4. table-of-contents links the table of contents in an e-book should link to the proper spots. in .html, these links consist of an [a] anchor with a [href] argument. 5. italics words/phrases which are to be italicized must be marked as such. *** and really, that's all you need for a _surprising_ number of books! i know, just 5 easy things, but believe me, that's often _sufficient_! *** for instance, since nobody has suggested an example book to use, let us examine "the rainbow", by d.h. lawrence, which was digitized for project gutenberg by jim adcock; this book is known as #28948. > http://www.gutenberg.org/files/28948 first of all, jim, thanks for digitizing this important book for p.g. i've taken a cursory look at it, and found a few errors for you to fix. in the .html version, chapter 9 is unfortunately called "chapter ixix". and chapter 11 is called "chapter xii"; that's true in the .txt files too. no need for thanks, jim, i'm happy to help, and won't see it anyway. oh yeah, jim, by the way, your .txt files are badly in error, because you forgot to indicate all of the italicized words with underscores! that's over 160 errors throughout the book, just on those italics... luckily, the italics _are_ indicated in your .html file, so you can just use that file to find where to make corrections to your .txt versions. because i'm sure you don't want to do a disservice to d.h. lawrence. anyway, back to the topic of the thread... i've appended a list of the .html tags that jim used. as you can see, the entire list consists of only the 5 types of tags i mentioned above. of course, we'll want to be able to handle more than just the simple basic structures of a straightforward book with any system we build, but it's nice to know you can _start_ with something simple and still be able to digitize an important book like this one by d.h. lawrence. -bowerbird > [p][/p] > [pre][/pre] > [i][/i] > [br] > > [h1 align="center"][/h1] > [h2 align="center"][/h2] > [h3 align="center"][/h3] > [h4 align="center"][/h4] > [h5 align="center"][/h5] > > [a name="ChI" href="#CI"][/a] > [a name="ChII" href="#CII"][/a] > [a name="ChIII" href="#CIII"][/a] > [a name="ChIV" href="#CIV"][/a] > [a name="ChV" href="#CV"][/a] > [a name="ChVI" href="#CVI"][/a] > [a name="ChVII" href="#CVII"][/a] > [a name="ChVIII" href="#CVIII"][/a] > [a name="ChIX" href="#CIX"][/a] > [a name="ChX" href="#CX"][/a] > [a name="ChXI" href="#CXI"][/a] > [a name="ChXII" href="#CXII"][/a] > [a name="ChXIII" href="#CXIII"][/a] > [a name="ChXIV" href="#CXIV"][/a] > [a name="ChXV" href="#CXV"][/a] > [a name="ChXVI" href="#CXVI"][/a] > > [h3 align="center"][a name="CI"][/a][/h3] > [h3 align="center"][a name="CII"][/a][/h3] > [h3 align="center"][a name="CIII"][/a][/h3] > [h3 align="center"][a name="CIV"][/a][/h3] > [h3 align="center"][a name="CV"][/a][/h3] > [h3 align="center"][a name="CVI"][/a][/h3] > [h3 align="center"][a name="CVII"][/a][/h3] > [h3 align="center"][a name="CVIII"][/a][/h3] > [h3 align="center"][a name="CIX"][/a][/h3] > [h3 align="center"][a name="CX"][/a][/h3] > [h3 align="center"][a name="CXI"][/a][/h3] > [h3 align="center"][a name="CXII"][/a][/h3] > [h3 align="center"][a name="CXIII"][/a][/h3] > [h3 align="center"][a name="CXIV"][/a][/h3] > [h3 align="center"][a name="CXV"][/a][/h3] > [h3 align="center"][a name="CXVI"][/a][/h3]

2 1

looking at the most meaningful indicator
by Bowerbird＠aol.com 21 Nov '11

21 Nov '11

keith said: > It is true that publishers have to > change towards the digital to survive. > Yet, the complete demise of the printed book > is still far off. people conflate "publishers" with "the printed book". what a huge mistake. is there a printer attached to your computer? if there is, and you have enough paper on hand, you can print a book. you could print a few of them, every day, each and every day, for a year or two, enough printed books to overflow your house. and speaking of your house, is there a copy-shop down the way? if there is, it could print enough books in a week or so to overflow every house on your block, and then the next block over, and so on. in the future, every book written has a chance to be "a printed book", not just those which publishers think they can sell for profit, meaning we will have _far_more_ "printed books" than we've _ever_ had before... -bowerbird

4 7

a review of some digitization tools -- 002
by Bowerbird＠aol.com 20 Nov '11

20 Nov '11

keith- most of the points you made in your posts this morning are ones which i feel do not need a response immediately, if at all, so if i get to them this weekend, i'll write a reply... and if not... not... the one exception is this... i said: > > so the first tool you'll use for digitization is > > your old friend, your trusty wordprocessor... keith said: > Are you really, advising people to > use bloat ware, MS and OO. > ;-)))) first, yes, i see the smiley, and i respond under that light... this series consists of reviews of e-book digitization tools. your wordprocessor is the first tool in your toolbox, so yes, indeed, i am really advising that people consider it as such. the wordprocessor serves a number of different functions. it's like a crescent-wrench -- it can fit a nut of any size... for that reason, the crescent-wrench is extremely valuable, undoubtedly the single most-important wrench in the box. > http://en.wikipedia.org/wiki/Adjustable_spanner nonetheless, most mechanics almost always use a wrench that's more specialized for the exact task at hand, such as a socket-wrench with a head that's the same size as the nut. > http://en.wikipedia.org/wiki/Wrench likewise, even though we _could_ be using a wordprocessor -- my "sweet grapes" series in october took that approach -- this series looks at tools _specialized_ for the tasks at hand. that's why much of the focus here is on digitization _tasks_. if you wanna review _tools_, you need to have a good handle on what you want those tools _to_do_ -- what to accomplish. the _overarching_ requirements here are simple to list: 1. obtain the text (via scanners, and o.c.r. software) 2. clean the text (the first set of tasks for our tools) 3. turn the text into e-books (the second set of tasks) i dispensed with the first requirement rather quickly, since that's not a job most of us need to do these days, thanks to the big scanning projects from google and internet archive. the second requirement is also fairly straightforward, in that you _could_ do the job by using your favorite wordprocessor. more to the point, though, was to list the tasks there: 2a. do a spellcheck 2b. fix spacey punctuation 2c. restore styling, e.g., italics once you're aware of the exact tasks, you can evaluate tools. for instance, the proofing rounds at distributed proofreaders exist to complete tasks 2a and 2b. and the formatting rounds have the job of doing 2c, and _some_ tasks in requirement 3. it is this backdrop of knowing the tasks that need to be done which allows me to rail loudly about the inefficiencies of d.p. they're wasting far too many human resources to accomplish these relatively-simple tasks; they could be doing much more. anyway... once you know tasks, you can evaluate the tools. in that regard, let's list the tasks for the third requirement. 3a. tagging the structural aspects of the text 3b. converting the text into .html (intermediate and final) 3c. converting intermediate .html into e-book output-files i'm gonna cut directly to the chase, and tell you that there are scripts to accomplish the 3b and 3c conversion steps, and thus the main focus from here on will be _tagging_the_structure_... at distributed proofreaders, which is important to examine since it _is_ an existing system of the type we want to make -- because the way to test this stuff is in _the_real_world_, and _not_ in the realm of some "theoretical discussions" -- some of 3a is done during formatting rounds, with the rest accomplished by offline, non-distributed "post-processing". a sizable minority of post-processors, including _lucy24_, do their work in a wordprocessor. lucy uses "subethaedit", a collaborative text-editor that truly is lean and powerful. > http://www.codingmonkeys.de/subethaedit/ but most d.p. post-processors use an app called "guiguts". guiguts was stagnant for a long time, but hunter monroe has recently taken on the burden of updating it again, so i'll give him a chance to step in now and tell you about it... hunter, are you here? -bowerbird

3 2

now keith wants me to jump in his tar-pit
by Bowerbird＠aol.com 20 Nov '11

20 Nov '11

keith, you seem to be having "theoretical" discussions on 3 or 4 vague phantom topics, while i'm talking about stuff which either actually _exists_ or has been solidly proposed. and there's no sense continuing when we're so badly out of sync. so let us summarize and finish. 1. i have systems, online and off, which are simple and work nicely. 2. keith is too busy to code stuff, but seems to have many opinions. 3. jim likes using bloatware, but is too cheap to buy commercial-grade. 4. lee wants someone else to do all his .html markup for him first. 5. don knows databases, but hasn't generalized from his d.p. experience the conclusion that any system which requires administrators will become corrupt because of the power-seekers. 6. tar-pits are sticky places. 7. sometimes listserves can be too. 8. i forget what 8 was 4. 9. number 9, number 9... 10. you need 10 items for a top-10. 11. is my favorite number. -bowerbird

3 3

ok let's take stock
by Bowerbird＠aol.com 20 Nov '11

20 Nov '11

given all of the recent topic-shattering, it's only natural to feel a little scattered. so let's take stock, and regroup. first of all, it's _cool_ that people feel free to share our individual e-book workflows. other individuals here might learn from it. and more importantly, we might come to learn something that could be of use to us in building a system for collaborative work. it seems to me that a collaborative system is more in tune with aims of this listserve... and if your workflow depends in a vital way on your own individual experience or talent, it's not gonna be of use to that, not really... likewise, if it depends on a software program (such as ms-word or open-office) which has grown to be a gnarly complex monster, then odds are people aren't gonna want to use it. fine for your use, sure, just not for our use... also, hand-in-glove here, if the app requires a good deal of hand-holding, that's not good. what we want, then, is simple software that takes care of a simple workflow, and which can run either offline or online, invoking a system for collaborative e-book digitization. in this vein, we find distributed proofreaders. they have such a system, and such software... and, for the most part, although they will say that they are not completely satisfied with it and have lots of features they'd like to add, they often turn down coders who volunteer... (when those coders will not be subservient.) so my impression is d.p. is happy with itself. and the same can be said of the p.g. machine. i don't get a feeling they want any help either. they don't want you to meddle with their stuff. in a way, that's pretty liberating, since it means we don't have to support any lingering baggage. we are free to design a system any way we like... i've built the engine of such a system. if people would like me to put it up again, for evaluation, i could do that. (it was the old "sitka" project.) roger frank also coded the engine for a system. he has also taken his project down, but i bet if enough people expressed interest in seeing it, he'd also remount it, so you could evaluate it... *** all these systems are pretty "standard", in that the interface has a text-field and shows a scan. if a review of the scan reveals the text is wrong, you edit the text. it's all very straightforward... other interfaces have been suggested as well. i made one where each text-line is displayed next to a checkbox that should be checked if the line contains any errors, so that that line will later be corrected by the administrators. i made another one where questionable lines get shown in various forms, and the person is asked to indicate the one which is correct. (that's for when multiple files get compared.) we also saw an interface from edward betts on this list recently, where lines are shown under the relevant slice from the scan, and the person clicks any incorrect word to fix it. so the interface can vary as well as the tool... *** it's easy to code an e-book digitization system. since the d.p. code has grown into a monster, people familiar with that code might laugh at such an assertion. but i make it with certainty: it's easy to code an e-book digitization system. i've already done my autumn project this year, so my plan was to wait until spring to do this, but if people want to go to work on it _now_, just let me know, and we can start a project to collaboratively code such a digitization system as an open-source project. since, as i just said, i've already done this, i could serve as a guide. but i'm not going to just hand over my code, so if you want it to be open-source, you'll have to do the vast majority of the coding of it yourself. and that's good, because it means that you can make the choice of the language by yourselves. let me know. if nobody's interested in tackling such a project, i'll do another how-to series, on digitizing tools. in fact, i might do that either way. yes, yes i will. so, by all means, if you want to share aspects of your personal e-book digitization workflow, do! most _especially_ if they might have relevance to a _collaborative_system_ for digitizing e-books... but if your individual workflow is unique to you, you must recognize that it's not of any use to us. -bowerbird

3 2

a review of some digitization tools -- 001
by Bowerbird＠aol.com 17 Nov '11

17 Nov '11

so let's review some digitization tools, shall we? the task of digitizing a book is now fairly simple, thanks to cheap scanners and good o.c.r. today... scanners have been incorporated into printers for a while now, and even the cheaper ones are able to do a satisfactory job on most p-books... also, the best-of-breed o.c.r. app -- abbyy -- is now cross-platform _and_ fairly affordable... but the best news is that you don't even need to do the scanning or o.c.r. yourself nowadays -- if you want to do a public-domain book -- because the chances are very good that google or internet archive has already done it for you. so search around to see if scans already exist. (if you want a book in a non-english language, you might find it at google or internet archive, but if not, there are other sites you can search, and you likely know more about 'em than i do.) you might even discover that a book has been scanned multiple times. that's a good thing... what you should do is look at all of the copies, and find the one with the best-looking scans and (more important) the most-accurate o.c.r. download the scans and the o.c.r., as well as a djvu copy (if available) and/or the .pdf copy. the first thing to do is to spellcheck the o.c.r. when doing this, you must look at the scans! keep in mind that you aren't "correcting" the spelling, per se, but rather matching the book. now, do not take this to a ridiculous extreme. if there was clearly an _error_ in the p-book, you will want to fix it. you might even wanna update some anachronisms, such as changing "to-day" to the version that we all use "today". (i typically delete periods from chapter-heads.) and there might be some changes you'll make in order to make the text work as an _e-book_. (i make sure that the table-of-contents entries _exactly_match_ the chapter-heads themselves, so my automatic link-creator works correctly.) but be conservative in making these changes, because you might change the book's flavor... for instance, in the early days, p.g. would strip out the pagenumbers from an index, "because an e-book doesn't have pages"... well, that's stupid, because those numbers didn't _hurt_ anything, and they might have proven to be quite useful down the line, so they _should've_ been left in, "just in case". don't make that kind of mistake. while you are doing your spellcheck, it is very important that you _add_new_words_ to your spellcheck dictionary -- such as character names, place names, and so on. not only does this mean you only have to deal with them _once_, it also means that when a similar-looking variant comes up, it's probable that it's gonna be a "scanno". adding words also means you can re-do the spellcheck after any/all future edits, with a result that's both quick and clean. that helps catch any errors that you have inadvertently introduced during editing, something which happens to everybody... _after_ spellcheck, depending on the text you have downloaded, you might have to _restore_any_text_styling_, such as italics. if o.c.r. output gets saved as "plain-text", it loses its formatting, including _italics_. so you need to restore it. in most books, there aren't many italicized words, so this is a manageable task, albeit very mundane. if your book _does_ have lots of italics -- and you should check this at the outset -- you might want to re-do the o.c.r. for it... specify that the output be saved as .rtf... and then use your wordprocessor to place _markers_ around all the italicized words. you can do this by using a global change. you can use underscores as markers, or the bracket-i/bracket-slash-i html tags, or another convention you might invent. you'll also want to mark any _bold_ too, but most books contain very little bold, except in their headers, which we will be marking separately in a later process, so you don't need to worry about them now. in this vein, note that what you're marking right now are _individual_ italicized words or phrases. if a whole paragraph is italics, or bold, make a note about it for the future. the other thing you'll have to do, again in your wordprocessor, is to "close up" any "spacey punctuation". for the most part, these are double-quotemarks (and periods and commas) with a space on _both_ sides, so it's rather easy to do a search for them. so the first tool you'll use for digitization is your old friend, your trusty wordprocessor... -bowerbird p.s. if you have a book you'd like me to use as an example during this series, suggest it.

2 1

we don't need corporate publishers to have physical books
by Bowerbird＠aol.com 17 Nov '11

17 Nov '11

keith said: > You call that what comes out of a consumer printer a book! for the most part, yeah. use "half-letter" or a5, for a good size. plus you need to "bind" the pages, maybe put a nice cover on it. but a heavy-duty stapler works well enough to do the "binding", and a wrap-around adhesive slick cover is simple to engineer... the internet bookmobile was doing this a decade ago already... > Nahhh! just a bunch of loose sheets of paper. wrong. > Even if I had a binding machine (true there are copy shops), > and the high quality paper for those nice color spreads > (costs to much for normal consumers) you don't need "a binding machine". unless you want one. and people are already printing full-color stuff regularly, most often missing-dog flyers and garage-sale posters... > why print it? well, there might be any number of good reasons, or at least reasons which the person who is doing the printing considers to be "good enough" to spend the resources to do the printing. but i'd guess that the most typical reason would be because the book means enough to the person that they want to have a tangible -- i.e., _physical_ -- copy of it, as a sort of talisman. you know, the kind of book that was extremely meaningful? the kind of book you _love_? the kind that changes your life? so you want something representing it that you can _touch_. in much the same way that a ticket-stub serves as a "reminder" -- or a souvenir -- of that concert that you and your sweetheart went to for your first date, the night you made love the first time. that's what i would suspect will be the most compelling reason. but i suppose every person will have their own unique reasons... and who are we to tell them that their reasons aren't any good? > I am very happy with sitting in bed, on the couch, > or where ever reading on my 17" lap top. The earth > disapating from its bottom is, almost as good as a warm > fire place(I do not have one). that's cool. (or warm, as the case may be.) and that's _you_. and i am most certainly not gonna tell you that you're wrong. whatever you want to do is fine with me. but likewise i'm not gonna tell anyone else that they're wrong to print out a book. if that's what they want to do, i'm gonna tell 'em to "go for it". > Why would anyone waste all that paper. except that they obviously don't consider it to be a "waste". they consider it to be a good use of resources they paid for. > As for me I do not print anything any more > unless I have to. It is just a waste. i'm very much like you. haven't had a working printer for years. maybe a few times a year i ask my girlfriend to print something on her machines at work, but for the most part, i don't miss it... but that's me. other people do things differently. c'est la vie. > I had a friend (has died), he use to print his emails > and read them from paper. perhaps god struck him down -- for wasting paper. :+( (oops. sometimes i do cross the line. but i always admit it.) > He simply could not get himself accostumed > to reading off a computer screen/monitor. well, see, there you go, another very good reason to print stuff. > Sorry, I think the future should be digital. the future _will_ be digital. correction: the future _is_ digital even as we speak... and it will only become even more so... but we are also still physical beings. who value physical stuff. that has always been true, and will undoubtedly always be true. still, we don't need corporate publishers to have physical books. -bowerbird

2 1

KFires arriving
by Jim Adcock 17 Nov '11

17 Nov '11

FWIW, I received my KFire this morning... ... but still no KF8 documentation nor support tools!

1 1

responding to threads i initiated
by Bowerbird＠aol.com 16 Nov '11

16 Nov '11

ok, i see that lee too is replying to threads i initiated... i don't know if he's responding to me, or to others who have posted in those threads, because -- as i've said -- i have reinstated my policy not to read lee's messages... i have also found that -- unlike keith, who i believe is often "genuinely" confused about the topic at hand -- lee is prone to intentional attempts to confuse things. (amusingly, jim falls into _both_ groups.) so it might not be apparent if lee is responding to _me_ or to _someone_else_. but whatever the case might be, the situation stands. if you would like to see me respond to anything that lee has stated, or questions he's asked -- whether to me personally or to anyone else as well -- or if you think i will be _stumped_ by what lee said and you wanna see me stumped, rephrase in your own words. i will respond, carefully and thoroughly, just like always... otherwise, i'll assume that y'all wish to see these issues die a graceful death, if that is indeed still a possibility... thank you for your time and attention. :+) -bowerbird p.s. some of you might wonder why my antagonists even put out their stuff, since i immediately refute it. the reason is pretty simple... they know that _many_ of the people in their clan have me kill-filtered, thus their target-audience never hears my refutation, and thus believes whatever falsehoods they have planted. but of course the misimpressions inside those heads will die when their bodies die, whereas the _archives_ of this list will persist for a _long_ time in cyberspace, a permanent record of all the recurring dishonesties...

4 3