- gutvol-d - lists.pglaf.org

the art of the book -- #01
by Bowerbird＠aol.com 11 Oct '11

11 Oct '11

as i said last week, i am considering whether to mount another demonstration here, using "the art of the book", a book that's currently over at distributed proofreaders, where it's now gone through the second proofing round. here's a brief preliminary report on the progress so far... the book consists of about 2500 lines in the o.c.r. preliminary indication is that ~80% were correct, as roughly 20% -- about 500 -- were changed by p1... and some 20% of those, ~100, were changed by p2. i haven't examined any diffs yet -- just counted 'em, but i can report 214 spacey quotes were in the o.c.r. p1 found and fixed 208 of those, missing a mere 6. p2 found and fixed 4 of the 6 that were present, 66%. of course, since almost every single one of them could have been found and fixed by automatic pre-cleaning, all of the time and energy that was spent was _wasted_. but i've already proven that fact here countless times, so there's no reason to go over the details once again. the main point is that a large percentage of the o.c.r. -- even full pages -- was totally correct to begin with. -bowerbird

2 1

Re: [gutvol-d] just sayin'
by Bowerbird＠aol.com 10 Oct '11

10 Oct '11

greg said: > Paranoia averted. have you noticed how "scenario" and "paranoia" share so many letters? -bowerbird

1 0

just sayin'
by Bowerbird＠aol.com 09 Oct '11

09 Oct '11

very interesting. this list has gone without spam for what, about 8 years? now it supposedly gets hit with 2 pieces within 5 days? i was suspicious after the first, but didn't say anything. i will now, though. you will remember that i said that the attack-pack here tried many times to get me "banned" from this listserve. those were the _public_ efforts. there were also several _backchannel_ efforts, because two of the favorite tools of any attack-pack are the back-stab and the blind-side. (new posters to this listserve over the years have _often_ e-mailed me to report they got several "warnings" about me backchannel from people on this list. imagine that!) but michael hart always protected me from "ban" efforts. so naturally, i always wondered what would happen when he passed away. the attack-pack would surely strike back. the lobsters had fallen silent, but we all know their _rage_ was still there, and it grew every time i turned up the heat. so now, with the water at full boil, it is extremely intense... and now michael hart has gone to his reward. so i've been waiting for "the move". with that as background, let me lay out a scenario for you. *** more spam hits the list. people ask "hey, what's up with this spam?" the people in charge of the list -- you know, the ones who ask you for your password if you wanna see the archives -- say "we'll probably have to have a moderator to pre-approve any posts from new subscribers"... and one will be chosen... the moderator will draw up a "policy" on how the decision is made as to whether a post by a new subscriber is permitted. this "policy" will have a raft of bland legal/moral boilerplate. then someone will ask "why are new subscribers subjected to a standard that doesn't also apply to the existing members?" there will be sage agreement amongst the people in charge that this is a good question which can only be answered by an application of the policy toward existing members too... because -- you know -- we want to be "fair" and everything. and thus will be born a tool to let the attack-pack "ban" me... and you, if you have the audacity to challenge their authority. because if they cannot silence their critics with sound logic, they will stoop to silencing them by use of the "ban" tactic... -bowerbird p.s. on another front, distributed proofreaders just installed a "code of conduct". i guess misconduct over there is rampant.

3 3

Just looking around
by gutvol-d＠pglaf.org 09 Oct '11

09 Oct '11

ciao, my best man! Love consists in this, that two solitudes protect and touch and greet each other. www.matchforyou.ru Hello sweetheart! I'm very fragile, delicate girl. Have hazelnut color eyes, thin lips and beautiful little nous, dark-cherry hair with shortcut like a boy. Have ear piercing. Actually I'm not very romantic person. Usually I have to be strict in the office cause I have a management position. But when I'm at home I like to relax a little, and to be subordinated to someone. It is so hard to be strong business woman all the time. I want to find a man who will be able to protect me and make my life easier! I would like to meet my beloved one, special one, best friend and loving person, who is merry, kind, tender, affectionate and liberal. sweety, thanks and have a good day

1 0

Re: [gutvol-d] some questions for brewster kahle and archive.org
by Bowerbird＠aol.com 07 Oct '11

07 Oct '11

alex said: > I've been crazy with school, work, > and possible testicular cancer. take care of yourself, especially your health... those millions and millions of e-books full of embarrassing dirty text ain't goin' nowhere... we'll talk next week... -bowerbird

1 0

Re: [gutvol-d] some questions for brewster kahle and archive.org
by Bowerbird＠aol.com 07 Oct '11

07 Oct '11

alex said: > That one wasn't so hard, keep trying, "old man!" :) sure thing, kid... ;+) > I have to say I agree with you entirely here. > It's something I really want the archive to fix good! > I will certainly (and delicately, heh) bring this up > at the next collections meeting with my boss, > to see what/if she thinks we might do. well, i never had the ability to sweet-talk a boss, so i'll leave that bit up to you, for the most part... but odds are that she will give you the "standard" response of "there's not much we can do about it"; and, sadly, she might really believe that... many do, which is precisely why the problem has been ignored. but that's wrong -- completely wrong -- dead wrong... there is a _lot_ that can be done to improve that text. let's take a good look at it, and we'll see why. first, probably the most distracting thing about it is the presence of the runheads, a terrible irritation, both esthetically and visually and in reader-attention. yet it's very easy to get rid of 'em, even automatically. that, all by itself, would be a tremendous improvement. pagenumbers are much the same. and spacey punctuation is a _constant_ in these books, yet it too is a problem that can be fixed automatically... again, the application of merely these _automatic_ fixes would do much to improve the texts, at least to the point where the users would no longer feel _insulted_ by them. moreover, as my recent series (not to mention all of the similar series i've done over the years) has clearly shown, the rest of the steps that need to be taken to clean a text are extremely straightforward, and elementary to code, such that a _system_ for _end-users_ to _fix_the_texts_ could easily be put in place, for only a minimal expense, which is something archive.org shoulda done years ago... in return, archive.org gets text that it can be _proud_ of. -bowerbird

2 1

these grapes are sweet -- lesson #25
by Bowerbird＠aol.com 07 Oct '11

07 Oct '11

and so we come to the conclusion of our lessons in this thread, where i've explained how to digitize a book quickly and easily... you can find the materials we used for this series of lessons by searching archive.org for "booksculture00mabiuoft" for the book. *** in our lesson before this one, i showed you this .pdf: > http://zenmarkuplanguage.com/grapes124a.pdf but to illustrate for you the process of _customization_, i have generated another .pdf, using other preferences... > http://zenmarkuplanguage.com/grapes125b.pdf this one uses a different font (verdana) in a bigger size. i think if you view the two, side by side, you will find that you have a definite preference for one or the other -- or perhaps you like the font in one and the size in the other. the point is, if we give end-users the capability to generate a .pdf which they customized to their personal preferences, that's clearly going to be the optimal experience for them... there are a lot more demo .pdfs which i could generate, not just different customizations, but of different types. for instance, these 2 .pdfs were created to be free-flowing. but we could generate one which retained the linebreaks and the pagebreaks which matched the original p-book... that .pdf would match the page-by-page website which i showed you in the previous lesson, and we could even do some things which _created_synergy_ between the .pdf and that website. specifically, we could put a web-link on each page of the .pdf that opens the respective web-page for it. if we then put a discussion-box on that web-page, then the offline .pdf would be a tool complementing an effort to provide a social-networking component to the book... you could read offline, but jump online to write or read annotations for a certain page if you felt moved to do so. i'll discuss some of this stuff at a later date, when i do a new "sweet grapes" series on the topic of my e-book tools. *** we'll recap in a minute, to finish this, but first: _caveats!_ the code i've shared with you in this thread is not "solid". it's meant to demonstrate one thing -- only one thing -- namely, that the code necessarily to accomplish the task of cleaning o.c.r. and making e-books isn't hard to write. indeed, if you are willing to write quick-and-dirty code, like the code that i wrote in these lessons, you can do it with a minimum of time and effort, even if you're doing that coding in a language that you've never used before, which i demonstrated by using a language i'd never used. the programming constructs you need are the simple loops and string operations that you can execute in any language. there's nothing difficult about this, nothing, in the slightest. having said all of that, though, quick-and-dirty code is still dirty code. it's not robust to violations of your assumptions, so it's fragile. and it most definitely won't do everything that you will find -- eventually -- that you need to be able to do... sometimes the code i wrote was outright bad. not in the sense that it had bugs -- it probably did, but for the most part, it all worked good enough to do the job for this book -- but it was most definitely not the _best_ way to write all those routines... sometimes i wrote it "the wrong way" because it was easiest to do it that way, and sometimes i wrote it "the wrong way" because i don't care to share all the hard-earned experience that i've gained over the years with the enemies i have here, and often i wrote it "the wrong way" just to make you think... i do have code that is solid, and capable, and flexible, and i know without question that it works across _lots_ of books, because i _developed_ it across lots of books. if i had wanted to give you the best code i could write, i'd have given you that. that wasn't my goal. my goal was pretty much the opposite, to share the _worst_possible_code_ that would _do_the_job_. so i wrote everything "from scratch", looking at no other code. and the bottom line is that you shouldn't "just trust" that code. use it and learn from it, especially what there is about it that enabled it to "work", in the sense that it "did the desired job". but don't "just trust" it. and if you do use it on some other book, know that you must examine the output closely, to see _if_ it worked correctly for _that_ book; you must find ways to _test_ for correct behavior. it's also the case that the _order_ in which i did the tasks was perhaps not optimal. i was _teaching_ and _demonstrating_, so i followed the order which optimized _that_, not "the job". and if you want to go through this exercise again, with me, on another book, to see what it'd be like if we did it again, i would suggest that we use _this_ book as our raw material: > http://www.pgdp.net/phpBB2/viewtopic.php?t=49047 it's going through d.p. now, so we'll have another digitization which we can use for comparison, so as to evaluate our work. *** now for the recap... this thread shows how to digitize a book quickly and easily. we've gone from raw o.c.r. to finished e-books, using only your wordprocessor and code written by this python virgin. congratulations, friends. now you know how easy it can be. and you will never again let anyone snow you with the _lie_ that it has to be some complicated and difficult procedure... it's a matter of running spellcheck, to correct o.c.r. errors, and doing a few consistency checks, to fix some other stuff. will a couple of "stealth scannos" survive? yes, it's possible. and you certainly won't catch all of the punctuation glitches. but if you ask your end-users to contribute error-reports -- and then _act_ on the reports _promptly_ (unlike p.g.), and give your reporters full credit for finding the errors -- you will soon discover that they'll be delighted to help you. plus, after your text is cleaned, you can just click a button to generate the three "canonical" output-formats you need. then distribute the text as-is -- that's right, the .zml file -- to allow your end-users to generate their own output files, personalized to their own idiosyncratic needs and desires... that is the way the cyberlibrary of the future should work! and thus concludes this thread of lessons. have a good weekend... :+) -bowerbird

1 0

if you want to see just how bad it could get
by Bowerbird＠aol.com 07 Oct '11

07 Oct '11

if you want to get an inkling about just how bad it could get if we let the copyright lobbies have their way, read this interview about copyright over in the arena of classical music... > http://www.techdirt.com/articles/20111006/12220616236/lawrence-golan-speaks… you'll see the reality of unbridled greed. -bowerbird

1 0

these grapes are sweet -- lesson #24
by Bowerbird＠aol.com 06 Oct '11

06 Oct '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** first, let's clear up that loose end about the final spellcheck... to recall, we approached our unique dictionary with grapes116: > http://zenmarkuplanguage.com/grapes116.py > http://zenmarkuplanguage.com/grapes116.txt you'll find it as the long list of words at the end of the output. we saved that file here: > http://zenmarkuplanguage.com/grapes.all.unique.dictionary.txt now we essentially repeat grapes116.py with grapes120.py: > http://zenmarkuplanguage.com/grapes120.py > http://zenmarkuplanguage.com/grapes120.txt the difference is, we're no longer using the regular dictionary, the "specials" dictionary, or the dictionary with british words... (if you look, you will see that they have been commented out.) so we _only_ use the all.unique dictionary -- words in this book. (we split it up into a "specials" part and a "regular" part, because _specials_ are case-sensitive, while regulars are case-insensitive.) now, the first time (or two, or three) that you run grapes120.py, there still might turn up a "notfound" word (or two, or three), but you just add 'em to all.unique and run the script again, until you finally get an output that tells you that no words were "notfound". that's the signal that your all.unique.dictionary is now complete... (if you're curious about what didn't turn out right in the first run, it was the roman numerals i, v, x, and xi, which should have been case-sensitive, along with a couple of initial-capped possessives.) *** and yet another loose end, brought to the surface by roger's questions about coding a page-by-page viewing apparatus... here's code that creates such a capability: > http://zenmarkuplanguage.com/grapes124.py > http://zenmarkuplanguage.com/grapes124.txt season to taste... you'll find the set of page-by-page webpages in this folder: > http://zenmarkuplanguage.com/grapes124-folder *** ok, so on to .pdf, and our finish-line... i explained why i'm not sharing my code for .pdf creation. but you can see a .pdf which i created: > http://zenmarkuplanguage.com/grapes124a.pdf this .pdf was output using one specific set of preferences. i might generate others, to demonstrate customization... or i might not. but either way... this ends our task of generating our trio of output-formats, which also completes the purpose of the thread in general... we've gone from raw o.c.r. to finished e-books, using only your wordprocessor and code written by this python virgin. we'll recap the whole thread in our next, and final, lesson... so if you have questions, now would be the time to ask 'em. -bowerbird

1 0

do you want to look backward? or move forward?
by Bowerbird＠aol.com 06 Oct '11

06 Oct '11

any discussion of "what .epub supports" is dishonest dialog. because the truth is that different .epub viewer-programs "support" different subsets of the "official standard", and thus the _reality_ of the format is that it's one huge mess, a "standard" that's been embraced and extended to death, whenever it wasn't locked-down with d.r.m. to begin with. and that is exactly how the corporate publishers _want_ it. because they're trying to stave off the revolution that will short-circuit their cash-registers and ruin their business. so what are they doing with .epub? why, coming out with _a_3.0_version_, of course, the better to boggle you with. you think things were confusing and fragmented _before_? well let's throw audio and video into the mix too, because the last decade of the web _proved_ they cause infighting. throw in some javascript, and html5, maybe flash as well, in order to _ensure_ that e-books are a nasty experience. it's no accident that amazon outpaced i.d.p.f. even while using a format that was supposedly "inferior" to .epub... but the end-run is inevitable, so all i.d.p.f. can do is stall it. study the "history" all you want, most especially if you want to distract people from _the_future_. but if you want to go _forward_, we need _a_clean_break_ from crap like .epub... -bowerbird

3 2