- gutvol-d - lists.pglaf.org

these grapes are sweet -- lesson #21
by Bowerbird＠aol.com 30 Sep '11

30 Sep '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. lesson #21! we've finally become an _adult_... good for us... *** ok, at last the cleaning is done, so now for some fun... here's the new code for this lesson: > http://zenmarkuplanguage.com/grapes121.py > http://zenmarkuplanguage.com/grapes121.txt so let's review that script and its intent, ok? we have an edited version of the book, containing its runheads and pagenumbers, but we'll want to output a version of it _without_ runheads and pagenumbers. easy enough. it's done. (we've been creating this as an interim text for many of the checks we have done.) you will find this version in the top part of the output. that version still retains the original linebreaks and pagebreaks; how about a version without 'em? done. you see this version in the bottom part of the output. (we used .html to ignore the linebreaks in its source.) that's kind of primitive. no linked table-of-contents, and "tag abuse" in that we used no "blockquote" tags, or "header" tags, but it will do fine for the time being. in the next lesson, we'll do versions for .epub/.mobi. *** further, i will revisit the matter of cleaning the text, doing a debriefing on the experiment we just ran, and showing how to consolidate all the steps that we took. to this purpose, i might take on another book, so if you have any suggestions for what book you want me to do, please feel welcome to let me know, frontchannel or back. *** over at distributed proofreaders, they consider the "proofing" part to be "easy"... it is _post-processing_ where they have their big logjam. which is ridiculous, as we have just demonstrated. once the text is clean, creating e-books should be a matter of a button-click. given the (poor) quality of thought typical on this list, somebody might respond with "that was an easy book", hoping you'll infer that button-click generation would be "impossible", or at least "difficult", for books which are more complex in nature. i tell you that's bullshit, and if you want me to write the code to prove it, i will. because i already have... in more than one language... *** have a nice weekend... :+) -bowerbird

1 0

these grapes are sweet -- lesson #20
by Bowerbird＠aol.com 30 Sep '11

30 Sep '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** ok, i ran a test on the end-of-line hyphenates... > http://zenmarkuplanguage.com/grapes119.py > http://zenmarkuplanguage.com/grapes119.txt behold, up popped "first-hand" and "pre-eminently"... although my dictionary currently accepts these as fully-joined, non-hyphenated words, _this_ book treated them both as _hyphenates_, even mid-line, so i changed them appropriately in my newest text. that "pre-" made me look at a few other common compounds, like half-whatever and co-whatever... that lead me to make a change to "co-ordination". my dictionary considers it a non-hyphenate, but this book had a case of "co-ordinating" midline, so i reinserted the hyphen, just to be consistent... *** so, our newest version of the text is now here: > http://zenmarkuplanguage.com/grapes007.txt *** plus i did final spellcheck with the book's dictionary... > http://zenmarkuplanguage.com/grapes120.py > http://zenmarkuplanguage.com/grapes120.txt we're good to go... *** now we can get down to an evaluation of our work... given that we are now done with the text-cleaning, i ran a check against my existing text of this book, which has been _thoroughly_ cleaned many times... and, all in all, the results were very good. recall the book is over 200k, and 279 pages. *** 16 lines with o.c.r. errors going uncaught, most of which were involving _punctuation_. 2 were on a name, which we shoulda checked. error> plete expressions, in that concrete right> plete expressions. In that concrete change> =================^=^=============== error> and exhausts while he instructs; the right> and exhausts while he instructs, the change> ===============================^==== error> bought Cary's crib, and took it with right> bought Gary's crib, and took it with change> =======^============================ error> read my Cary's Plato. It so hap- right> read my Gary's Plato. It so hap- change> ========^======================= error> The latest of them, Count Tolstoi's right> The latest of them. Count Tolstoi's change> ==================^================ error> mass the facts about any given period; right> mass the facts about any given period, change> =====================================^ error> The reality of this element of per- right> The reality of this, element of per- change> ===================^---------------- error> all they saw and knew a part of them- right> ail they saw and knew a part of them- change> =^=================================== error> process, and culture and genius stand right> process, and culture and genius stand, change> =====================================^ error> there has always been not only a right> there has always been, not only a change> =====================^----------- error> stories and constantly touched upon right> stones and constantly touched upon change> ===^------------------------------- error> exploration, travel, and discovery; he right> exploration, travel, and discovery; ha change> =====================================^ error> There are, it is true, a few men and right> There are. It is true, a few men and change> =========^=^======================== error> from the days of the earliest Greek right> from the days of the earliest Greek, change> ===================================^ error> sion, thought, impulse, which never right> sion, thought. Impulse, which never change> =============^=^=================== error> grance, and growth which lie enfolded right> grance, and growth which he enfolded change> =========================^^^^^^^^^^^^ *** 4 new paragraphs that were missed, which shows you really can't skip this check like i did... > This faculty of draining all the > This contact with the richest per- > This deepest and most vital of all > The myth-makers endeavoured to *** 4 "errors" that were present in the p-book... > XIV. Racial Experience ......... 165 > vironment, and the history of races > man. This in not true of secondary > selves throug all fields of thought, *** ...plus i found two errors in the "criterion"... ...which just goes to show perfection is hard... *** again, all in all, a great performance for the amount of work we put into cleaning, a sturdy example of the cost-benefit ratio. -bowerbird

1 0

some questions for brewster kahle and archive.org
by Bowerbird＠aol.com 29 Sep '11

29 Sep '11

ok, alex, here come the questions... :+) before i begin, though, i should note outright that i do not talk shit about people behind their backs. if i've got something to say about something you do, i will address you directly... at least, if i am able to... so you should know that i have indeed addressed questions just like these over at archive.org itself. well, at least until i was banned from their listserve... just so you know... *** also, alex, i fully realize these are probably _not_ questions you can answer, not to my satisfaction, and most likely not even to your own satisfaction. so you shouldn't feel i am putting _you_ on the spot. my questions are about stuff above your pay-grade. that's why i gave this post the subject-header i did... heck, i expect that you will agree with me completely. because, honestly, the points i will make are obvious. i am the little kid saying the emperor has no clothes. *** so here goes... *** here's the main question: 1. how can archive.org put out such crappy e-text? i mean, seriously... i've appended a few pages of text from the book that i have been using in my current "digitization lessons". that's what users get in the .mobi. and in the .epub. look at it. it's raw o.c.r. is what it is. it's _bad_. it's _really_ bad. it's _so_ bad that i would be _embarrassed_ to put it out. so embarrassed that i would _refuse_ to put it out at all. yet archive.org puts it out, trumpets it in press-releases, and acts like it's doing us a favor instead of insulting us. look, if brewster can tell the world, as he has done a lot, that digitization (i.e., scanning) costs 10 cents per page, why isn't he smart enough to bump it to 12 cents (or 14) so "digitization" includes the cost of cleaning the o.c.r.? that's the first question i have, alex, my _main_ question. and i've got more, but let's let that big one sink in first... -bowerbird > Chapter I. > Material and Method. > TF the writer who ventures to say something > more about books and their uses is wise, he will not > begin with an apology; for he will know that, despite > all that has been said and written on this engrossing > theme, the interest of books is inexhaustible, and that > there is always a new constituency to read them. So rich > is the vitality of the great books of the world that men are > never done with them; not only does each new generation > read them, but it is compelled to form some judgment of > them. In this way > 7 > > Material and Method. > Homer, Dante, Shakespeare, Goethe, and their fellow-artists, > are always coming into the open court of public opinion, > and the estimate in which they are held is valuable chiefly > as affording material for a judgment of the generation which > forms it. An age which understands and honours creative artists > must have a certain breadth of view and energy of spirit; an age > which fails to recognise their significance fails to recognise the > range and splendour of life, and has, therefore, a certain > inferiority. > We cannot get away from the great books of the world, > because they preserve and interpret the life of the world ; > they are inexhaustible, because, being vitally conceived, > they need the commentary of that wide experience which > we call history > 8 > > Material and Method. > to bring out the full meaning of the text; they are our > perpetual teachers, because they are the most complete > expressions. In that concrete form which we call art, > of the thoughts, acts, dispositions, and passions of humanity. > There is no getting to the bottom of Shakespeare, > for Instance, or to the end of his possibilities of enriching > and Interesting us, because he deals habitually with that > primary substance of human life which remains substantially > unchanged through all the mutations of racial, national, and > personal condition, and which Is always, and for all men, the > object of supreme interest. Time, which is the relentless enemy > of all that Is partial and provisional. Is the friend of Shakespeare, > because It continually brings to the student of his > 9 > > Material and Method. > work illustration and confirmation of its truth. There are > many things in his plays which are more intelligible and significant > to us than they were to the men who heard their musical cadence > on the rude Elizabethan stage, because the ripening of experience > has given the prophetic thought an historical demonstration ; and > there are truths in these plays which will be read with clearer eyes > by the men of the next century than they are now read by us. > It is this prophetic quality in the books of power which silently > moves them forward with the inaudible advance of the successive > files in the ranks of the generations, and which makes them > contemporary with each generation. For while the mediaeval > frame-work upon which Dante constrenerations, and which makes > them contemporary with each generation. For while the mediaeval > frame-work upon which Dante constructed the " Divine > Comedy " be- > lO

2 1

Re: [gutvol-d] some questions
by Bowerbird＠aol.com 29 Sep '11

29 Sep '11

alex said: > I think I was in 10th grade, maybe? Time flies :) you like to make an old man feel ancient, do you? "you kids get off my lawn... and stop having fun..." -bowerbird

3 2

Re: [gutvol-d] some questions
by Bowerbird＠aol.com 29 Sep '11

29 Sep '11

alex said: > Okay, feel free to fire them away at me. oh, sure, you say that _now_... but just wait... ;+) > As far as whose emails you choose to read, > I suppose that's where the warning on > http://lists.pglaf.org/mailman/listinfo/gutvol-d > emanates, hah. :) well, in part, yes, but only in a very small part... the history is that i had a point of disagreement with a position held by many people here "back in the day", as the kids say, regarding x.m.l. versus "light-markup", back when everyone said x.m.l. would rule the world... the pack turned against me, in force, in a mean way, but i refused to back down, and eventually prevailed, to the point that few issue even a whimper any more. but for several _years_, the dialog was quite venomous. there were many days here when literally _dozens_ of acrimonious posts would hit the list. it was unpleasant. there's nothing quite like a mob attacking a dissenter. lee was a part of the pack, but by no means its leader, or even the worst offender. that would be _marcello_, who even mounted a "fan-site" in my "honor", but who has since had to recant entirely and adopt light-markup -- a la restructured-text -- instead of his beloved t.e.i. so that's the history, alex. but in the last 3 years or so -- ever since the pack admitted defeat and grew sullen -- this list has usually been boring rather than contentious. *** alex said: > Mobi *way* predates ePub, so if you will, ePub could > be considered a "more modern" format than Mobi, > not Mobi is a "dumbed down" version of ePub. of course that's true. and lee has been around for long enough that we know that he _knows_ that is true. but he misrepresented that truth in order to make a cheap political jab, which happens to be an excellent illustration of my point about why i'll no longer bother responding to him... he is not interested in an honest discussion... -bowerbird

2 1

Re: [gutvol-d] a new bevy of kindles
by Bowerbird＠aol.com 29 Sep '11

29 Sep '11

jim said: > The "fun" part of all this is that to be effective PG has to > be willing to meet PG users "where they want to read" > which ever-increasingly means the need for useful and > effective implementations of ePub and Mobi books, > an area where even a casual inspection of PG offerings > will display that PG continues to frequently fail. you're right, jim. p.g. does continue to frequently fail there. however, other entities -- most significantly archive.org -- fail in even worse ways, indeed often in _spectacular_ ways. which introduces us to my next post... -bowerbird

1 0

some questions
by Bowerbird＠aol.com 29 Sep '11

29 Sep '11

alex said: > I'm a collections management intern at the Archive. oh boy... do i ever have some questions for you... *** oh yeah, and because i haven't said this recently... i ignore lee passey's posts, because if i read them, i would feel an obligation to do corrective surgery, and i have learned that that is a pointless exercise. be careful about believing his misrepresentations... -bowerbird

2 1

these grapes are sweet -- lesson #19
by Bowerbird＠aol.com 29 Sep '11

29 Sep '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** ok, let's finish up with just a couple more cleaning routines... again with a search for capitalized words in mid-sentence: > http://zenmarkuplanguage.com/grapes117.py > http://zenmarkuplanguage.com/grapes117.txt this list is a little long, but it found a handful of instances my earlier routine had missed, obviously due to some bug. *** i thought i'd also do a routine on mid-line hyphenates: > http://zenmarkuplanguage.com/grapes118.py > http://zenmarkuplanguage.com/grapes118.txt that didn't turn up anything, but i'm now off to do a check on the end-of-line hyphenates, for the sake of completion. -bowerbird

1 0

Re: [gutvol-d] a new bevy of kindles
by Bowerbird＠aol.com 28 Sep '11

28 Sep '11

jim said: > Not sure exactly who is doing what > re Kindle support on archive.org, > but something new is happening there. "something" might be happening, and it might be "something" that "someone" wasn't bothering to do before now, but it most assuredly isn't anything _new_... > I downloaded a "Kindle" book > from archive.org "at random" what book? and what was the exact method by which this "random" selection was made? > and what I find is much more readable > than what I have found there in the past can you be less specific? i mean, seriously... > -- clearly some new OCR technology, archive.org uses abbyy finereader. always has. > readable, but surprisingly the OCR > doesn't bother to "heal" all the linebreaks > found within words. thank you for your "summary", but without any evidence for your points, it doesn't mean much. there are certain things that archive.org and/or amazon.com could be doing to make those books useful and beautiful and powerful and accurate, as i have been documenting in my current series... whether archive.org and/or amazon.com _is_ doing those things, or not, i don't really know -- or care... -bowerbird

1 0

dinosaurs benefit by acting like mammals (but for how long?)
by Bowerbird＠aol.com 27 Sep '11

27 Sep '11

throwing in the towel on the many benefits formerly obtained by "economies of scale", harpercollins embraces print-on-demand... > http://www.harpercollins.com/footer/release.aspx?id=959&b=&year=2011 -bowerbird

1 0