- gutvol-d - lists.pglaf.org

top 10 reasons why this current thread will go nowhere -- #8
by Bowerbird＠aol.com 06 Oct '12

06 Oct '12

#8. if your workflow has no supporting tools, you have no workflow. #9. this list has proven it can ignore a solution right in front of it. #10. that's where these threads always go, and nothing has changed. -bowerbird

1 0

call up your sled-dogs
by Bowerbird＠aol.com 06 Oct '12

06 Oct '12

greg said: > This is, more or less, exactly what I said we needed. > There is no resistance to any of this. then why haven't you put it into place? > I even asked for input on figuring out what the > requirements & enforcement would look like. jim has pointed out examples of problematic .html. many times. so have i. simply remove those negatives. if you would rather have positive examples instead, examine my z.m.l. test-suite, and how it is handled. *** i must add, however, that even though you say that "there is no resistance to any of this," it certainly _seems_ like your whitewashers couldn't care less (as evidenced by that "update" to huck finn), and to describe marcello as anything less than "hostile" would be an injustice to truth, not to mention that the fine people over at d.p. will impasse your ass... so i think that puts you in over your head, greg... and yes, i do remember that you are rather tall. still, over your head. though perhaps you could call up your sled-dogs to pull you out to safety... but how good are they at writing code? and docs? -bowerbird

1 0

top 10 reasons why this current thread will go nowhere -- #9
by Bowerbird＠aol.com 05 Oct '12

05 Oct '12

#9. this list has proven it can ignore a solution put right in front of it. #10. that's where these threads always go, and nothing has changed. -bowerbird

1 0

patches to epubmaker
by Bowerbird＠aol.com 05 Oct '12

05 Oct '12

paul flo williams said: > maybe one of us could > actually submit some patches to epubmaker. i'd love to see that, to see what marcello would do. based on my experience, he'll tell you that it's no good, and you're stupid, and you don't understand the issues, and your interpretation of everything is wrong, and then he'll make the change himself and take the credit for it... but hey, you know, if you're ok with that, then go ahead. -bowerbird p.s. i'm still not reading your posts, marcello, so... you know, don't even bother preparing a "response". or, better yet, attack me with yet another low blow, confident that i won't respond because i won't see it.

1 0

Re: [gutvol-d] blah blah blah blah blah
by don kretz 05 Oct '12

05 Oct '12

So what are you suggesting? I can't imagine people out there saying to themselves"This is simple; I think I'll make it complicated." or"This is complicated. I think I'll oversimplify it." What is a constructive suggestion? Sent from my Phone From: James Adcock Sent: 10/5/2012 9:51 AM To: Project Gutenberg Volunteer Discussion; Tapio Riikonen Subject: Re: [gutvol-d] blah blah blah blah blah >Books are complex beasts, and not always easy to typeset or encode. We have a lot of books which are pretty simple, and could fit into a common methodology which should be able to run successfully "everywhere." We have a few books which are very complicated, and are not going to fit into a common methodology, at least not one which we can teach or motivate the average volunteer to use. There is a strong countercurrent however, who want to take simple things and make them complicated, and well as another thread who want to take complicated things and oversimplify them. _______________________________________________ gutvol-d mailing list gutvol-d(a)lists.pglaf.org http://lists.pglaf.org/mailman/listinfo/gutvol-d

2 2

Dumbed-down outline of PG process
by don kretz 05 Oct '12

05 Oct '12

This is absolutely without regard to method, technoiogy, markup, standards, or responsible parties. Those can come later, but this is to enumerate the steps in the PG workflow (although "flow" is a stretch in some cases) in order to provide scope for any discussion of PG workflow per Greg Newby's post. These have to be accomplished one way or another in order for a book to become an ebook in the PG catalog, and support multiple format output and ongoing refinement and error correction. Some can be simultaneous and there's overlap. Some may be provided by external sources (e.g. images.) Participants include volunteers, PG, and customers. 1. Choose a book to digitize. 2. Collect and record metadata, and confirm Public Domain status. 2. Obtain images of all the pages. 3. Produce text with OCR. 4. Refine the text to match the OCR. 5. Locate and mark significant artifacts in the text. 6. Submit package of text(s) and images to PG. 7. PG review and post to catalog. 9. Select layout and format for desired device or application. 10. Select or create layout and format for text and artifacts. 11. Issue ebook per spec (possibly including production.) 12. Identify errors or other revisions to text. 13. Locate and describe revisions. 14. Submit revision to PG. 15. Back to step 7 to validate and apply revision. There are obvious definitions to be provided, including metadata, refine, location and mark, significant artifacts, package, etc. etc. Don

1 0

top 10 reasons why this current thread will go nowhere -- #10
by Bowerbird＠aol.com 05 Oct '12

05 Oct '12

#10. that's where these threads always go, and nothing has changed. -bowerbird

1 0

blah blah blah terminology blah blah
by Bowerbird＠aol.com 05 Oct '12

05 Oct '12

don said: > I've never seen the term "vanity html" before. i've used the term "snowflake" _many_ times before. i'm quite sure jim is using "vanity" in a similar way... > The only effect I see for using it is > to add to the hostility level and > decrease the likelihood you will get cooperation. i am the one who uses tactical means to increase the odds that y'all won't act on my advice, not jim. i do that because i am _amused_ by your resistance to the message due to your dislike of the messenger. i find it quit entertaining that you guys would rather thrash around, back and forth, for _years_ (literally!), instead of adopting a solution i have proven to work. in other words, i don't need any of your "cooperation". i'm doing just fine, _laughing_ here on the sidelines... as for jim, i think the term "vanity" probably represents the fact that he sees the "snowflake" .html as selfish... and i think he's probably right about that. even though i use a different term (one that might be less evaluative), i say jim's entitled to use "vanity" to express his feelings. that doesn't mean he's using it to decrease cooperation. > Same for "dumb-downed" but not as aggressively. i disagree that this term is being used for "spin" either. i think it accurately depicts a viewpoint which is common. i also believe, and feel i have proven, that that viewpoint is not based in fact. you definitely have to change stuff, to make it work on today's somewhat primitive e-readers, but the resultant product still works sufficiently properly to avoid the label of "dumbed-down". indeed, at times, the "fancy" .html used for desktops can get "too cute" and trip over itself, and ends up giving a degraded experience. so i strongly _disagree_ with jeroen, and the other people who use this term, because i believe it reflects a mindset that doesn't have to be held to describe reality accurately. (it's certainly true that one _can_ "dumb-down" the .html. but i say that it's not _necessary_ to do so to make it work. i've been able to make all common book-elements work.) but even though i disagree with the term "dumbed-down", i understand that it reflects the mentality of some people, a mentality which is not accurate, but still one they hold... and thus, i certainly don't think it's being used for "spin", or to increase the hostility, and most certainly not for the purpose of sabotaging any possibilities for "cooperation". indeed, "dumbed-down" doesn't have any ad hominem in it. (and even "vanity" is only ad hominem if you interpret it so.) > I think better terms might be "complex" vs. "simple" > and "portable" vs. "non-portable." those poles are no better. in some ways, they are worse. "conversion-capable" versus "snowflake" is the essence. this stresses that _conversion_ is the main objective... "snowflake" is the obvious word to use for the one pole, because everyone knows that every snowflake is _unique_. there are lots of ways to accomplish something in .html. if you multiply all the possibilities, they become infinite. you need to have common coding conventions in order to chop down that infinity into a number that can be handled. to sum up, it must be _standardized_to_a_limited_palette_. (where the word "limited" pertains to numbers, not quality.) this means that if there are 89 ways to do something, you might have to chop that down to a mere 23. or 7. or, if you can formulate code to handle all 89, then there's no reason from that standpoint to set a limit, although you might impose a limit anyway, so as to make it easy on the humans maintaining the library. for the purpose of computer conversion, then, you'll need to actually _write_the_code_ so you know _how_many_ and _which_ of those 89 different ways your code can handle... avoiding snowflakes is a _necessary_ condition for conversion. but it's not enough, in and of itself. in order to convert properly, you need to understand the structural nature of every single element within the book. now, it's possible to have a complex scheme which is still _incapable_ of conversion because it doesn't communicate the necessary information about the structures in the book. on the other hand, it's possible to have a scheme which is _simple_, yet still manages to communicate the information. sure, there's a possibility for a complex scheme that works. (although then there arises another question about whether that would be a good solution for a volunteer organization. but for this reply, we're concentrating on terminology use.) and there is, of course, the simple scheme that doesn't work. but it's just wrong to think that any simple scheme will fail, and any complex scheme will succeed. so that label is bogus. as for portable/non-portable, i'm not sure what you mean. i suppose you could say you meant "convertable/snowflake", but if that's what you really meant, then don't be ambiguous, say it in a way that doesn't require the internal interpretation. -bowerbird

1 0

Blah**5 - Virtualising layout
by Robert Gibbins 04 Oct '12

04 Oct '12

There is an interesting question not quite getting discussed in this debate about what exactly IS epub-friendly HTML. Greg's last append, and this isn't the first one, seems to imply that if only contributors would use only structural HTML, then all would be well. I spent much of the 1980's writing my technical reports in GML (a predecessor of HTML (it's where the <p>, <ol>, <li>, <h1>, <h2> etc. tags in HTML came from)) in which you could specify little MORE than document structure. For technical reports conforming to a well-understood corporate layout policy, this worked pretty well. When I started looking at how to improve PG epubs a year or two ago, I assumed initially that declaring document structure in this way was indeed the main objective. But it now seems to me that just declaring document structure in HTML is inadequate, because there is meaning in the layout of books, which isn't easily or obviously coded in simple HTML structural tags. One might object that HTML extended with CSS styles is much more flexible and customisable than plain old HTML - you can use the facilities of CSS to effectively create your own tags (e.g. lots of <div>'s and <span>'s of specifically declared classes). This is perfectly true, but the danger if you do this is that by doing so you have superimposed your own conceptual model of the structure of the book contents onto the author's original work. And what you may well actually be doing is recording your misunderstanding. To give an example, if you look at many 19th century novels there are quite often many letters given in the text (Austen, Brontes, Trollope etc). You could come up with a standard set of styles for letter headings, openings and closings which would then standardise how to format them. Initially I thought this was what I should do (and from a brief look at TEI that seems to be what it does), but looking a little closer, the particular style and layout of the headings, etc. often conveys something about the formality or intimacy of the relationship between writer and correspondent. To satisfactorily encode this into a discrete set of CSS styles one would first have to have a profound understanding of letter writing conventions when the book was written, and how they changed over time, and secondly a way to encode those conventions into HTML/CSS in a sufficiently expressive way. My current thinking is that although much of the coarse-grained structure of a literary work (the division into front matter, main body and back matter, the division into volumes, chapters and sections, the division of much of the text into paragraphs) CAN be satisfactorily encoded using standard HTML structural tags, for the finer-grained structure it is more realistic, and probably more likely to convey the author's original meaning, to try instead to simply 'virtualise' the original layout (using CSS styles to document the virtual layout). What I mean by virtualise, is to do the markup to preserve layout in as scalable, reflowable and device-independent a way as HTML allows. For example: - not quoting fixed font sizes - specifying margins in scalable units like ems - using the hanging-indent technique quoted by Jim Adcock a few months ago where appropriate I'm sure other contributors to this forum could produce many more such techniques. Whenever in the past I have suggested that a little standardisation of HTML might be a good idea this has been rejected by Greg, more or less out of hand on grounds like: - PG is not interested in individuals who want us to admire their pretty layouts - PG doesn't want to place artificial restrictions on submitter's HTML Actually I agree with both of these points, but my belief is that if PG wants to produce readable epubs (something which many people agree it isn't currently doing), there isn't a simple one-line answer that achieves this. PG is going to have to step up to the plate and think about standardising its HTML contributions. And (ahem) from what I can see of their contributions, it's not totally clear to me that the current PG inner circle necessarily have all the experience and skills to do this on their own. Bob Gibbins

3 2

let's step through this process.011
by Bowerbird＠aol.com 04 Oct '12

04 Oct '12

now for the debriefing on our update walk-through... *** we re-digitized "pride and prejudice", using this source: > http://archive.org/details/harvardclassicss03elio all of our work-files are here: > http://zenmagiclove.com/prhpr the final "updated" master z.m.l. file is here: > http://zenmagiclove.com/prhpr/prhpr.zml the derivative versions are found here: > http://zenmagiclove.com/prhpr/prhpr.html > http://zenmagiclove.com/prhpr/prhprp222.html *** ok, let's start this debriefing with the main headline: digitization isn't as hard as some of you seem to think. you might remember that i did this project because jon hurst wanted to re-do the top-10 p.g. downloads. he offered up some grand plan that would've involved d.p. in some complicated workflow, and it was obvious to me that he was overestimating the work required to re-do one of these classics... thanks to plentiful scans and a variety of digitizations out in the wild, it can be relatively quick and easy to do books from start to end. so i decided to prove it with a bit of my own pudding. now, one week later, we have a newly-done "update", including an extensive documentation of the process, from little old me working a small number of hours on a handful of days. (it took more time to write it all up.) by doing it myself, i have shown that you can do it too. but if you _do_ choose to take action, take my advice: it is _not_ the smartest way to just jump in like i did, and start it up, not if you don't have the _experience_ that i have, working with those archive.org materials... so here's some advice on choosing your material... this is about archive.org, since i'm stuck with them, being that i gave up on google a while back because google seemed to be making it harder and harder to scrape their material, whereas archive.org is _open_. but archive.org stuff can suck, so you must take care. particularly avoid the stuff that archive.org scraped from google, because then you get the worst quality of both institutions, and that can be a lethal combo. (it might also be the case that now that google has brought out their commercial arm, the quality of the work they offer up to the public has improved. i haven't checked, to know, but it _is_ a possibility.) but back to internet archive... my first piece of advice about archive.org material is to choose it very carefully. some of their scans are not-good. if a book has bad scans, _skip_the_book_. life is too short to spend time staring at lousy scans. if you can't find good scans, move on to another title. you should particularly avoid scan-sets archive.org has "liberated" from google, because they are awful. sometimes the _scans_ can look good enough to use, but the _o.c.r._ turned out lousy. skip those as well... the o.c.r. which i used in this project was _terrible._ the letter "e" was frequently misrecognized as "c", and as you might know, the letter "e" is _common._ and there were lots of other misrecognitions too... i was able to deal with it because i know the ropes, but until you clean a dozen archive.org o.c.r. files, you won't, so you should avoid the hassle entirely... (and once you _have_ cleaned a dozen such messes, you will know quite well you don't want to do more.) you should especially watch out for some o.c.r. files from archive.org which have _lost_ their em-dashes! restoring those em-dashes is a huge pain in the ass. speaking of the o.c.r., the easy thing to do is to grab the djvu.txt file that's listed right on the "files" page. that's what i did, but it is _not_ the right thing to do, not if you want to minimize the time it will take you. it's much better to pull the text out of the x.m.l. file that's saved by abbyy finereader in the o.c.r. process. that's because _styling_ information gets saved there, and in many cases, it might well take a ton of time to reconstitute it. so i recommend you extract it initially. the problem with my recommendation is that you will have to write yourself an app to grab that information. (yes, i wrote such an app, but you may not use mine.) if you can't write an app, or use somebody else's, then i guess you'll just have to use the djvu.txt file instead. it won't be the end of the world. heck, i did it myself, for this demo project, because i knew i already _have_ all of the styling information for "pride and prejudice". but if you can write an app, then go ahead and do that. another benefit of using the abbyy.xml file is that you will see that it contains those _em-dashes_ which the incompetent technocrats over there lost in the deja-vu. also, if you will be working in a web-based application, you can squirt the scans from archive.org to your site, without doing all the dirty-work of downloading them to your machine and then uploading them to your site. those are the big points. if anyone has any questions about smaller-grained issues, i'll be happy to answer, to the best of my ability. most glitches have bitten me. as far as the step-by-step workflow of doing the edits, i have belabored that many, many times in the past, so your best bet in that regard will be to hit the archives. *** just a couple more notes. i programmed all my tools for this "pride and prejudice" project _from_scratch_... so even if you have to start from scratch, it ain't hard. i don't think i hit hard enough that this last round of page-by-page .html files have an error-report form, right on each page, so it makes it _very_convenient_ for someone to report an error, unlike the p.g. way... and yes, the text i created does needs smoothreading. i'm not vouching for its full accuracy, by _any_ means. i miss the good old days, when jose menendez would appear every time i posted something, and do a reveal of the 197 errors he found in it. that kept me humble. so go ahead and find the errors that i could not find! it'll make me happy to have my human-ness exposed. because, and wraps up my summary, i think that i did a pretty good job. i think this text is amazingly clean. if there is a cleaner "pride and prejudice" _anywhere_, i would very much like to see it, so do please tell me... and not only that, but i did the entire job in one week. *** so, once again, the executive summary: digitization isn't as hard as some of you seem to think. -bowerbird

1 0