- gutvol-d - lists.pglaf.org

stop changing the message-headers
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

jon said: > I can only interpret your vociferous replies > to mean that you consider permanently dumping accented characters "permanently"? as i said, as soon as unicode works everywhere, i will embrace it. those of us out here in the real world know that time is not yet here. in the meantime, my viewer-tool will actually _display_ all those accented characters, even when they are not present in the e-text, if the user chooses that option. (it's all about user-choice for me.) if you want to help with that, create a list of such accented words. > to be an *important* requirement to implement your system. "my" system? michael's philosophy of having the e-texts work on all machines, specifically including trailing-edge machinery, is _the_ factor that has made his e-library the premier one in all of cyberspace, thank you very much. you got a high-tech solution? fine, use it. and watch it wither, just like every other one before it has... as to my tools in particular, they will support unicode fully, long before unicode works with all the other tools out there, so you're barking up the wrong tree, buster. i'm done with this stupid thread! done, done, done! aarrgghh! :+) -bowerbird

4 3

Re: [gutvol-d] stop changing the message-headers
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

jon said: > cañon i came across one project gutenberg e-text that used "cañon" throughout, including a word-capped reference to "the grand cañon" -- you know, the one in arizona with all the pretty colors. anyway, guys, call me when unicode works on all apps on all machines. until then, i have put this issue to bed. by the way, it is _still_ so easy to digitize the average book that, once you have the scans, an average person can do it in one evening. -bowerbird

1 0

re: [gutvol-d] lest the message be missed
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

jon said: > But I believe it is also essential to preserve > all accented Latin and non-accented characters > found in *all* books. once again, the minutiae is being brought to the surface. why doesn't anyone here respond to the main message? because you have no response, that's why. the main point doesn't correspond to your petty-politics of throwing mud at michael, so y'all continue to try to shift the emphasis. > But I believe it is also essential to preserve > all accented Latin and non-accented characters > found in *all* books. we know that's what you believe, jon. you've said it over and over and over. and i have said, over and over and over, that _i_ believe it is _not_ essential, not in *all* books. so there we have it. i will do things my way, and i expect that you will do things your way. fine! let's leave the other people here alone! as usual, you look only at the _benefits_, without factoring _costs_ into the equation. the _cost_ of including high-bit characters is the e-text then _breaks_ for some users, ones who are using viewer-programs that are not encoding-savvy, or who don't have all of the correct fonts on their computer. or other reasons i haven't come across yet. if the unicode people had done their job right, and made unicode follow the mac philosophy -- "it just works" -- i would be up there on the unicode bandwagon with you and your friends. but it doesn't "just work", not for everyone -- not yet -- and until it does, i don't want to talk about it. and _after_ it does, i don't want to talk about it _then_, either, i just wanna use it and have it work. for everyone. wanna do something useful? _make_it_work_! not just on the new machines, with certain browsers and not any other viewer-programs -- on _every_ machine, with _every_ program. but until then, just stop bugging all of us about it. we've heard it, too often, and we are unconvinced. and buddy, you are _not_ going to convince us by repeating the same old argument _again_, or by asserting your beliefs again and again... with all the time i've wasted discussing this stupid topic for the 829th time, i could have cleaned up the rest of that "my antonia" text. go away. oh never mind, i will... -bowerbird

4 3

re: [gutvol-d] stop changing the message-headers
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

carlo said: > Would you replace a four-voices canon > with a four-voices caÃ±on ? with a _what_? i see a capital "a" with a curvy squiggle above it, followed by a plus-or-minus sign. what you sent me back is _not_ what i sent you; it's been changed; just like that game of "telephone" that brad was telling us about. so it looks like our software here isn't handling the encoding correctly. which is precisely my point. and in the cases like this, it's great to give the user the option to go back to the 7-bit letters, so there is a semblance of normality. because we _know_ that "canon" is always going to be "canon". -bowerbird

3 2

re: [gutvol-d] stop changing the message-headers
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

carlo said: > Would you replace a four-voices canon > with a four-voices caÃ±on ? but to answer your question, there is only one "canon" in "my antonia". and yes, i will do the conversion on a book-by-book basis in cases where that is necessary. (how many of these terms do you think you can find -- with a non-accented and an accented version -- where _both_ are listed in an english dictionary? look away, my friends, because every one you find is one that makes my lookup-table more extensive.) and even then, if a change is not completely reversible, you'll need to give me the entire sentence in each case where the change is to be made. (or, if it's easier to do it the other way -- each sentence where the change is _not_ to be made -- you can do it that way instead. the only requirement is an absolute non-ambiguity.) you can _count_ on the fact that i have thought things through _well_past_ the first exception to everything. you will have to burrow down to a _much_ deeper layer if you really want to trip me up. and i dare you to try... -bowerbird

1 0

re: [gutvol-d] stop changing the message-headers
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

brad said: > How many users have told you that they want to toggle accents? toggle them on? or toggle them off? :+) according to some people here, there is a great desire out there to see the accents. so i'll try to reintroduce them when possible so far i have these words in my lookup-table: > 142 é Bénédictine -- Benedictine > 142 é naïveté -- naivete > 144 ê crêche -- creche (my spellchecker gives another accent?) > 149 ï naïve -- naive > 149 ï naïveté -- naivete > 150 ñ cañon -- canon > 170 ™ ™ -- (TM) > 174 Æ Æneid -- Aeneid > 190 æ antennæ -- antenna > 231 Á Ántonia -- Antonia please do feel free to send me more... :+) > If you knowlingly strip out accents you give up any claim to > have created a faithful and accurate edition of a text. contrary to what some people would like for you to believe, that doesn't have to be the only objective, or even the main one. for me, _mass_usability_ reigns supreme. call me a heathen... > and drops your text down to the level of > a bootleg Harry Potter translation[1]. my understanding is that some harry potter pirate digitizations have attained an extremely high level of fidelity to the source... i'd be leery of the crummy ones. as would most people. which is probably why one writer organizations has been advised to put out crappy "pirate" digitizations, so as to sour people on underground editions. so perhaps the bad version your wife got was planted by the publisher? if so, it seems to have had exactly the desired effect, eh? (although it sounds like it was a paper-book as well?) i heard one "potter" e-book was finished _within_24_hours_ of the release of the paper-book. and when my tool is released, i expect that the pirates will become even _more_ efficient! > Why is this so important? i don't know. but to hear some people talk, you'd think it's a matter of life-and-death! > It's the old game of whispering a sentence into someone's ear > and then they repeat it to someone else etc. > After passing through a few people the sentence get's mangled. hence the importance of complete reversibility, already mentioned. > After passing through a few people the sentence get's mangled. like the way the word "gets" got mangled in your sentence? :+) or the way "knowingly" turned into "knowlingly" up above? ;+) -bowerbird

2 1

re: [gutvol-d] lest the message be missed
by Bowerbird＠aol.com 09 Mar '05

09 Mar '05

carlo said: > How do you manage words that are written in the same way, > except the accent? This is quite common in french and italian. i wouldn't strip away high-bit characters on a french or ltalian book; they are an essential part of those languages. i've said that repeatedly. jon noring seems to turn every listserve into an endless merry-go-round. *** robert said: > With some of the OCR I've seen latesly, > this is probably about 90% right, for 90% of books if you want me to estimate some hard numbers, i'd say 75% of the e-texts in the library now could be done to our standard in one evening. the "standard" we've been talking about thus far is 1 error every 10 pages. if an individual can take an e-text to that level of accuracy, then it can be turned over to a quasi-public process i call "continuous proofreading", discussed below. another 20% of the e-texts in the library now would take two or three evenings, and might require the help of a "specialist" of some kind, someone knowledgeable about a certain arena, like greek, or tables, or indexes, or graphics, etc. here's where the value of distributed proofreaders will most come into play in the future, in my opinion, being able to "fix up" the work of independent proofers. the remaining 5% of the e-texts in the library now (remember, i'm just guessing at the numbers here) might be too difficult for any individual to take on, for reasons of size, complexity, or what have you. again, distributed proofreaders will shine here... > provided you can get good scans, jon noring promises us good scans, at a high resolution. i guess he's got a lotta money, to be able to afford the maintenance costs of storage and bandwidth for them. if he delivers on that promise, we don't need to worry... if he can't -- and he has no track-record in this arena, so maybe we shouldn't count on him -- then we have the scans that internet archive is making for toronto. they've promised us "thousands" of scanned books by this spring, but the last thing i heard was that they were "pausing to do an evaluation of their quality", which probably means they've come to realize that they must do a much better job than they have been. i've heard that some books are done very well and that others leave a lot to be desired. consistency in this regard is often a difficult goal to achieve. but i believe brewster will eventually get it right. and of course, there's always google. we still don't know what kind of job they'll do, or if we can use their scans. but if they do a good job, and release their scans freely, that will provide us with a ton of scanned-image books. some people reading this undoubtedly work in an office that has one of those "multi-functionality" machines. some of those babies can scan over 60 pages a minute, straighten the scans, and upload them to the internet, all while you sit there and whistle and pick your nose. at that rate, the 450 pages of "my antonia" would take 7.5 minutes. i don't know how crafty _you_ might be, but my fellow poets are _highly_ skilled at hijacking office machinery for our own nefarious purposes... ;+) (heck, that's the only reason some of us get a job at all.) finally, there are millions of home computers out there that were sold with an all-in-one printer/scanner/fax. and, as before, the quickest and easiest way to get to a million scan-books is for a million people to scan _one_. so, it's fairly easy to predict that, from one or more of the above factors, there will soon be an _avalanche_ of books that have been scanned and need to be converted into text. and every scanned-book will have at least _one_ person who will want its text badly enough to do a little work. what we need to do is _give_that_person_a_good_tool_ that enables their little bit of work to get good results. that's what i intend to give them. all i ask of you is that you stop telling people that this job is difficult. it's not. > and provided you are willing to let a few > hard-to-detect classes of error you'll have to explain what you mean by "hard-to-detect". in my experience, if an error is serious (in any meaningful way), then it'll be detected by a person who's actually reading the book. some errors are unforgivable, such as an incorrect word that won't even pass spellcheck. those should _always_ be caught. trivial punctuation errors, like a missing comma, are... well, trivial. (although i haven't mentioned anything about it until now, a great way to catch some errors is to have the computer speak the text aloud to you as you follow along reading it; stealth scannos, for instance, are handily exposed by this.) > and provided you are willing to let a few > hard-to-detect classes of error > go until post-production. "post-production" has no meaning in my scenario. i repeat myself, again and again, by saying that once a person gets the error-level on an e-text down to 1 error in 10 pages, we can make it available via "continuous proofreading" and let readers-from-the-general-public zoom it to perfection. 1. scan. 2. "fix" the scans. 3. do the o.c.r. on them. 4. run the post-o.c.r. tool. 5. do quasi-public "continuous proofreading". 6. consolidate the corrections into a public release. 7. release the e-text out to the public as a single file. 8. continue doing a full-public "continuous proofreading". 9. take error-reports from the people reading the file offline. i will also state, for the record, that i think step #4 can do _far_ better than 1 error in 10 pages if we sharpen our tools. look at the "my antonia" example. jon had a team of _seven_ proofreading it. i don't know how many looked at each page, but, according to my analysis, they took the error-rate down to about 1 every 70 pages. when i used my bag of tricks on it, i removed 3 errors from the 210 pages i subjected to scrutiny. (which leads us to predict 3 more errors in the second half.) to the best of my knowledge, there are no errors in my file, i.e., the first half of the book. (full book by friday, hopefully.) i'm not saying it _is_ free of errors even now, since someone with a different set of tricks in _their_ bag might be able to locate 2 more remaining errors i couldn't find, but i will say that it is more than clean enough to turn loose on the public... (that is, i think we could skip steps #5 and #6 on this e-text.) -bowerbird

3 2

re: [gutvol-d] lest the message be missed
by Bowerbird＠aol.com 08 Mar '05

08 Mar '05

frank said: > Maybe because it got lost between all the other stuff you wrote? no, i think it "got lost" because it was buried under minutiae. and it is _still_ being subjected to attempts to bury it... > Ah, I see you mean: > > you can take an average p-book from scans to e-book in one evening. well, a better phrasing of that main point might be that "_any_average_person_ can do an average book in an evening..." and the follow-on point would be this: "...so let's start informing the people who might wanna do a book, once the avalanche of scanned-books arrives, so they'll realize it is within the realm of possibility; let's stop spreading the false meme that it's difficult." > Well that's great, so start going:) it's wiser for me to build the tool that enables people to do a book in an evening, rather than spend my time doing books... but yes, i will "start going" on that, right away! just as soon as i help jon find the rest of his errors. (as i'm sure you realize, those two go hand-in-hand.) but really, there won't be a need for that tool _until_ after the avalanche of scanned-books becomes available. most people just won't be motivated enough to do a book until there is a scanned-book they want to have as text. until then, i'm in no hurry. been working on this tool for well over a year now. no reason to rush things. just waiting for jon to get some more books scanned... ;+) -bowerbird

1 0

lest the message be missed
by Bowerbird＠aol.com 08 Mar '05

08 Mar '05

lest the main message be missed in all the minutiae... you can take an average p-book from scans to e-book in one evening. one evening. the people who want to convince you that it's difficult are _wrong_. the fastest and _easiest_ way to get a million p-books digitized is for one million people to convert one book in the next month or two. *** also, for the record, all of the global changes i made to "my antonia" are completely reversible, if you're smart enough to know what you're doing. -bowerbird

3 2

re: [gutvol-d] lest the message be missed
by Bowerbird＠aol.com 08 Mar '05

08 Mar '05

codepoints for macroman encoding: > 142 é Bénédictine > 142 é naïveté > 144 ê crêche > 149 ï naïve > 149 ï naïveté. > 150 ñ cañon > 170 ™ Edition™ > 174 Æ Æneid > 190 æ antennæ > 231 Á Ántonia -bowerbird

2 1