- gutvol-d - lists.pglaf.org

re: [gutvol-d] Policing the list
by Bowerbird＠aol.com 19 May '05

19 May '05

branko said: > I started filtering out the flunkeys, and currently > have the following addresses in my blocklist: > Shimmin, Jon Noring, JWaddell, Gutenberg9443, > Joshua, CannonA, Brandon, and Doug Schroeder. you forgot marcello and david starner, two of my favorites. -bowerbird

1 0

Sayonara
by N Wolcott 19 May '05

19 May '05

At least it will be as soon as I find out how to. N Wolcott nwolcott2(a)post.harvard.edu

2 1

bowerbird addresses jon noring's "challenges"
by Bowerbird＠aol.com 18 May '05

18 May '05

ok, let me get to jon's "challenges" before i'm banned here... :+) yes, it's long. as the saying goes, "the difficult we do immediately; the impossible takes a little longer." jon says that what i am doing is "impossible". as you'll see, it's just a determined application of some good old common sense... but explaining it all made this long. sorry. if you don't feel like reading it, just go on to the next post. but then don't come around crying later, saying i didn't explain it... *** jon said: > For an example of this "multiple uses of the same typography", > the venerable "italics" is a good one. Italics are used for: > linguistic emphasis > literal emphasis (not the same as linguistic emphasis!) > names of ships > titles of certain types of books and documents > word used as a word > foreign phrases > sometimes used for headers > etc., etc., etc., etc., etc. ,etc., etc., etc. well, my first reaction is that you've got a pretty good list going there, jon, i'm impressed. italics certainly seem to be a workhorse, don't they? my second reaction is, wow, this list must just be the tip of the iceberg! because _look_ at all those etc.s at the end of the list! even one etc. means that "there are more items in this list", so _8_ (count 'em, 8!) must mean that there are _lots_and_lots_ of more items in this list! so now you've got my curiosity inflamed, jon! please, could you give us the _complete_ list, the one that has all of those etc.s fully delineated? considering how overloaded italics are, how do humans figure them out? but you know what? we do. personally, i can't remember a single time -- not a single one -- where i was ever _confused_ about why something was italicized. so of course, my response is to "just italicize them as the p-book did, and let the reader sort 'em out, just like the readers of the p-book did". so the full list isn't necessary, jon, except you've really got me curious! i can't even _remember_ the last time i say 8 etc.s in a row like that! as for "accessibility" concerns, i think those readers are just as capable as sighted readers at filling in the semantic inferences of the story, and i will not give them less credit than they deserve. it would be insulting. if sight-impaired readers tell me they want elucidation of these things, i'll be responsive. but in the absence of that, i will assume capability... for the intellectually curious, though, yeah, it would certainly be possible to write routines that could make a good prediction about which one of the possible meanings were the one that was responsible for the italics. and with the right set of look-up tables, the predictions might be sharp. in that regard, aside from "signs" in the current case, a block element would include: notes, invitations, death-threats, placards, notices, flyers, correspondence, diary-entries, suicide-notes, love-letters, cease-and-desist notices, warnings, equations, eviction notices, lyrics, poems, newspaper-clippings, quotations, out-of-order signs, magazine/journal articles, pull-quotes, ledgers, tables, and what else? *** so this italics example is like other variables that _freak_jon_out_, and which would be terribly expensive and time-consuming to code, but for which there is _simply_no_demand_from_the_users_ to have, so it makes no sense for us to worry about doing the coding of them. for the few people -- academics or dilettantes or pedants or whatever -- who are concerned with this specificity of detail, let _them_ do the work. *** in spite of lots more verbiage about how the whole idea is impossible, jon's only other real "challenge" was involving the detection of headers. first of all, let me say that people are very good at recognizing headers, just as they are very good at figuring out the specific reason for italics within the context of a story. so one form of my reaction could be to say, "don't worry about it, because the readers be able to figure it out fine." however, the headers in a book _do_ need to be detectable by a viewer-app, to make them big and bold, create a hotlinked table of contents, and so on. so, for headers, we _do_ need to think about how they can be signified. fortunately, this is very easy to do. zen markup language call for headers to be indicated unequivocally by a set of 4 or more empty lines preceding them, so detection is simple. use more and fewer blank lines to indicate headers of different levels. problem solved. a hotlinked table of contents is created automatically, and placed in a "contents" menu on the menubar for further convenience; headers are displayed in big and bold text, just like readers expect, and sections begin on fresh screens in the best practice of typography, etc. of course, this requires that the person who creates the digitized text takes a concrete action on each header -- i.e., puts in those blank lines. since that's such a dirt-simple task, i don't consider it to be a big deal. it shouldn't take but just a few minutes, even in a rather-complex book; time well spent in the _4_hours_ i allotted to digitize the average book. besides, in the grand tradition of "zen markup language" philosophy, these blank lines are _something_that_people_already_do_anyway_, for the most part. it's only natural to give a header "breathing room". it feels like the right thing to do. indeed, one of the formatting rules for project gutenberg e-texts is to precede a header with blank lines. although i don't recall what the guidelines specify about the _number_ of blank lines to use -- especially with headers of different levels -- most e-texts end up being able to be evaluated by using this variable... *** jon _might_ have been referring to auto-recognition in o.c.r. output. no problem. because headers _are_ so important, i spent some time on recognizing them, and it ends up that i can now handle that very well too. if you've scanned the book with the right settings, the information about the "space above" a specific line -- in terms of number of blank lines -- might well be available to your routines, so they can judge on that basis. but even if that information is missing -- as, for instance, it is when you copy text out of a .pdf, which often loses all info about blank lines -- it's still possible to evaluate a line to determine whether it's a header. my program has a 30-item checklist to determine if a line is a header, and -- even though i haven't optimized it yet -- it works like a charm... i'll eventually run it against the p.g. library, and check its performance. maybe i'll find i don't even need to optimize it. but if i do, then i will. (the optimization would involve a multiple-regression on the 30 items; in the non-optimized checklist, most items are weighted fairly equally.) in regard to jon's made-up example, i don't usually bother to deal with fabricated tests, because they are easily manipulated, and my concern is how well my routines work on actual e-texts, not on fake examples. nonetheless, jon's examples look to be representative of real-life text, so i will answer them. i won't give full details on the whole checklist, because i'm still debating whether or not to keep it as "a trade secret". but i can reveal enough of it to deal with the two examples jon gives. so let's look at jon's first example: > Here's one very small example to illustrate what I'm saying: > > (exhibit 1 -- I made this up, inspired by Sherlock Holmes) > > ****************************************************** > > ... she walked up to the door, and on the door was a small sign with > a message in stark, bold black letters which read: > > NO SOLICITORS OR SALES PEOPLE > > Ignoring the sign as if it wasn't there, she knocked on the door, > intending to make the sale... > > ********************************************************* this example -- which is not a header, but a sign -- is recognizable to my checklist as not-a-header, because the paragraph that precedes it ends in a colon. if a paragraph ends with a colon, that indicates that some kind of block-text follows, so we can be assured that that section has not been terminated. real paragraphs simply do not end with a colon. i made this point over on one of the forums at distributed proofreaders, when people were wondering what to do in the absence of typographical indication of a block-quote. although the item -- a diary entry -- was clearly understood by a reader to be distinct from the regular body-text, it was set in the same type, so the question was whether it should be marked as a block-quote. it ends up the typographical indication _was_ indeed present -- and big as day in that colon -- they just didn't see it, because they weren't knowledgeable about this typographical nicety... *** > (exhibit 2; adapted from an Encylopaedia Britannica article) > > ********************************************************* > > ...Government weakness allowed the mutiny to spread; and although > order was eventually restored in Istanbul and more quickly elsewhere, > a force from Macedonia (the Action Army) led by Mahmud Sevket Pasa > marched on Istanbul and occupied the city (April 24). > > DISSOLUTION OF THE EMPIRE > > Abdulhamid was deposed and replaced by Sultan Mehmed V (ruled > 1909-18), son of Abdulmecid. The constitution was amended to transfer > real power to the Parliament... > > ********************************************************* jon reports this is a header, and it would almost certainly be recognized as such by my 30-item checklist, since it fits very many of the criteria. there does remain a possibility that it would be judged not-a-header, depending on if the other lines in the file that are judged to be headers exhibited similarities to this one. my thrust is to look not just at a line in isolation, but within the context of all lines (and headers) in the file, since headers almost always bear some kind of similarity to each other, particularly headers at the same level. however, given the information that i have, for this line in isolation, yes, i would likely call it a header. there is another consideration here as well that might have relevance. given an outline-structure that includes several _levels_ of headers, some minor-level headings might not get full "header" treatment, e.g., they might not start on a new page, or be listed in the table of contents. (a table of contents that stretches over more than 4 pages starts to lose its value as a tool that gives the reader an overview of the book. in that case, the main table of contents would include major sections, and then each major section would start with its sub-contents page. sorry to get so picayune here, but remember that i was "challenged".) since jon "adapted" this from an encyclopedia article, that is likely to be the case here. this line certainly isn't a major-level heading, but it's impossible to know where it fits in the hierarchy of levels, not without solid reference to a fuller analysis of the entire file. whether or not this line might be one of those minor-level headings is something i cannot say without knowing the outline of the whole book. but i include it here to give you an idea of the depth of my analyses... back to the line, though, and the things that help identify it as a header. some obvious indicators from the checklist that i can mention here are the all-caps nature of the line, the absence of trailing punctuation, and -- as you should've gleaned -- a properly-closed paragraph preceding it. *** i'll end with an observation. when i set out to create my header checklist, i expected there might be a half-dozen indicators. i found two-and-a-half. thus i found the task is surprisingly simple. in retrospect, i realized why. headers are -- by their very essence -- _intended_to_be_conspicuous_. if a header doesn't "stick out", it is -- by definition -- not doing its job. because they are _designed_ to be obvious, they become very easy to find. indeed, this allowed me to develop a brain-dead way to find the headers -- and in the process of doing so, create an outline of the publication -- one that works with surprising efficiency with all kinds of documents: delete all the lines from the book with any non-bold characters, and the lines that remain will likely constitute the book's outline. try it some time, with any kind of document -- paper or electronic; for instance, although i couldn't get to the site to confirm it, in the past i have found that this approach works perfectly on noring's own website: > http://www.xml-ebook.org/group/xml-ebook-guidelines.html pulling out the oversized text is another good way to go about the task. of course, once you've thought about it, it's rather obvious -- is it not? -- that headers tend to be big and/or bold. in retrospect, it's all too simple. that observation made me think that structures which were trying to be _inconspicuous_ might be the ones that were really the hard ones to find. but i was wrong about this. it ends up that, if you consciously _look_for_ the lines that are trying to be "inconspicuous", they're easy to find as well. a good example is _footnotes_. they _try_ to hide, putting themselves at the bottom of the page, and rendering themselves in a smaller point-size. pretty sneaky little devils, aren't they? but not too smart, it ends up. because if you _look_ for inconspicuous -- small type, at page-bottom -- it's remarkably easy to find footnotes too. even if you have stripped out the fontsize information from your o.c.r. file -- which, just by the way, is an _incredibly_stupid_ thing to do, even though many "experts" do it, including the good people over at distributed proofreaders -- you can _still_ recognize the footnotes easily, since their smaller type meant that their lines end up having more characters than a line of body-text. so if you count the characters, to look for longish lines, there they are! (this assumes that you've kept the original linebreaks, which the good people over at distributed proofreaders _do_, but other "experts" do not. what can i say?, except that if you make _enough_ stupid decisions, yes, you too _can_ make the digitization job _difficult_, if you _want_ to...) *** anyway, jon, i hope that answers your "challenge" on header-recognition. it's an easy task. i know you've spent years telling people that it is hard, that it's a job requiring "artificial intelligence" that will not be available for the next two or three decades -- "if then" -- but you are plain wrong. indeed, i put up an entry in my blog some time back on this very topic, announcing "the fastest-growing quiz sensation in america", which is the game "am i a header?", where i ask people to submit lines to me and have me guess whether or not that line is a header. come play sometime! my blog is at: http://journals.aol.com/bowerbird/bowerbirdseyeview oh, and jon, if you have any more "challenges" for me, let me know, but hurry up, because you never know when i'll be banned from here. oh, and also, if you ever want to take up my "challenge" to you, and mark up that test-suite of mine, you should -- of course -- feel free. and i'm still waiting for that list, from you and/or networker, on what kinds of semantic structures you think need to be marked up. -bowerbird p.s. it wasn't my "logic" that john ockerbloom had a problem with. he just didn't want me to say that "my resolution for 2005 is to get off of the jon noring merry-go-round, because it's so senseless to keep on debating the same old stuff over and over and over and over." but since people _here_ want you and i to get off the merry-go-round, i'm guessing their only objection is that i haven't done a better job of keeping my resolution...

1 0

Re: [gutvol-d] Technical Collection for Consideration.
by Jonathan Ingram 18 May '05

18 May '05

--- Marcello Perathoner <marcello(a)perathoner.de> wrote: > Jonathan Ingram wrote: > > > Sadly the overly long length of modern copyright terms mean that we won't > see a > > large influx of public domain computer programming material for a while. > > Many FSF books are copylefted. Indeed, but this is a very different thing from being public domain. -- Jon Ingram __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail

3 2

re: 20+ hours per week on PG
by Bowerbird＠aol.com 18 May '05

18 May '05

greg said: > There are only a handful of people who > consistently spend 20+ hours per week on PG i just had to respond briefly to this statement. there are a few dozen people over at distributed proofreaders who report they work anywhere from 20-40 hours each week... -bowerbird

1 0

re: [gutvol-d] so, jim, please tell me what it is that you do
by Bowerbird＠aol.com 18 May '05

18 May '05

greg said: > I'll write a quick intro to what Jim does. Meet Jim: thanks, greg. but i know what jim does. i've known what jim does for a long time. i'm even one of a probably-small number of people who has gone through the source-code of gutcheck and chuckled at jim's humorous comments there... i'm sure jim does _more_ than i know about, perhaps even more than you know about, greg, but of the things you've told us that he does, all of those are things that _i_ already knew... i don't know how many hours jim spends doing it, but he does _so_much_ that i would be _amazed_ if he wasn't spending lots and lots of hours doing it. i don't know how he finds enough to do it all, frankly. i wasn't knocking jim, and his valuable contributions. but he was knocking me. so i wanted to know why. more to the point, though, i think jim was saying that i don't understand what _everyone_ there does -- not just him personally -- and he might have meant a "mission statement" sense as well as actual tasks. (or maybe not, i'm only speculating.) again, while i am sure there is a dizzying array of a myriad of tasks necessary to keep the big machine whirring away, some of which i am blithely unaware, i firmly believe that i have a _very_ solid grasp of steps necessary to digitize a paper-copy of a book into a full-fledged, high-powered electronic-book, and to do that in the high-volume way intended to create the cyberspace library of the future which will make every book available to every person in every place on the globe at every hour of every day. furthermore, i've had a very solid grasp on all that for several decades now. but i'm still not too old of a dog to learn new tricks, if you can teach any... so teach, if you think there's something i need to learn. because _i_ am always willing to hear someone tell me the things that they think that _i_ need to hear... -bowerbird

1 0

re: [gutvol-d] Severe Resistance to EBooks.
by Bowerbird＠aol.com 18 May '05

18 May '05

branko said: > I think annecdotal evidence is no evidence at all. then why call it "evidence" at all? it is indeed a certain type of evidence, and as long as you understand and accept its limitations, it's useful. > It is useless to discuss the growing or > lessening popularity of ebooks in general > based on a few of Vijay's personal observations. it would be unwise to trust his short-term observations over our long-term observations if the two conflicted, but so far i would say they've been in close alignment... perhaps it upsets some people here that a person does not need to be an expert to realize the important issues, but that's the way this particular situation is... > Those personal observations can be fun to talk about > in their own right, but they are not evidence of a trend. they are _not_ much fun to talk about any more, are they? isn't that why nobody's really responding to vijay's posts? they've already been discussed to death on the listserves. but perhaps it's not good for us to be so jaded, and thus reminders of the issues by seeing them through new eyes might be just the ticket for waking us from complacency. for instance, notice how the issue of the viewer-program surfaces so often in vijay's observations? that's a focus that i think is extremely important to the end-readers, but most of the "experts" here just gloss over it entirely... -bowerbird

1 0

Severe Resistance to EBooks.
by rvijay07 18 May '05

18 May '05

I spoke to about 10 people in the last week that I personally know. The aged are hesitant to stay long infront of the Computer due to eye strain etc., Those who go to school etc., are busy with their studies. In general society still hasn't got comfortable with the idea of EBooks. It is something that they will read if they have to or have no other choice it appears. Even I myself started liking EBooks only like a month ago. Hence, PG may have a bumpy growth as and when more people get comfortable with using EBooks. These people by the way are not familar with ebook readers as well and the advantages that they offer. Share your comments/experiences in this regard. Regards, Vijay _______________________________________________ No banners. No pop-ups. No kidding. Make My Way your home on the Web - http://www.myway.com

7 6

re: [gutvol-d] Listening to EBooks in Text Format !!!
by Bowerbird＠aol.com 18 May '05

18 May '05

vijay said: > Has anyone here tried this software ? > http://www.spacejock.com/yRead.html text-to-speech is nice, but perhaps even more relevant here is his viewer-program: > http://www.spacejock.com/yBook.html i haven't tried it, because it's p.c.-only, but ybook gets fairly good reviews, and it has a 2-up facing-pages interface, which i like. it will download the p.g. catalog for display, and then let you download individual e-texts, so it's a cutting-edge illustration of how people will soon be able to relate to project gutenberg as a _library_, and not just a bunch of e-texts... -bowerbird

1 0

re: [gutvol-d] no, the problem is
by Bowerbird＠aol.com 18 May '05

18 May '05

melissa said: > Last time he went away from the DP forums, > he threatedned to tell the world the truth about DP > --is that the one? "threatened"? to "tell the truth"? are there some desperate housewives over at d.p.? that's quite a vivid imagination you've got, melissa. there are some problems with the d.p. workflow -- some rather glaring ones, when you consider how many people are laboring under those flows, who should see where they need improvement -- and i will make independent digitizers _smarter_ about what chronology they should use instead, but that's hardly an expose on "entertainment tonight"... so you're just more noise here, intended as a smear. and speaking of gutvol-d as an "umbrella" listserve, i'm wondering how many of the subscribers here are interested in the slightest bit about the specifics of .tei markup, of the type permeating the threads today? it's another example of the techies chasing people away. -bowerbird

4 3