July 2005 - gutvol-d - lists.pglaf.org

re: [gutvol-d] XML: Interpublication and deep linking
by Bowerbird＠aol.com 15 Jul '05

15 Jul '05

jon said: > One argument brought up here yesterday is that > the interlinking of publications, especially deep linking > to spots within documents, is so brittle as to be useless > (e.g., move the resource, the URL changes, voile' the link is broken.) that wasn't my "argument" at all. my argument was that people who make promises about their ability to pull off deep linking cannot back them up _unless_ they posit a means of dealing with impermanence. but if such a means is accepted, _anyone_ can do deep linking. > It does, however, require a unique identifier for each resource. and here it is in jon's post: weaseling via a "unique identifier". heck, if everything has a "unique identifier", _i_ can give you a deep-linking system based on zen markup language. easily. and -- as always -- _without_ the cost of any heavy-markup. you're just shifting the permanence requirement from the u.r.l. to the unique identifier. it's sleight-of-hand, and nothing more. > the W3C suite of related technologies of XLink/XPointer/XPath/xml:id/etc. -- one trick that a hand-waver counts on to distract you is to point to a bunch of acronyms as if _they_ solve the problem. but if you look at them closely, they're mired in the same muck. > It is not *my* system at all (as the poster alluded to), > and makes no difference whether I myself > build some sort of linking system or not. it makes a difference whether _anyone_ builds such a system or not. considering how widespread and well-developed you claim x.m.l. to be, why isn't such a system _already_ developed and out proving its worth? as it is, i see the ability to link to paragraphs, but _only_provided_that_ every paragraph that you want to link to has heavy markup attached. i suppose we could deep-link to every darn _character_ if we wanted to, provided we were willing to bloat up every file to an unbelievable level. (in this regard, see > http://diveintomark.org/archives/2004/05/30/pink-numbers#w200405302127-53 for a silly word-based version.) > To reiterate, XML provides standardized and convenient hooks > for linking and deep linking into publications. you always _state_ that x.m.l. solves the problems. but you never tell us _how_ it solves the problems. you drop acronyms, wave your hands, and that's it. as usual, you're asking people to buy a pig in a poke. and as usual, i'm asking to see the pig, not the poke. -bowerbird

2 1

re: [gutvol-d] Scans and Texts (Re: Copyright Verification?)
by Bowerbird＠aol.com 15 Jul '05

15 Jul '05

greg said: > An automation process, to pull all images from DP > at a time an eBook is posted, is very much non-trivial. copying images from one place to another seems "trivial" enough to me. the non-trivial part will be setting up the ground-rules for the page-scans, and then making the current set of scans conform to those ground-rules... although i would imagine (or hope, anyway) that are smarter now, as recently as last year, i encountered _extremely_ stiff resistance at d.p. for suggesting even such _basics_ as file-naming conventions (e.g., that the scan for page 87 should be named "rootname087.png" not something like "rootname094.png", as was typical.) in a nutshell, if page-scans are maintained in the chaotic way that many other files are being handled in the library, you'll have a nightmare on your hands. one e-text can have literally hundreds of page-scans associated with it; thus, with 17,000 e-texts, we're talking now about 2-4 _million_ files. if you approach a task like that haphazardly, it will bite you back _bad_. simply put, the scans weren't created and haven't been maintained with an eye toward making them publicly available. that's not the _fault_ of the d.p. people, because that was not their concern, but it is a reality that needs to be faced if we want to make them public. it's going to be a _lot_ of work to mold them into something useable. juliet recognizes that -- that's precisely what she was telling people. > But just doing one or two titles as a sample would help. well, that would depend on whether wise conventions are adopted first, or develop out of those samples. if these samples instead merely boost the "do it however you want" idea that permeates the rest of the library, they will do more harm than good. it's also very important to understand that the stimulus here is all wrong. rather than driven by some vague notion that scans "should" be available, a "needs analysis" has to be done to determine _who_ will use the scans, and _how_, so that the policies that are put into place are _wise_ ones. for instance, i might be wrong about this, but i think the current policy is to wrap all the page-scans up into one zip file. there are merits to that, but it's also the case that a zip file is not the best thing for many useful applications, not the least of which is checking the scan of a single page (e.g., to see if an error-report is supported or negated by the page-scan). (in regard to this specific point, i think scans should be stored both ways -- as individual files and as a single zip-file -- perhaps on different servers.) as there is, at present, no user clamor for the scans, we _do_not_know_ how the end-users might want to use the scans, so we're _in_the_dark_ about the factors that we should apply in the development of any policies. so if i were a decision-maker, i would wait until a clamor actually developed before i moved forward on this. perhaps the absence of many volunteers who are willing to actually expend their energy on this project will serve as the brake necessary to keep it from lurching ahead prematurely. > As Juliet & others mentioned, the *archiving* is already being done. > The next step is distribution. well, i would imagine (or hope anyway) that a formally-trained librarian would use the term "archiving" with a little more sensitivity. the scans are being _saved_, but there is a huge gulf to cross before they can be considered to have been "archived". the page-scans as they are now are a _very_ long way from being ready for the "distribution" step... maybe jon can mobilize some volunteers to do all the work necessary. but otherwise, i don't see any people stepping forward at this time... -bowerbird

2 1

XML: Interpublication and deep linking
by Jon Noring 15 Jul '05

15 Jul '05

One argument brought up here yesterday is that the interlinking of publications, especially deep linking to spots within documents, is so brittle as to be useless (e.g., move the resource, the URL changes, voile' the link is broken.) Thus, it is concluded, this particular advantage of XML to enable interpublication and deep linking systems, is not pragmatically an advantage at all. (With XML documents, deep linking can be accomplished using the W3C suite of related technologies of XLink/XPointer/XPath/xml:id/etc. -- in XHTML there is the well-known "id" attribute to enable deep linking using fragment identifiers, which web browsers *now* use.) A recent article by Sam Vaknin shows, however, that there are clever ways to maintain the integrity of links even if target resources are moved around: http://www.webpronews.com/ebusiness/contentandcopywriting/wpn-6-20050712Dig… DOI was brought up as one solution, and DOI is certainly powerful. In addition, the Publication Structure Working Group at OeBF delved into this issue a couple years ago, and although issued no specs, concluded that linking (both interpublication and internal) using a special URI (with embedded XPointer stuff) is possible where the target resource does not have to exist at a particular spot in some network. It does, however, require a unique identifier for each resource. (PG already has a crude identifier system, but because of how it is assigned/structured makes it somewhat more difficult to use for interpublication linking purposes -- a WEMI-based modularized system for identifiers is definitely more powerful and flexible.) The important point is that XML markup provides the needed hooks to enable all kinds of systems to interlink between, and deeplink into, publications. It is not *my* system at all (as the poster alluded to), and makes no difference whether I myself build some sort of linking system or not. To reiterate, XML provides standardized and convenient hooks for linking and deep linking into publications. That's all. This is an advantage, one of several, of using XML for mastering PG's digital texts. That's why mastering the PG texts in XML makes sense: we are preparing the texts not only for benefits today, but for yet unforeseen future uses. Jon Noring

1 0

Re: [gutvol-d] Initial thoughts on a PG/DP scan repository
by Joshua Hutchinson 15 Jul '05

15 Jul '05

----- Original Message ----- From: "Jon Noring" <jon(a)noring.name> > And finally, since DP is the major player, should we move this > design discussion to the relevant forum at DP? Or keep it here on > gutvol-d? > If you want anything to happen any time soon, keep it here. DP will *not* be moving on this issue any time soon (not without an influx of new developers that decide to push forward on this issue, anyway). As JulietS said, there are plans to get there eventually, but there is a fair bit between where we are now and where that step comes into play. JHutch

2 1

re: [gutvol-d] Scans and Texts (Re: Copyright Verification?)
by Bowerbird＠aol.com 14 Jul '05

14 Jul '05

marcello said: > Are these scans online and accessible at DP ? > If so, linking them from the PG catalog would be > a matter of a few hours work, > assuming I can get a list of etext-no => page-scan-url would that it were that easy... *** not to pick on marcello, since i wouldn't do that, because i know he does a lot of work on the p.g. site, so he's busy with other things... but... this thread would be a lot more productive if people would familiarize themselves with the actuality of the scans on the d.p. site... some of them are not of very high quality. many of them are not very well-organized. (although it might not appear that way to the naked eye, that last sentence is the very model of understatement.) *** juliet said: > Aside from not having the development resources to > set up some kind of system for accessing and using the scans, > we also have not yet found someone who will wade through > all the archived material to sort it out so that it can actually be used. i heard jon noring volunteering! just three posts back! really! :+) *** david said: > I've looked at a similar system on Project Runeberg, > but it doesn't look very successful at > outputing a number of accurate and complete etexts. > DP does a very good job at keeping attention focused on a few texts > and keeping them moving forward page by page in the system, > where as those system seem to disperse the effort over a lot of books > and a lot of pages that are corrected more or less at random. that's a good analysis. of course, it's also important to note that neither of these projects has volunteers anywhere near the numbers that d.p. has. if they did, their performance would be better; they'd be more focused, and get better results. note that the general problem here -- which is "how do you know when each page is _done_?" -- is one that distributed proofreaders has too. but because of its numbers, and the dedication of its proofers, d.p. has the luxury of answering "when it has gone through x number of rounds." even though that's not the right answer to the question -- the right answer is "when no more changes are being made" -- by the time you've hit 4 rounds (plus post-processing), the odds are much better that all the errors were found. another difference between all of these systems is that the d.p. interface is remarkably better than the others. furthermore, the post-processing apps over at d.p. are some of the best around right now, so that helps too... but if you built a system of continuous proofreading that had an interface as good as the one over at the d.p. site, with as many volunteers spending as much time, and with as much care, it would do as good as d.p. does. or better... still, d.p. _has_ the volunteers, right now, working hard, so there's little need to channel them to another method. they've got a nice little cult there, i mean "community", :+) and as long as they're happy, we should just cheer 'em on. and since there are very few calls -- outside of jon -- for the kind of "transparency with the source materials" that putting scans online would serve, i don't see much point in doing it yet. eventually, diskspace and bandwidth will be plentiful enough to do that without any reservation; but that day ain't here yet. indeed, distributed proofreaders doesn't even leave its scans on brewster's internet archive after that book has gone through proofing, simply because they don't have enough diskspace to do that. it's true. the more important issue here is one that jon is trying to slip under the door while he makes a big noisy distraction -- his emphasis on a complex and heavy form of markup... jon wants you to believe this heavy markup is necessary to deliver a whole bunch of benefits that he will promise. but he has no way to deliver those promises. and i suspect that when he gets deep enough in the fertilizer he's pushing, he'll find his complex systems start tripping over themselves. for instance, even some of the most diehard of x.m.l. people are now saying that x.s.l.t. is too complex for many purposes. or let's look at inter-document linking, which is one of the things that jon likes to say is on his agenda, while he's waving his hands. this is an arena that many good people have wasted a lot of time on. i say "wasted" because either one of two conditions must apply here: 1. documents are permanently available at a u.r.l., or 2. they are not. if condition one is the case, anyone can build an inter-linking system that works perfectly. and if condition two is the case, _nobody_ can. oh, they'll make you _promises_. and they'll be able to build something that _kinda_ works, _most_ of the time anyway. but anyone can do that. so think long, hard, and very carefully about adopting heavy-markup... (but hey, if you _are_ gonna do heavy-markup, _when_will_you_start_? i came to this listserve over a year-and-a-half ago, asking you that, and you _still_ haven't gotten the ball rolling yet. what is the delay, folks?) -bowerbird

1 0

the trouble with listserves
by Bowerbird＠aol.com 14 Jul '05

14 Jul '05

the trouble with listserves is that there seems to be no collective memory. so a person -- like jon noring -- can take everyone on a merry-go-round, discussing the same old topics over and over and over and over, without ever building on the wisdom of past trips. and -- like it or not -- jon _is_ the king of vaporware, along with his publicist, david rothman of the "teleblog", where openreader (which, um, does not exist) has been mercilessly flogged to death for the past year and a half. ccel and runeberg might be early examples of "continuous proofreading", but i've laid out a better and more modern description, with demo. the software it requires is so simple to write that i can do it myself. i've also described the whole workflow around it, a workflow that does _not_ have a high cost attached that makes it dependent on the goodwill of volunteers, volunteers who have _not_ flocked to support jon's ideas. so you can settle for jon's handwaving away the problems with his approach if you want to. but why would you? -bowerbird

1 0

Orphan works roundtable
by Wallace J.McLean 13 Jul '05

13 Jul '05

The key point being that the deadline to request a seat at the table is imminent! Get your names in by Friday. The comments and reply comments during the first round of this consultation were very interesting. If anyone who's planning to attend either the Washington or Berkeley sessions would like to discuss (offlist) the failings and foibles of the Canadian "unlocatable" regime, drop me a line. ----- Original Message ----- >From "Branko Collin" <collin(a)xs4all.nl> Date Wed, 13 Jul 2005 01:21:10 +0200 To gutvol-d(a)lists.pglaf.org Subject [gutvol-d] round-table consultations on the orphan works issue Someone posted the following to the Project Gutenberg forum of Distributed Proofreaders: "There will be formal round-table consultations on the orphan works issue this month and next: "http://www.copyright.gov/fedreg/2005/70fr39341.pdf "Quote: The public roundtable will first be held in Washington, D.C. on Tuesday, July 26, 2005, from 9:00 a.m. to 5:00 p.m. E.D.T., and on Wednesday, July 27, 2005, from 9:00 a.m. to 5:00 p.m. E.D.T. An additional roundtable will be held in Berkeley, California on Tuesday, August 2, 2005, from 9:00 a.m. to 5:00 p.m. P.D.T. Requests to participate in the roundtables must be received by the Copyright Office by 5:00 p.m. E.D.T. on July 15, 2005."

1 0

re: [gutvol-d] Copyright Verification?
by Bowerbird＠aol.com 13 Jul '05

13 Jul '05

jon said: > My focus on scanning goes beyond just OCR purposes -- > I think if substantial work is being expended to acquire and scan a book, > it takes only a little extra effort to scan at archival quality, which is > at least 600 dpi optical > (and 256 color greyscale for bitonal and even better 24-bit color.) i used to agree with you on this, jon. when a page was originally printed as black ink on white paper, i do not see why to scan it as anything but b/w. you're just increasing the difficulty. it was your example project of scanning "my antonia" that convinced me of this. :+) the huge size of those scans made them unnecessarily hard to create and process, in terms of time, difficulty, and resources. (far more than "only a little extra effort".) the realization struck home more fully with your "kama sutra" scans, when you asked for advice in what was essentially converting them _back_ to black/white. color pages? certainly. scan them in color. black-and-white pages? black-and-white scans. and 300 d.p.i. is probably a high enough resolution. if the future wants 'em higher, let the future scan 'em. > http://www.geoffhorton.com/pictureocr/instructions.html > has an explanation of what I do. geoff, i certainly didn't mean to negate your efforts at using a digital camera to scan. as i've said before, i think if you've figured out the factors that will lead to better results when taking that approach, you've learned some very important information, and i applaud you for stepping forward and sharing it with people... :+) -bowerbird

5 4

re: [gutvol-d] DP ReCoded
by Bowerbird＠aol.com 12 Jul '05

12 Jul '05

juliet said: > Bottlenecks are developing, as we expected that's only natural at first. > Bottlenecks are developing, as we expected > between some of the rounds. But, > the material that gets through the process > is of much higher quality and is making the > jobs of the post-processors much easier. that's great to hear, since a big part of the intent was to address the backlog in the post-processing queue... maybe some volunteers here who tried d.p. and maybe didn't like it too much before will go back and give it a new shot... thanks for the report. i'm glad to hear it's going so well! :+) -bowerbird

10 10

round-table consultations on the orphan works issue
by Branko Collin 12 Jul '05

12 Jul '05

Someone posted the following to the Project Gutenberg forum of Distributed Proofreaders: "There will be formal round-table consultations on the orphan works issue this month and next: "http://www.copyright.gov/fedreg/2005/70fr39341.pdf "Quote: The public roundtable will first be held in Washington, D.C. on Tuesday, July 26, 2005, from 9:00 a.m. to 5:00 p.m. E.D.T., and on Wednesday, July 27, 2005, from 9:00 a.m. to 5:00 p.m. E.D.T. An additional roundtable will be held in Berkeley, California on Tuesday, August 2, 2005, from 9:00 a.m. to 5:00 p.m. P.D.T. Requests to participate in the roundtables must be received by the Copyright Office by 5:00 p.m. E.D.T. on July 15, 2005." -- branko collin collin(a)xs4all.nl

1 0