September 2012 - gutvol-d - lists.pglaf.org

Re: [gutvol-d] DP protocols
by don kretz 28 Sep '12

28 Sep '12

Jon/Roger This appears functionally equivalent to the effort undertaken by several of us 3 or 4 years ago to extract the text from several projects and post them to readingroo.ms as non-PG certified "preview" texts. Advance approval was given by Jon Noring on behalf of PG, and was required from each of the Project Managers. Louise was aware but functionally silent. The transfer mechanism was established and put into operation by volunteers using existing DP interfaces entirely and disrupting no existing DP operations or personnel. No one in the DP hierarchy showed any positive interest or volunteered assistance. As I recall PM participation was mixed; but this was advertised as being for a few willing participants, not the general run of projects. After a short period of operation several dozen projects were transferred. At that point Louise declared the whole operation illegal and demanded it be terminated and any existing project texts be removed without exception from readingroo.ms, which was done. The "rule" that was established thereby was that the only texts allowed to escape into the wild would be those which had undergone the entire DP process from P1 through PP/PPV. Except, as I said before, by going through Louise first. Sent from my Phone From: Roger Frank Sent: 9/27/2012 3:05 AM To: Project Gutenberg Volunteer Discussion Subject: Re: [gutvol-d] DP protocols On Sep 26, 2012, at 2:03 PM, Jon Hurst <jon.a(a)hursts.eclipse.co.uk> wrote: > Quick question to Don/Roger/Anyone who PM's at DP. > > Assume Greg has got us a scan that of P&P. What DP rule are we breaking > by skipping P3, F1, F2 and doing a simplified PP consisting of diffing > against extant text and converting to UTF-8? I don't think there is a "rule" for this at DP. If you want to attempt it, get DP management to buy off on it first and then be very clear in the project instructions that the users' work is not going to be taken to a finished book on the DP site. I believe some users would choose not to work on a project, especially with LOTE rules, that is only doing P1 and P2. That's not the DP workflow. If you were to get that done, I understand that the P2 output is the RTT in your scheme. That means that anyone who derives a final version (HTML, epub, kf8 etc. file) from it has to find and handle all the markup like intalics and superscripts on their own. It's not coded in the RTT if it's coming out of P2, since formatting is not present in the proofing stages, even inline formatting. That is a serious shortcoming to me. --Roger _______________________________________________ gutvol-d mailing list gutvol-d(a)lists.pglaf.org http://lists.pglaf.org/mailman/listinfo/gutvol-d

2 1

Re: [gutvol-d] blah blah blah blah blah
by Bowerbird＠aol.com 28 Sep '12

28 Sep '12

jon said: > I'm up for this, but I would suggest that > we use a scan that Greg nominates > and uploads to PG as a master scan. if greg finds some scan-set other than those 3, then i will be happy to upload that one as well... > That way the result will be > objectively better than the extant text > as it will be a version of the master scan > rather than a version of an unknown edition, p.g. doesn't consider its current e-texts to be inferior just because they come from "an unknown edition". on the contrary, p.g. often tries to cast this as a "benefit". your mileage might or might not vary... > and there will be no excuse for not > replacing the old version with the new. unfortunately, you're thinking inside _your_ head instead of thinking inside the whitewasher heads. the whitewashers will not replace the _current_ version of an e-text with one that you submit... (so your "old" and "new" terminology doesn't fit.) they'll assign your new text a brand-new number, and it will start with zero downloads, and thus it will automatically be at a _distinct_disadvantage_. their thinking is that why should your version "benefit" from the downloads accrued by the earlier version, done by a pioneering volunteer. and you can't argue that that is unreasonable. (and even if you think you can, you will not be the judge as to what constitutes "reasonable"; the whitewashers will be making that decision. and you can bet that they will find _their_ logic to be much more compelling than _your_ logic.) the whitewashers make an exception to this rule when _they_ update their _own_ submissions, but you would expect them to do that, wouldn't you? now, maybe since you've made a stink here, they will also grant an exception to your project, but if i were you, i'd wanna get the promise in public. otherwise, just get used to the fact that your work will largely be ignored. if you're ok with that, great. you will still know, underneath, you did a good thing. > I would also like to do a LaTeX version, so will > need a stage before you add your formatting -- > it doesn't have to be my RTT, > but the lines need to correspond. i'm just mounting the projects. somebody else will have to clean the text. and add the formatting. i mean, i _might_ do some of that stuff, but i'm not making any guarantees that i _will_... but i doubt i will. i have bigger fish to fry... also, i think it's silly not to add the formatting, so you'll have to _remove_ any that i've added. (but that would strike me as immensely stupid, because then you'll just have to add it back in.) and, as for the lines needing to correspond, that is an acceptable format for a z.m.l. file, and it has been a thing for which i've agitated for a good many years now. just so you know. i think rewrapping lines is a stupid bad idea... *** the scans are up now, for that first mounted text, meaning we can now make the outline of a system. > http://zenmagiclove.com/prxpr/prxprp123.html adjust your browser's window-size and zoom level until you get your best presentation of the display... -bowerbird p.s. another interface should be created to _diff_ this o.c.r. against a clean text to correct the errors. i might do it, if i think it'll be fun, and i have time; but i invite anyone else who wants to play to build it. if you do so, here's a clean text that you can use: > http://zenmagiclove.com/prapr/prapr.zml >

5 8

let's step through this process.003
by Bowerbird＠aol.com 28 Sep '12

28 Sep '12

yay, another update! "prhpr-005.zml" is now online: > http://zenmagiclove.com/prhpr/prhpr-005.zml there were 7,607 lines changed from "002" to "003". and there were 515 lines edited from "003" to "004". and 275 lines were revised between "004" and "005". as we complete more of the "global"-type changes, the number of edits decreases, as you would expect. as before, you can view all of the revised lines via: > http://zenmagiclove.com/prhpr/docomp45.py and yes, the pattern continues with an .html file: > http://zenmagiclove.com/prhpr/prhpr-005h.html plus we've now updated all the page-by-page files: > http://zenmagiclove.com/prhpr/prhprp222.html you will have to adjust the zoom-level and the size of your browser-window for the best presentation. *** as i said up above, many of the "global"-type edits have now been done. that's the fun part for me -- seeing what find-and-replace operations are needed, predicting how many "hits" each one will enjoy, and then seeing how accurate i guessed, all while moving the book toward perfection via thousands of changes -- but from here on out, it loses much of its appeal, at least for me. turns into something more like work, and then into a job (an unpaid job), so i ain't gonna continue with this much longer. but i will do a _few_ more steps, to show you, and see how they turn out. the first one is to drop the text into a little app that spits out all the words that are not in the dictionary. that's absolutely the fastest way to find o.c.r. errors. i might (or might not) fix the bulk of those errors... yet another step i'll do is compare this text to a file that is _clean_, using the scans to resolve any diffs... that's also how i'll inject the styling the o.c.r. left out. all that will give me an output that is extremely solid. from there, it'll be time to call in the beta-readers... so, what've you been doing the last couple days? ;+) -bowerbird p.s. so nobody noticed that thing i've been hinting?

1 0

before you leave, jon
by Bowerbird＠aol.com 28 Sep '12

28 Sep '12

before you leave, jon, maybe you can answer a couple of questions for me... you once coded a little demo that had a page-scan that scrolled line-by-line in a window, according to the position of the cursor in an adjacent textarea... my questions are concerning that code. specifically, was it python or javascript? i think you made the code open-source. if that's correct, where can it be found? and, maybe the most important of all: how was it, exactly, that you did that? the synchronization part's easy enough, but how did you get the coordinates of each line? did you do that on-the-fly? can you get them for an arbitrary scan? i do this in my offline (realbasic) apps by assessing the color of the pixels at several specific points in the scan, but i don't see how to do that in a browser. so is _that_ what you were doing there? and finally, if that is indeed what you do, can you do such pixel-color-probing on any old div (or screen location), or will it only work on graphics, like page-scans? -bowerbird

1 0

Re: [gutvol-d] blah blah blah blah blah
by don kretz 28 Sep '12

28 Sep '12

Sent from my Phone From: James Adcock Sent: 9/28/2012 7:22 AM To: gbnewby(a)pglaf.org; Project Gutenberg Volunteer Discussion Subject: Re: [gutvol-d] blah blah blah blah blah > It's OK to choose a title we already have. Tell that to your webmaster and WW'ers who are sending a different message. Again, it is not always trivial for volunteers to provide images, as sometimes the organizations making the images available, rightly or wrongly, claim to put restrictions on the use of those scans. Clearly the text of a "risen to the public domain" work is something PG can accept and post, and the text input of a "risen to the public domain" work is certainly something a volunteer can make and send to PG, so I think that part is well understood. What seems to be less well understood is that providing the actual images could in some cases prove to be more troublesome, in practice, to volunteers. When images can be found in some public location, like archive.org, or are available in multiple places, I usually provide that information as part of the copyright clearance request. If PG thinks that getting and storing a copy of those resources is appropriate, PG should do so. _______________________________________________ gutvol-d mailing list gutvol-d(a)lists.pglaf.org http://lists.pglaf.org/mailman/listinfo/gutvol-d

1 0

One last post
by Jon Hurst 28 Sep '12

28 Sep '12

David Starner wrote: > For all your claims, you advanced RTT as a master format and are now > storming off when it didn't get accepted as the master format. Please do not think I am "storming off". I had a plan that I thought might work. I did the reconnaissance. The plan had a fatal flaw that could not be mitigated. The people that I would need buy-in from didn't buy the core principal that you could have a foundation without specifying a master format. Everyone _wanted_ a master format; so much so, indeed, that they immediately assumed what I was trying to sell must de facto _be_ a master format, and a poor one at that. I am not Bowerbird. I don't think I have all the answers, I don't think that I am always right, I don't think I can insult people into consensus and I don't think that running a solo project and trying to show everyone how clever I am achieves anything whatsoever of consequence, particularly when that solo project isn't actually very impressive. I have listened to what everyone has said and my conclusion is that the problems with the PG library cannot be solved in the way that I suggested they might be. I now have an good understanding of why that is, so I can happily move on to something more productive. Cheers Jon

1 0

DP protocols
by Jon Hurst 28 Sep '12

28 Sep '12

Quick question to Don/Roger/Anyone who PM's at DP. Assume Greg has got us a scan that of P&P. What DP rule are we breaking by skipping P3, F1, F2 and doing a simplified PP consisting of diffing against extant text and converting to UTF-8?

5 7

a master format that has no formatting
by Bowerbird＠aol.com 27 Sep '12

27 Sep '12

roger said: > That means that anyone who derives a final version > (HTML, epub, kf8 etc. file) from it has to find and handle > all the markup like intalics and superscripts on their own. > It's not coded in the RTT if it's coming out of P2, > since formatting is not present in the proofing stages, > even inline formatting. > That is a serious shortcoming to me. it's more than "a serious shortcoming". it's a fatal flaw. anyone who's done a digitization all the way _knows_ that inline formatting is one of the excruciating parts. even the "overall" formatting can be a whole lot of work, but it's generally among the more "fun" parts of the task, which offsets it a bit. but we still have to admit it's work. and some of the finer points (like _proper_paragraphing_) often take lots of time, relatively, and drive you crazy too. so to pretend that a text-file that contains no formatting can be considered as a "master" in any sense of the word is a denial of reality, one that's silly, if not plain ridiculous. and jon will learn this very quickly, just like everyone else. but all you people who know better shouldn't just let him proceed blithely as if this part of his plan was reasonable... because it's not. it's a fatal flaw. -bowerbird p.s. yes, i'm aware that jon said that he'd dialed back the notion of r.t.t. to sidestep all the "religious" wars. that doesn't remove the fact that r.t.t. is unworkable. p.p.s. no, the irony is not lost on me, not lost at all. sometimes people on this listserve make _assertions_ that something is true, when that thing is _not_ true, and has never been true, and likely never _will_ be true. yet other times when there's something that you _know_ to be true, you just sit on your hands and don't say jack. it's as if _truth_ means absolutely _nothing_ to you. weird.

4 5

i must tell you
by Bowerbird＠aol.com 27 Sep '12

27 Sep '12

i must tell you that i am enjoying these threads immensely. because while y'all are doing your endless discussion thing, there's an app right over on my site that is the pudding that is proving that z.m.l. works quite nicely as a master format. you put z.m.l. into a field, and then click a button, and boom!, out pops nice .html, and an .epub, and a .mobi. the offline equivalent app spits out a nice .pdf as well... coming soon is output in the form of an _e-book_app._ so keep up the chatter, kids... it's immensely entertaining... -bowerbird

1 0

let's stop treating our computers as if they are retarded
by Bowerbird＠aol.com 27 Sep '12

27 Sep '12

don said: > We're currently not collecting most of that, > except by inference from the layout; of course, "by inference from the layout" is _precisely_ how we humans understand the structure of our books. despite the fact that headers are not _labeled_ as headers, just to give the most obvious example that comes to mind. we still seem to be able to point out a header just _fine_! likewise, thank goodness we humans don't seem to need to have every paragraph _labeled_, because that would be extremely obtrusive, and unnecessarily awkward, right? let's stop treating our computers as if they are retarded, and need to have everything explicitly spelled out before they are capable of understanding and categorizing it... there are no stupid computers; only inept programmers. for instance, it's quite easy to write a routine that can recognize the headers in any project gutenberg e-text with a very high degree of both certainty and accuracy. and in the cases where our easy-to-code program does make a mistake, we can easily edit the offending e-text, so that it will _not_ cause a problem for our nifty script. ditto with everything else you mentioned, don... -bowerbird

1 0