April 2010 - gutvol-d - lists.pglaf.org

Re: Removing spurious break lines
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

al said: > However, it seems to me that simply indenting > not-to-be-formatted lines, and doing nothing else, > is at least somewhat pointless. it's not "pointless" to make _some_ "corrections", but not all, to a text. that's what whitewashers do quite often, when _you_ "correct" a text... > Many of PG's older texts have other problems than that > (missing illustrations, ASCII only rather than Latin1 or UTF8, > missing/incomplete indexes, etc, etc, etc.) that's right. but you whitewashers don't fix all of those problems. you apply the corrections that've been submitted, and you run the text through the new version of your tools, and you might also do some other checks (and thanks for doing that), and then you post it. > A far more desirable approach would be to > pick an old PG text, find a scanset in IA/Google/wherever, > get a copyright clearance, and do a new version of it, either > from scratch, or a complete re-proof of the existing file(s), > doing whatever is needed to bring it up to current standards. i fully agree that that would be "more desirable". it would also be a heck of a lot more work. and that's the trade-off, is it not? still, someone who does _some_ of the "corrections" -- whether that is to make the text robust to rewrapping or some other subset of stuff -- is not engaging in a "pointless" exercise. they're improving the text, and -- just as i thank you whitewashers for improving the text when you do _your_ "corrections" -- i would thank any other person who improved the text when they do _their_ "corrections". i fully approve of an iterative process that steadily cumulates "partial corrections"... > Note: Complaints that the Repost team (mostly myself and > David Widger, between us doing considerable clean-up work > on several thousand of PG's old files) "should have done more" > will fall on deaf ears. We're only two people, we're also 2/3 > of the Whitewashers, and 2/3 of the Errata team, and we both > produce independently. Back off. i have never "complained" about the "corrections" by whitewashers. i _have_ pointed out that these "corrections" are _not_ complete, in the sense that many errors and inconsistencies and omissions _survive_ this "correction" process. but again, i do not condemn any "corrections" because they are not complete. i welcome and appreciate _all_ "corrections", even incomplete ones, because they move the text closer to _perfection_, and that's what i advocate... i wasn't "complaining" to report your corrections" are incomplete. i felt the need to point that out because you _do_not_ point it out. you say that "errors were corrected" and you simply leave it at that. i believe that many people probably conclude, from your statement, that you've made a good-faith effort to actually _find_ all the errors -- such as by comparing the text with a newly-obtained scan-set -- when, in point of fact, you have not actually gone to those lengths... nobody is "blaming" you, or "criticizing" you for doing what you do, or for not doing what you're not doing. so there's no need to tell us to "back off". that's insulting, and you really shouldn't be so sensitive. we're just stating the facts. _clearly_. because you haven't done that. so, can we all agree that "partial corrections" are _not_ "pointless"? because that would be a huge step in the right direction, yes it would. -bowerbird

2 1

Re: Typesetting ("gods and fighting men")
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

looks like we lost our mad scientist... or perhaps he's off deep into programming mode, creating wondrous new tools for project gutenberg. at any rate, for anyone out there who's interested, here is my latest .pdf of "gods and fighting men": > http://z-m-l.com/misc/14465-take6.pdf feedback, public or private, would be appreciated... this version retains p.g. linebreaks, for the most part. (i did some rewrapping, to remove egregious orphans.) the use of 10.5-point type made even the longest lines manageable, given the 4.5-inch measure that i used... that's obtained by .5-inch margins on a 5.5-inch page. (if you were printing this at lulu.com, you could specify a 6*9-inch page, which would allow a bigger fontsize.) however, as you'll see, the lines are extremely ragged, with many short lines, since they were wrapped by the character-count, not the length of a proportional font. (furthermore, there was some real weirdness on this, in that many lines seemed to have been counted short; in particular, it was as if the algorithm was _trying_ to create very short lines as the last line of the paragraph. i don't recall having seen this before; it was _strange_.) anyway, because many of the lines were counted short, using full justification on this text would be a disaster. but otherwise, this is a _respectable_ job of typesetting. i'm gonna rewrap this text, using a bigger fontsize, and i'll mount that .pdf later this week. enjoy this one now... -bowerbird

2 1

Re: Removing spurious break lines
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

al said: > I've contacted Joaquin directly. well, i agree that there's no need to engulf an innocent newcomer in an involved discussion about p.g. policy. but that doesn't mean the discussion should be swept under the rug, eh? we'll need an answer to the question: will p.g. accept e-texts that have been "corrected" by virtue of having their not-to-be-unwrapped lines indented? transparency is the new black. sunshine is the best disinfectant. the most colorful fish deserve the most apparent aquarium. so... what is the p.g. policy on this? -bowerbird

3 2

Re: Removing spurious break lines
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

jim said: > In general, if derived formats including ePUB and MOBI from HTML, > also HTML from txt, also unwrapping txt from wrapped txt, > are to work “correctly” then there needs to be *some* degree of > expectation on the formatting of the incoming texts. Otherwise > these tasks cannot be successfully automated. that's true. but i'm not talking just about "derivative formats", because there's no need to create a "derivative" if you'd rather just use the .txt file itself to drive the display, a la "eucalyptus". however, the .txt file does have to be formatted "correctly" if it is to be _displayed_ correctly. that's what's driving my motivation... > Going the other way, the automated wrapping of txt is has > built-in support by most (all?) modern text tools, including > web browsers, e-book readers, text editors, etc. stop trying to derail the thread, jim. there's no way that project gutenberg is going to mount files that don't have mid-paragraph hard linebreaks... _no_way_... so that's not what we're talking about here. and we aren't _going_ to talk about that here, no matter how many times you try to bring it up. so stop trying. what we _are_ talking about now is formatting the .txt files _correctly_, so that they can be unwrapped automatically... -bowerbird

2 1

Re: DP Archives/OLS (Was: Re: DP output ...)
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

dakretz said: > Except for the ones that are purportedly in Charlz' garage "purportedly"? tone down the skepticism, man. save it for when it's needed. > and/or they have a policy not to make available. well, um, yeah... d.p. made a policy decision to "respect" the wishes of some institutions _not_ to repost the scans, even though those "wishes" have no basis in legal rights... in other words, d.p. traded away your public-domain rights for the purpose of maintaining "friendly" relations. oh well. the good news is that, for the most part, anyone else can retrieve the scans from the same place where d.p. got 'em. i don't know if the "o.l.s." tells where the scans came from, but the "credits" portion of the posted e-book often says so. > donovan - are they in fact available somehow from within DP? it probably depends upon who is asking, and for what purpose. then again, don, i'm sure you're well aware of _those_ caveats... -bowerbird

1 0

Re: Typesetting (back on track, calling the mad scientist)
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

we seem to have lost the mad scientist. mike mcd, are you still out there? if so, i have some questions for you... here's another take on "gods and fighting men": > http://z-m-l.com/misc/14465-take6.pdf this .pdf just has the first page of each chapter, but it came outta my program, not a text-editor. as we can see, the p.g. linebreaks make this text practically unusable, so we'll have to do a rewrap, especially if you want to have the text _justfied_... (if not, i can just rearrange the unwieldy lines and leave the vast majority of p.g. linebreaks in place.) going on, is this text-size (10-point) good for you? (again, print out some pages so you know for sure.) how about the leading? it's 15-point leading, so that's generous for 10-point type, and you might feel it's _too_ big, but i thought i'd show it to you. on pages 12 and 101, you'll see _blue_ headers... those are lines that needed to be _shrunk_ a bit, so they would not spill over into the margin area. on page 12 it's 15.5-point instead of 16-point, and on page 101 it's 13-point instead of 14-point. (and that latter one still intrudes on the margins.) the program attempts to "copy-fit" all the headers to the same size, but i'm experimenting here with allowing slight variations in size on freakish lines. (rather than letting the freaks dictate that the other header-lines be smaller to accommodate the freaks.) so the question is, are these small variations bad? noticeable? too much so? do they bother you much? -bowerbird

2 1

Re: DP Archives/OLS (Was: Re: DP output ...)
by Bowerbird＠aol.com 26 Apr '10

26 Apr '10

and the takeaway is that p.g. can copy those scans and mount them any time that it chooses to do so. -bowerbird

2 1

Dim view of P3ers
by Karen Lofstrom 26 Apr '10

26 Apr '10

On Sat, Apr 17, 2010 at 12:15 PM, Jim Adcock <jimad(a)msn.com> wrote: > I have tried leaving project comments and the P1s and the P2s tend to read and follow the project comments whereas the P3s ignore them and undo the good work of the P1s and P2s. And earlier Jim wrote: > I have stated repeatedly that I found found extremely competent and dedicated volunteers at all levels of DP -- and the converse. Bizarre. In one post you're drawing back from blanket accusations and in the next, you repeat them. Jim, I don't understand WHY you feel impelled to keep throwing stones at DP. You don't like the way we do things, you've left ... it's all behind you, right? But no, you have to join the grouch group here at PG and repeatedly attack the organization that is providing the overwhelming majority of the texts submitted to PG. I suppose I ought to just killfile you, as I have Bowerbird. -- Karen Lofstrom

13 43

Re: Removing spurious break lines
by Bowerbird＠aol.com 25 Apr '10

25 Apr '10

joaquin- someone from p.g. should be along to answer your questions any minute now... we just went through a long bruising discussion about "spurious" linebreaks, which is likely why they're a bit reluctant to get on that horse and ride it again so soon... they took a _little_ step toward some progress by accepting a script that would remove those "spurious" linebreaks from a properly-prepared file. but the _big_ step that they still need to take is to make sure that all the files in the library are "properly-prepared". the don quixote text was one such file which is not "properly-prepared", as you discovered. (if it had been properly-prepared, the verses would've been indented, and thus you would have found it extremely easy to identify them.) so you have made them face an issue that they would rather not face, especially right now... -bowerbird

2 1

Removing spurious break lines
by Joaquin Cuenca Abela 25 Apr '10

25 Apr '10

Hi, some books, like "Don Quijote" (http://www.gutenberg.org/etext/2000) have spurious break lines all over the text. From what I understood PG generates all the derived formats from the HTML, if there is one, or from the raw text format otherwise. In this case there is an HTML version, but it also contains the spurious break lines. My guess is that the HTML was automatically generated from the text, and the text breaks the lines at ~79 - 80 characters. Are there guidelines on how to format the raw text to make it more amenable for automatic conversion to other formats by the PG tools? Is it ok to reformat this text removing the spurious break lines in the raw text? Was the HTML automatically generated? or do I have to fix also the HTML? How can I check the results in other formats before sending it to PG? Also, are the conversion tools open source? Cheers, -- Joaquin Cuenca Abela

1 1