- gutvol-d - lists.pglaf.org

Re: !@!!@!!@!Re: Re: so what is so important about pagination?
by cmiske＠ashzfall.com 24 Feb '10

24 Feb '10

I agree as well that the original digitalization should be as lossless as possible. The various outputs of this original can then be as lossy as anyone cares to make them. If the original is lossy, then the information is lost 'forever.' Lossy originals reduce the choices of readers and forces a convention upon them of what is and is not important. The readers of the books should decide what they want/need. Carel -------- Original Message -------- Subject: [gutvol-d] Re: !@!!@!!@!Re: Re: so what is so important about pagination? From: "Keith J. Schultz" <schultzk(a)uni-trier.de> Date: Wed, February 24, 2010 3:23 am To: Project Gutenberg Volunteer Discussion <gutvol-d(a)lists.pglaf.org> Hi All, Been following this thread loosely. But the fact remains that the problem is the encoding of the digital text!!! NOT, the output. I have said it time and time again, the the encoding has to encode the information of the original scan or p-text. Thats is a rendering of such an encoding will produce a fairly close copy of the original. (I use rendering here as a way of producing the text; either in text or display) For the output you simply use those parts of the encoding for your output that you need. The page number problem can be handled because out of convention they are placed in the header or footer of a page. So if you do not want the original pagenumbers then just skip them. If you want the original page breaks use them and output it where you want it. I agree with BB, but we have different ways about doing it.

1 0

Re: so what is so important about pagination?
by christine 24 Feb '10

24 Feb '10

I have been following this conversation, and would like it to lead to a slight change in the PG pagination rule. As post-processor at DP-Int, I often have the feeling of uploading unfinished works because I have to remove the pagination from the text files. I don't mind in novels, but for books with index, it is another matter. Some time ago, I received a mail asking me why I format my page numbers as invisible in the html files. One of my reason is that I can not see the point of showing them, since most of everything is linked (TOC, LOI, Index, etc...) When I started post-processing, I included the page numbers in the text files, and often had to remove them to have the project posted. When I asked about the reason, I received the answers "it disturbs", "there is no need for it, the reader can do a search", etc. Now, it does not seem very logical to show them in a linked file (html) and leave them out in the text file. Doing a search is not a very good solution either, in most of the case, it is like searching for a Smith in a british telephon book, more finds than you care for. As example: I am currently working on "The Life of Napoleon", one of the index entries says "N. returns to Paris". Well, searching for "Napoleon" or "Paris" in that project would brings too many finds, as for "returns" this would not bring you to the proper page either, because, there, you find "is driven back". So, would it be possible to just relax that rule and let the volunteer decides *if* the pagination in the text file of a project is needed or not. Christine (aka Lostpaces)

5 6

Re: !@!!@!!@!Re: Re: so what is so important about pagination?
by Bowerbird＠aol.com 24 Feb '10

24 Feb '10

michael said: > If you have a point to make, you missed. i think lots of people got the point. but you evidently missed it. > Take better aim and try again, or give it up. my aim is true. i've done enough. i have no need to "give up", _or_ to continue. the future is the entity that will deal with this. doesn't matter how much friction we generate. the future will deal. and this is how the future will do that dealing: 1. it will need to tell which texts are accurate and which are not, because it will find itself awash in e-books without provenance... 2. it will do this determination by comparing digital text to scans. (it will test the validity of the scans by comparing them to p-books.) 3. digital text which retained pagination/linebreaks will lend itself better to this comparison process, so it will generally tend to "win", and the "best practice" will come to require this aspect, and strictly. 4. digital text which is difficult to get to (e.g., zipped up in .epub, obscured by some opaque format, unduly burdened by d.r.m., etc.) and text which has discarded its pagination/linebreaks will lose out. i have no interest in engaging in "debate" on these 4 points, because it's perfectly clear to me that they're all _inevitable_. the question here is whether p.g. wants its e-texts in group #3 or #4. -bowerbird

3 2

Translation project
by cmiske＠ashzfall.com 24 Feb '10

24 Feb '10

I've just recently started studying a language that is foreign to me. Although I am a long way off from being able to translate anything beyond an early reader, I was wondering if there is a translation project out there that is similar to DP? I looked about the PG site, but could not find a link to anything of that nature. Thanks; Carel

3 2

the d.p. opinion on "prerelease" of e-texts
by Bowerbird＠aol.com 24 Feb '10

24 Feb '10

just in case you were wondering, dkretz _did_ copy michael hart's saturday post over to d.p., giving michael's position in favor of prerelease of texts while they're still "in progress" at d.p. that was on saturday evening, and since then there have been over 100 posts in response: > http://www.pgdp.net/phpBB2/viewtopic.php?t=42940 there was an earlier discussion on the issue too: > http://www.pgdp.net/phpBB2/viewtopic.php?t=42695 the earlier discussion had over 100 posts as well, and had a poll, which got a high number of votes. the poll question? > Making preview texts available on a dedicated site, > with a clear label that they're work in progress. > Good idea/bad idea? the poll results? > 45% -- I'd rather you didn't > 17% -- I couldn't care less > 36% -- Great idea! some people are quite agitated at the thought of the idea. they drag out the old rot about how "d.p. means _quality_". -bowerbird

1 0

Re: !@!!@!!@!Re: Re: so what is so important about pagination?
by Bowerbird＠aol.com 24 Feb '10

24 Feb '10

michael said: > You missed your real point, that it was already time for you to quit. i seem to have found it now. you're wrong, michael, and the future will deliver the _real_ point. -bowerbird

2 1

so what is so important about pagination?
by christine 23 Feb '10

23 Feb '10

I am not talking about adding big chunk of blank lines, but of either including the page number like that [p.101] in the text, I do not like this solution since it disturb when doing a search; or adding the page number at the start of the 1. paragraph starting on a page, e.g.: change of page, blah blah end of paragraph. [p.101]Text of the new paragraph. With a TN explaining that the page numbers are not accuratly placed at the start of the page, but at the start of the 1. paragraph starting on that page. The aim is to make the use of the files as easy and comfortable to the reader as possible. Christine >From: "Michael S. Hart" <hart(a)pglaf.org> > > >I'm reading in a 34 line window. > >Let's say there are only 2 blank lines before and after page numbers, >that's 5 lines, or about 14% of my screen, when they go by. > >Anything over 10% is going to be distracting, according to perception >psychologists. . .I find my eye is attracted there. . . .

3 2

Re: ok, let's take a look at gardner's book, just for the exercise
by Bowerbird＠aol.com 23 Feb '10

23 Feb '10

i've analyzed the 159 diffs on gardner's books, and placed them into the following categories: 60 comma/period diffs 33 quotemark diffs 29 stealth-scanno diffs 12 various-punctuation diffs 12 formatting diffs 09 real-spelling-question diffs 04 should-have-been-preprocessed diffs ---------- 159 total note that i haven't resolved the diffs yet, i've just classified them, so we can understand their nature. *** more than anything else, these finds reflect the fact that when archive.org does o.c.r. on a google scanset, the results are not as impressive as they might be... the o.c.r. attained by archive.org on its own scansets is _much_ better. although the comma/period problem is a common one, the high number of confusions in this text is strange... ditto with the quotemark diffs. (but, for the record, i could've detected many of those with my quote-check, but i didn't bother to run it, since i had gardner's text.) i don't know if the google scans need to be despeckled, or if the contrast is bad on them, or what the problem is, and since google probably wouldn't improve them for us anyway, even if we knew the reason and asked 'em to fix it, it's probably not worth figuring out. or maybe it would be, if there was something that we could do to fix it ourselves. lucky for me, i don't have much investment in any book, and if i did have one, i would be willing to scan it myself, so i'm not desperate enough to deal with a google scanset; i can afford to wait until archive.org scans the book instead. one final note is that the number of stealth scannos is high, at least relative to the other archive.org books i've checked. again, this might be due to the low quality of google's scans, but it might also be that archive.org has activated a kind of "intelligence" that tries to _guess_ a word it cannot recognize, and the guessing routine is bad, at least from our standpoint. we should certainly hope and pray that's _not_ the case, since stealth scannos are hidden icebergs to a comparison method. you can find my breakdown of the 159 diffs into categories at: > http://z-m-l.com/go/gardn/gardn-diffs-classes.html -bowerbird

1 0

Re: ok, let's take a look at gardner's book, just for the exercise
by Bowerbird＠aol.com 23 Feb '10

23 Feb '10

i did some more work on gardner's book, "the advocate". the (presumably incorrect) lines from the o.c.r. are in red, while the lines from gardner's proofed copy are in blue... (at least i hope i didn't screw up the colors like last time.) here's a sample url, using page-by-page mode: > http://z-m-l.com/go/gardn/gardnp123.html here's a single web-page containing the whole book: > http://z-m-l.com/go/gardn/gardn-hybrid6.html and here's a web-page with _just_ the diffs on it: > http://z-m-l.com/go/gardn/gardn-159diffs.html we've got 159 diffs. that's about all the preprocessing that can be done on this file. 159 diffs on ~4000 lines is _great._ saves a ton of time on word-by-word proofing. in this last round of preprocessing, i did a spellcheck... i should explain that i do not use a regular spellchecker. i copy all the text into a program (which i coded) that spits out a list of all the words _not_ in its dictionary. if you're interested, here's the dictionary that it uses: > http://z-m-l.com/go/regulardictionary.txt then i go through the list and cull out the words that -- for whatever reason -- look to me to be correct... (most typically, they're names and compound words, neither of which class happens to be in my dictionary; doing this culling saves me a lot of time that would be spent checking and then by-passing these valid words; it's basically a quick way to create a custom dictionary.) this leaves me with a list of words that i want to check, which i then feed into my banana-cream app, so it will show me all the occurrences of those words in the text, with the relevant page-scan displayed at the same time, so that i can check the text against its scan. it's handy. by the way, i can also feed banana-cream a list of lines, which it will then search for and display automatically... this is what i'll do to resolve those diffs displayed here: > http://z-m-l.com/go/gardn/gardn-159diffs.html -bowerbird

1 0

so what is so important about pagination?
by christine 23 Feb '10

23 Feb '10

The format text file is unsatisfying anyway, but as long as we insist on having it, we should make the best of it. Even if the page numbers are only given every second pages because of long paragraphs, it is IMO still better than not giving any at all. And putting it at the start of a paragraph would not make it useless in case of indices. It would reduce the search of the user to a couple of pages instead of a lot of them. Christine > > > ---------- Forwarded message ---------- > From: Marcello Perathoner <marcello(a)perathoner.de> > To: Project Gutenberg Volunteer Discussion <gutvol-d(a)lists.pglaf.org> > Date: Tue, 23 Feb 2010 19:22:27 +0100 > Subject: [gutvol-d] Re: so what is so important about pagination? > christine wrote: > >> I am not talking about adding big chunk of blank lines, but of either >> including the page number like that [p.101] in the text, I do not like this >> solution since it disturb when doing a search; >> >> or adding the page number at the start of the 1. paragraph starting on a >> page, e.g.: >> >> change of page, >> blah blah end of paragraph. >> >> [p.101]Text of the new paragraph. >> >> With a TN explaining that the page numbers are not accuratly placed at the >> start of the page, but at the start of the 1. paragraph starting on that >> page. The aim is to make the use of the files as easy and comfortable to the >> reader as possible. >> > > Probably would not work with Ulysses or Kant or many other texts that have > multi-page paragraphs. > > > > So we are left with: > > 1. put the page number where it belongs, even if its in the middle of a > word, or > > 2. put the page number where it irritates less, even if its up to one whole > paragraph off. > > Of course 2. would make it useless for indices, the use case you advocated > in the first place. > > Both cases are unsatisfying, IMO. > > > The only place where page numbers belong is in invisible markup that can be > made visible on demand (CSS edit or so). > > > -- > Marcello Perathoner > webmaster(a)gutenberg.org > >

1 0