jon said:

I hope you get an error rate that is one per ten pages

i'll do my best. :+)

even if you do, I still believe a DP-like process is necessary to catch errors that OCR can't handle

human readers will _always_ be necessary. (and easy enough to find. if no one wants to read a book, there's little call to digitize it.) thus a system of "continuous proofreading" will be quite good enough if we can make the computer-guided processing accurate enough.

and for someone to properly assemble the pages, structure the document, etc., after the OCRing/proofing is complete.

that's part of what i include in "post-o.c.r. processing".

I don't quite put the same level of faith in OCR as you seem to.

except that once you see the evidence i lay out, you will realize "faith" has nothing to do with it. as i've been saying all along, for professionally typeset books, the structure is _in_ the presentation. so o.c.r. gives you all the information you need, if you know how to look for it, and do so diligently.

Btw, I believe as you do that an error reporting system is a good idea so readers may submit errors they find in the texts they use -- sort of an ongoing post-DP proofing process.

post-d.p.? i see it _replacing_ d.p. for most books. and good thing, too. once the coming avalanche of scanned-books engulfs us, it'll be the only way most books have a chance to surface. that will take the pressure off distributed proofreaders, and they'll be able to focus on the books that _need_ them.

Obviously, it is necessary to make available the page scans of the source document to aid in this process. How can an error be properly verified and corrected when the source work is not available?

i've always said i think that page-scans should be publicly available. particularly if your mission is _transcribing_an_existing_edition_. (although, to remind people again, copyism is _not_ the mission that michael hart chose to embed within his project gutenberg.) but even in the case of project gutenberg's "amalgamated" e-texts, i believe that a page-image graphic-version should be made available. this would allow people to view it on a dvd-player, just as an example.

Scanning took quite a while (much more than four hours)

that doesn't surprise me. nonetheless, i'll limit myself to 4 hours. that's quite enough time to devote to it. and to prove the point too.

I deemed it important for processing purposes that the name of the image contain semantic information of what it represents, and that naming be consistent for file sorting purposes.

as one improvement, i would suggest _not_ using "001.png", etc. instead, preface each one with a string that will make it _unique_, such as "ma2005feb001.png". it's easy to tell that to the o.c.r. app -- you just type it in one time -- and it's an unmistakable stamp. and of course, if you're going to do hundreds or thousands of books, you want to cook up a naming convention that conveys information. on big multimedia projects, it is not at all uncommon to have one _full-time_ employee dedicated _solely_ to maintaining filenames. because if things go wrong, it can waste a whole lot of man-hours. oh yeah, one more suggestion. your front-matter filenames were prefaced with an "r". my typical recommendation is that they be prefaced with an "f", and that the regular pages be named with a "p", so the front-matter files will sort _on_top_of_ the regular pages. i want to be able to depend on the operating-system filename sort to give me pages in the exact order they appear in the book itself. so i use a "q" on back-matter files, so they will drop to the bottom. for illustration plates, i use a name that sorts _them_ correctly; for instance, if an illustration page is between pages 168 and 169, name it "p168a.png". (and don't forget the blank verso side either!, which you will name "p168b.png".)

The publisher simply chose to start at page 3. Was this common?

it's not uncommon. oftentimes there is a "title-page", consisting of nothing more than the name of the book, which is considered "page 1", with its blank verso being "page 2", so chapter 1 starts on "page 3". sometimes chapter 1 starts on page 7. or page 11. publishers are weird.

Maybe there was an intent to insert a page there, which after typesetting it was decided not to.)

sometimes that happens too, yep. an "unnecessary" page gets dropped when the typesetter realizes they didn't plan the signatures correctly. or when the preface runs two pages longer than was originally intended. or any number of other snafus spring up. shit happens.

It was my intent to reproduce each page for direct reading purposes -- that is, if somebody wanted to read the book as it was printed, then they could.

yeah, and sometimes people want to do exactly that. which is why the page-images should be made available. for many illustrated books, the text alone is not enough. you want to be able to see the pages as they were printed. my viewer-program will work with either, text or images. it'll even work in "hybrid" mode, so you can display the text in one of the 2-up pages, and the page-image on the other side. (and of course that is the mode which is used for proofreading.) that's why things like _blank_pages_ are so important to include. because if you toss them out, you screw up the left/right sequence. a convention of paper-books is that odd pages always go on the right. screw that up and you make yourself look silly. anyway, that's all for now. -bowerbird