jon said:
>   Well, we've both had our say,
>   so let the few others who
>   are following this topic
>   decide for themselves.

how about instead we have people
forget about what you said so they
can pay attention to what i said?             :+)

the system you want to lay on people
is so convoluted from the standpoint of
their _actual_ workflow right now that
they reject what you say out of hand,
and end up unwilling to discuss the topic.

and that makes it all that more difficult
to get them to accept reasonable ideas.

as david put it:
>   That is true. That will continue to be true
>   what ever standard is agreed upon, since
>   people aren't going to suddenly change
>   their scanning patterns based on what the
>   final product, a long ways down the line, looks like.

now, the reality of the situation is that
a change in their "scanning patterns"
might well _improve_ their work flow,
make them more efficient, and so on.

but since you don't understand that flow
-- and since they know that you don't --
you can't really hope to talk at that level.

i'm sorry to have to keep on harping that
you don't know what you're talking about.

i'm aware that some people hear that and
think i'm some mean person dumping on you.

the truth of the matter is i don't like saying it.
i don't like that you make it necessary to do so.

but _somebody_ has to level with you, jon.

take another example, from your latest post:
>   But your system requires that each image,
>   when it is first saved, needs a human being
>   to eyeball the page, determine the publisher
>   supplied page number (if any; may be implied),
>   and then manually save the page
>   using the publisher number.

so i tell you again that you need to look at the
auto-naming feature in your scanning program.
it saves the page automatically, then increments
the page-counter in anticipation of the next file.

(if you don't have such a capability, you really
need to get yourself a more modern program.
but since it was common even a decade ago,
i assume that your program has this feature.)

auto-naming is what most all d.p. scanners use.
they don't enter a name for each individual file,
as that would waste energy and slow them down.
they get into a dream routine where their body is
flying in a ritual of page-turning and positioning.
some of 'em can scan hundreds of pages an hour.

if you had scanned in more than a few books, and
scanned 'em in the manner that d.p. scanners do,
you would know this, and you wouldn't say things
so inconsistent with the reality of their processes.

***

here's what _really_ needs to be done
to take page-image scan-sets public...

1.  scanning should be done such that
the auto-naming of the scanned images
fits the outcome we want them to have.
however, if the scanner for some reason
cannot or did not do this, it's no problem,
as our tools can rename the files _easily_.
i developed such a renamer using an e-text
that had unnumbered plates and blank pages
(some of which were scanned and some not),
plus a whole raft of missing pages (part of an
english/hawaiian facing-translations section),
and so on.  and the renaming tool worked fine.

2.  quality-control needs to be done
_during_ the process of scanning, so
errors (like missing pages or duplicates)
are caught when they can be fixed easily.
i believe d.p. scanners have learned this.
you do a visual inspection of each page,
to make sure it is acceptable quality, and
then a check where you step through all
of the images checking their page-numbers
to confirm you got every single one of 'em.
again, regular d.p. scanners do this already,
because they've learned that it's worthwhile.
finding bad pages after fact = pain in the ass.

3.  all of the scans need to be _cleaned_.
they need to be deskewed and regularized,
both in terms of size, and placement of text.
since this job can be done fairly automatically,
without much need for human intervention, and
since it will help the o.c.r. accuracy, it should be
done _before_ the o.c.r.  the original files can be
discarded; the foibles of scanning are unimportant.
i have not lobbied on behalf of such cleanup before
because i figured d.p. would learn it sooner or later.
(now if only the large scanning projects would too!)
but if you take scans public, they must be cleaned.
but again, if this wasn't done originally, it can easily
be done later, since it is generally rather automatic.

4.  the o.c.r. process should output individual files,
so as to retain the linkage between text and its scan.
at this point, any weirdnesses in the page-numbering
can be addressed, and blank scans inserted if needed.
i won't get into the reasons why now, but these files
should have all styling (margins, italics, etc.) saved!
folks, do _not_ strip your o.c.r. files to plain-ascii!

5.  the posted e-text-files -- ascii, html, all of 'em --
need to have the page-breaks indicated in some way,
so that end-users and their tools can easily correlate
the text with the exact page-scan from which it came.
so pagebreaks must be retained every step of the way!
stop discarding this useful information from the files!
if you need input on how to represent them, ask me!

6.  all the way through the cleaning-up of the text,
a person should be able to deal with one page of text
_or_ with the entire book, whichever is convenient
for the particular clean-up task at hand at the time.
this is, of course, a function of the _tools_ they use,
rather than the scan-set per se, but i mention it now
since the implication is you'll need to be receptive to
feedback and requests you get from your toolmakers;
an example of that is my input on naming of the files.

7.  every page-scan should be accessible separately.
bundles -- such as djvu and .zip -- can also be made,
but the world must be able to grab each page-scan
_individually_ when one is all that they want to see;
it's unreasonable to expect 'em to download 'em all.

8.  a system of "continuous proofreading" should be
initiated to make the best use of these public scans,
as it will leverage the immense power of user eyeballs
to march the e-texts toward an error-free perfection.

9.  any all-image-format should be easily ported to
the current and future platforms that can utilize it,
like the sony playstation portable and the nokia770.
it would be great if you offered such porting online.
i'll be offering it as a feature in my viewer-program,
with the text-file as input, but it would be nice if you
could offer it using the original page-scans as well...

10.  whenever possible, the converted e-text should
accompany the scans, to overcome the weaknesses
of an image-format (e.g., the lack of searchability,
the inability to copy-and-paste text, and so forth...).

-bowerbird