At DP, most of us use ABBYY Finereader (versions ranging from 5.0 to 7.0) to do the OCR work. It does not currently have an option to save the result as XML, though I suppose they might well implement something like that eventually. Also, for proofreading purposes, it is much easier to work with material that does not yet have all the XML tags, etc. We have always planned to have formating rounds, and, in fact, they are currently in active development and I hope they will be in place by the end of the year. I expect that the nature of the formatting rounds will change with time. My hope, however, is that in most cases, even the people working in the formating rounds will not have to see all the verbosity that goes with XML and that can be represeted unambiguously in a more reader friendly way. Paragraph markers are an example that springs easily to mind. They can be added automatically later and it would be a serious waste of volunteer time to have to type those in in place of the blank line that we currently use. That case is trivially obvious, but most others may not be. One of the things that we will have to work out through experience is exactly what kinds of markup happen at which stage of the process. When one really gets into the details, there are a staggering number of them. A formating/markup issue that we've struggled with recently is teaching people how to know when to include a period inside italics and when not to. And then getting them to do it correctly. Yes, I know this doesn't matter for html, and won't matter for XML, but it does matter for plain text versions that use underscores to mark italics. I mention it only as an example of the tiddly, little, significant details that must be worked out in a day-to-day production environment. But back to my point. I expect that we will end up with a combination of automatic tools and manual intervention. Exactly what will happen where in the process remains to be determined. We'll try something, which inevitably won't be the right thing, and we will proceed with incremental changes until we end up with a system that works reasonably well. Ideally the output of that system will be an XML master file that can then be used to generate versions in whatever form any individual user requests. And answering another request from another message: I have LOTS of scanned title pages. Where would you like them? JulietS DP Site Admin ----- Original Message ----- From: "Joshua Hutchinson" <joshua@hutchinson.net> To: "Project Gutenberg Volunteer Discussion" <gutvol-d@lists.pglaf.org> Sent: Friday, October 22, 2004 8:54 AM Subject: Re: [gutvol-d] Re: barriers to XML posting ----- Original Message ----- From: Steve Thomas <stephen.thomas@adelaide.edu.au>

A question (possibly better put over on the DP list):

Is it possible to OCR a scan directly to XML? Or is the output from OCR always going to be text?

That is a very DP related question, but I'll answer here as best as I understand the future plans (and let others correct me where needed). The plan at DP is to move from the current 2 round proofing model to a (probably) 4 round proofing/markup model. The content provider will take the scans and OCR them normally. That part doesn't change. Then, there are 2 rounds of proofing that concentrate on typos, spelling, etc. Very similar to the 2 rounds we have now. Then, there are 2 MORE rounds of markup. Here is where all the markup like poetry, italics/bold, footnotes, chapter headings, thoughtbreaks, etc, etc are done. Then, when the final result gets out of 4 rounds, it is nicely marked up (in theory) XML. The post-processor does his/her normal magic, combining all the pages, running validators on it, etc. As far as the OCR process, we currently run some pre-processors on text to fix common scannos, etc. I'd be surprised if those pre-processors didn't improve/change as the XML world emerges at DP. Josh _______________________________________________ gutvol-d mailing list gutvol-d@lists.pglaf.org http://lists.pglaf.org/listinfo.cgi/gutvol-d