I am excited about the potential of Project Gutenberg of Canada. But so far, I don't know much about it.

Andrew, thanks for starting the discussion! I'm officially, the "founder" of PG-Canada, and I know only a little more than you at this point. That's the idea behind this dicussion list. :-)

So I've put together a list of questions regarding various aspects of it, generally trying to see if there is anything we can learn by comparison with PG and PG-Au.

I think we can learn a lot by comparing to PG and PGAU... mostly, what _NOT_ to do.

I believe it would be worth-while to consider these issues and have a consistent plan ready rather than just wait until a situation comes up and then deal with it on an ad hoc basis.

I concur.

Some of these topics have shown themselves to cause strong differences of opinion on the gutvol-d mailing list. I would request that we avoid flame wars about them here.

Again, I concur. :o)

Filenames and Directory structure

As the post-10,000 changeover at PG shows, this is worth taking the time to consider carefully. I like the way the current PG system puts all the files relating to one eBook in the same subdirectory. PG-au is using one subdirectory for each release year, with each filename including the year, ebook number in that year, and version. However, that will not scale upwards well if ebook production increases. Given a choice between these two, I personally prefer the post-10,000 PG method.

Between the two options you stated, I prefer the PGAFS (Project Gutenberg Archive File System) over the PGAU idea much better. To be honest, I'm hoping to come up with an even better idea, as I don't really like PGAFS much either.

Will we have one basic, authoritative file format?

...

From postings I've seen on the gutvol-d mailing list, I would hazard a guess that some form of pgxml will likely be proposed by James. If so, that would seem to me to indicate a likelihood of using UTF-8 as a default encoding. Another seemingly logical choice would be to use mostly ISO-Latin-1, and other parts of ISO 8895 when appropriate.

Yes, we will have one authorative file format - XML of some sort, and probably UTF-16 encoding.

Do we restrict or encourage certain formats or encodings?

At PG-US the preference is still to go with having a plain 7-bit ASCII version available whenever possible. At PG-AU, Col has said posting just an ISO-Latin-1 version is fine. My own preference is to go for consistency in whatever route is taken.

ONLY the XML will be considered authorative. All other formats will be generated automatically from that XML master. This is _one_ concept that is not really open to discussion. PG's biggest problem is format of the ebooks, a problem we will _not_ have. Other formats will be accepted, but only if they can be converted to the XML format, preferably via automated means, but at worst, manually. Naturally, we want to keep such submissions to a minimum, so it will be key to provide tools to create the XML format (or at least a psuedo XML that we can easily import) for people to use.

How will we handle corrections?

At PG-US, when there are a small number of corrections, the file is changed with just a "last updated" line to show it was changed. For a larger number of corrections a new file is posted to superced the old one, and the old one is still kept in the archive. At PG-Au, a corrected eBook has "last updated" info added and the old file is deleted, with the new one taking its place.

ALL changed to a text will be logged, and the proper versioning noted in the file. We will make a backup of the master before updating it. Versions will be in _standard_ notation: major.minor.revision In this context, each part of the version number means: Major: the first "release" will always be major version 1 (1.0.0); Changing a file's primary structure (ie: fixing a missing chapter or page, etc) will result in a major version change. Minor: a minor version change will be done when 10 or more (approx) changes are done at once, so long as they are not changes as noted in the "major" definition. The minor changes would include adjusting metadata, fixing multiple typographic errors, etc. Revision: a revision change will be done when small corrections are made such as fixing a broken paragraph or adjusting the indenting on a stanza of poetry, etc. ALL corrections to a text will be done by automated systems (ie: web interface), so versioning will be handled by the system as well. Additionally, a 3 character language code will ALWAYS be appended to the version. IE: 1.0.0-eng would mean the text is the first release in English. This means that a text with the version of 1.0.0-spa would be a direct Spanish translation of 1.0.0-eng.

Selection criteria: exclusively Canadiana?

As I understand PG-Ca is hoping to have funding (from a government source?) does that mean we will have a mandate to pursue just Canadiana?

Initially, I think we should concentrate on Canadiana initially, because it will be easier to get funding with such content, but I have no intention of limiting it. That would be repository suicide, eh? Also, it would be nice to get a definitive collection of digital Canadiana online -- a good flagship project for a group called "PG of Canada". :-) Personally, I plan to work on locating and digitizing all of Nikola Tesla's work (ranging from 1890s to 1930s) at some point, even though he was Croatian / American. It's of personal interest to me, and said content is not generally available already, at least not a lot of it. Another "flagship" project that will, hopefully, put PG Canada "on the map".

Selection criteria: Overlap with PG and PG-au

PG-au is likely to have some similar texts as their copyright laws are similar to ours. Would we want to have a policy of trying to include everything found there--or keeping a distinct collection with no duplication--or just letting things happen as they may? Assuming a focus on Canadiana, would we want to make an effort to include all Canadiana in PG in this collection?

We will overlap work if ANY of the following condition are NOT met: 1) Full book scan images must be available. 2) _Legal_ copyright clearance assured and record. 3) Content is properly structured and marked up.

What metadata stored in database

I would love to have, right from the start, a clear expectation of what metadata we hope to record for each item, and as consistent as possible a way to record it.

We will start with a fully automated system, including cataloging. The catalog will exceed standard library catalog standards and will extend to include any metadata we need to store. At minimum, metadata will be: Title Author(s) Editor(s) Illustrator(s) Preparer(s) Publisher(s) Edition(s) Published Year(s) LOC call number (if available) LOC subject heading Dewey Decimal assignment (if available) ISBN (if available/applicable) Language(s) Description (of original print media, IE: hardcover, dimensions, etc) If item is from a periodical the following will be added to above list: Volume Publication Month(s) ISSN (if available/applicable)

Will we have an official copyright clearance procedure?

Just recently, I was thinking that one of the strong points of the Project Gutenberg collection is that it may be the largest collection of documented public domain material in existence. From what I understand, all copyright clearances are saved, so they can be referred back to if needed. In contrast, at PG-au, from what I've seen, you can just submit an eBook without getting formal clearance before-hand.

Yes, we will have an official clearance system as part of the automated system. Anyone know of a Canadian lawyer or ten who might be interested in pro bono work for PG Canada?

Mirroring? Backup? Long-term stability.

One reason I feel my time is well-spent as a PG volunteer is that the collection appears very permanent, and I feel that my contributions are not going to be lost over the years. I could not envision the PG archive disappearing as long as there is an internet. What plans would be ideal to encourage the same for PG-ca?

I already have tentative agreements with two well-known, very permanent places. I'll give out names when I have things more firmly worked out, but I will say they are already familiar with PG. I'm also working on a couple of Canadian mirrors. Any other locations that might be interested would be good - a project like this can never have too many backups/mirrors. Regards, James Linden Founder, Project Gutenberg of Canada jlinden@projectgutenberg.ca http://www.projectgutenberg.ca/

The issue of filenaming and directory structure is very important, and worth being discussed in the context of a coordination between PG-like projects. I include in this definition not only projects that use the Project Gutenberg trademark, but also projects that share the goal of PG, and might be partners in a world-wide "Gutenberg Federation". I would like to see the directory structure of several projects to be compatible; this means two important features: - there are no name incompatibilities: a file from one project will not overwrite a file from another. This requires coordination, but a numerical scheme like PG-main or PG-AU forces incompatibilities - the directory structures put together related books of different origin. I would like to see togeter the different volumes of the same work even if they are released in different times; different books of the same author, and originals and translations of the same book, independently of the project that prepared it. A big virtual library might be obtained copying the directory structure of all the projects under the same root; this might be utopian, but an approximation might already be significant. I think that the structure used by Progetto Manuzio is a good one, but should be improved to be adopted. They have a subdivision per letter of the author name, then author name, then a directory per work, and inside a directory per file type. So for example they have a/alighieri/ with index.htm giving informations on the author and a list of available works; inside, a/alighieri/convivio_edizione_busnelli_vandelli/ and a/alighieri/convivio_edizione_vasoli_de_robertis/ and moreover a/alighieri/convivio_edizione_vasoli_de_robertis/rtf/ a/alighieri/convivio_edizione_vasoli_de_robertis/txt/ What I like less, is that both directories contain convivio.zip, hence there is an overwriting possibility when uploading the files. Of course, with 8.3 naming conventions this is unavoidable, but I believe that this now can be overcome. I would like better something like a/alighieri/convivio/edizione_vasoli_de_robertis/txt/convivio_vasoli-txt.zip and a/alighieri/convivio/en/translator_name/txt/convivio_en_tn.zip Of course, such a structure requires a coordination between projects; to avoid having a centralized authority, i would suggest that a default competence be identified, that can be decided by origin (dutch authors of competence of PG-NL) or by first inclusion. The centralized catalogue is however needed: before including a book, an authorization (issued by automatic criteria, possibly by a robot on the basis of the metadata) should be obtained. Of course, such an approach requires advances in the constitution of a "Gutenberg Federation" to handle at least the format of the metadata and the unified catalogue; compliance to the recommendation should be voluntary, but I believe that if the recommendations are useful they will be followed. Carlo