- gutvol-d - lists.pglaf.org

re: [gutvol-d] entire Gutenberg ebook collection on a flash drive
by Bowerbird＠aol.com 13 Sep '05

13 Sep '05

aaron said: > Not really. A lot more could be on there > if we had added ZIP files instead of uncompressed text, > but that wasn't really the point. and the point -- as it was -- was quite phenomenal. although the website says differently, if i remember, i believe i counted some 3,000 e-texts on that c.d. now, since plain-text compresses greatly when zipped, i would love to see a c.d. full of zipped text-files as well. i'd especially love it if they were all in z.m.l. format... ;+) > You're probably right and > I imagine that that will be the case eventually. cool! -bowerbird

3 2

Re: [gutvol-d] What the Hack?! and other conferences next week
by Tony Baechler 13 Sep '05

13 Sep '05

Hello. I am blind so the video does me no good at all obviously. Yes, I would really like to hear just the mp3 audio of a lot of the presentations. There seems to be another done by Greg Newby on the same page. Also, why not upload the presentations to the Internet Archive since the files have a CC license? I think they generate mp3 from the mp4 anyway. Just the audio would be very appreciated. Why not post just the audio to PG? You could actually post both but the huge video file might deter people with limited disk space or slow connections. I think it would be nice to see more audio of various sorts being posted anyway. At 06:19 PM 9/11/2005 -0500, you wrote: >>The answer is: >> http://rehash.whatthehack.org >> >>The files are very big. > >If anyone on dialup or a slow connection would like to hear the >presentation, please let me know and I can put it up as a small mp3 or >windows media file if the video is important to you. > >Aaron Cannon

3 2

Re: [gutvol-d] SF author thinking of donating rights to PG (gutvol-d Digest, Vol 14, Issue 9)
by Lee Passey 13 Sep '05

13 Sep '05

Robert Shimmin <shimmin(a)uiuc.edu> wrote: > Andrew Sly wrote: > >> My recollection from the last time this was mentioned >> is that PG has had a few authors causually mention ideas >> like this over the years, but no one has actually gone >> ahead with it yet. I gather it would require some formal >> written statement, legally relinquishing copyright at >> a certain date, or under certain conditions (perhaps >> such as being out of print for a certain number of years.) >> I could understand most authors being reluctant to >> do that. > > They don't have to do that. They merely have to license PG with > perpetual, non-exclusive, world-wide rights to distribute the book on > a royalty-free basis. See the FAQ, question V.71. > > http://www.gutenberg.org/faq/V-71 > > -- RS Following the recommendations made in the FAQ may satisfy Project Gutenberg's requirements, but in may not satisfy the author's. If you look at the language in the FAQ you will see that a this type of a license grants to Project Gutenberg the right to make and distribute copies of a book (presumably; the proposed language is a little unclear on this point). It does not grant to anyone who obtains a copy from Project Gutenberg the right to _re_-distribute the book. Now, pretty much everyone understands that Project Gutenberg is not really an archive. The provenance of the e-texts it distributes is uncertain, editorial changes are undocumented, and as a result of the focus on over-simplified text "many things that should not have been forgotten have been lost." The true role of Project Gutenberg has rather been to reduce the barriers to access for a large number of classic texts. The notion that texts downloaded from Project Gutenberg cannot be shared seems to be antithetical to goal of making e-texts accessible. On a somewhat unrelated note, I have heard anecdotes of people who have downloaded PG e-text and then actually gone out and _purchased_ commercial electronic versions of the same text in the hope (sometimes vain) that the commercial version would provide a better reading experience. If I were an author hoping to preserve my work well into the future, I would be sure that 1. the grant of privilege would include rights of redistribution and 2. the work could not be modified in any way that would remove provenance, metadata, or semanticaly significant markup (such as logical emphasis). The Creative Commons non-commercial, no-derivative-works licence actual statisfies both of these goals well, and I note that Charles Stross, the author about whom this thread started, has actually released his latest novel in electronic form using just such a license (http://www.accelerando.org/book/) The real question is how Project Gutenberg choses to deal with works which are not public domain and to which a specific license grant has not been made to Project Gutenberg, but which _are_ freely distributable via a Creative Commons license. As I read the CC license there is nothing there which would prevent PG from simply taking such works and adding them to its catalog (although it may be unlawful to add the inconsequential Project Gutenberg boilerplate). On the other hand, there was some discussion here last month about whether or not PG would be willing to accept into its catalog scholarly annotations of existing classical text already part of its collection. The answer seemed to be that it would not, apparently based on the fear that Project Gutenberg might become a forum for any splinter group seeking to promote a special agenda (e.g., a commentary on _The Voyage of the Beagle_ by someone promoting "intelligent design" as a scientific theory). I believe there is some wisdom in this position. But it does raise the question about what to do when someone with an entirely new work wants to distribute it via Project Gutenberg. Perhaps the answer should be that Project Gutenberg will only accept new works which 1. have an existing publication history through a non-vanity press, and 2. can be freely distributed, and re-distributed, at least non-commercially, and that Project Gutenberg will only catalog and distribute such books at the request of someone _other than_ the author. It may be that Project Gutenberg will want to limit itself to digitizing and popularizing only Public Domain texts that are no longer current, and leave to other organizations, such as the Internet Archive, the role of archiving current texts.

2 1

re: [gutvol-d] entire Gutenberg ebook collection on a flash drive
by Bowerbird＠aol.com 13 Sep '05

13 Sep '05

brandon said: > http://www.gutenberg.org/cdproject/ that d.v.d. has less than 10,000 e-texts on it. it was up-to-date in december of 2003, when i helped michael pass out copies of it in berkeley, but it's outdated now, to the tune of 7,000 files. considering its capacity, the c.d. is phenomenal. *** aaron said: > http://snowy.arsc.alaska.edu/pgiso/ well, that u.r.l. seems to be timing out right now. perhaps it's because i put in "1-17000" as a range of .txt files for it to fetch, which might've overwhelmed it. at any rate, since i would think this might be a relatively common request, it might be nice to provide a constantly updated version of it, without pestering that iso-creator... -bowerbird

3 2

entire Gutenberg ebook collection on a flash drive
by Aaron Cannon 12 Sep '05

12 Sep '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 It may be closer than we think. http://www.businessweek.com/ap/financialnews/D8CIRUE02.htm?campaign_id=apn_… In short, Samsung has developed a new flash memory chip that holds a lot more data. From the article: "By putting together 16 of the new NAND chips it's possible to create a 32-gigabyte product that can store 8,000 MP3 audio files or 20 DVD movies." Perhaps in the future, we'll be sending out flash memory cards instead of DVDs. :) Time will tell. Sincerely Aaron Cannon - -- E-mail: cannona(a)fireantproductions.com Skype: cannona MSN Messenger: cannona(a)hotmail.com (Do not send E-mail to the hotmail address.) -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (MingW32) - GPGrelay v0.959 Comment: Key available from all major key servers. iD8DBQFDJcfiI7J99hVZuJcRAsJgAJ9WzQcCnlGSrfv49LnG3BdYxLvuswCg+EjJ mjuzuDYw7LBIJPUsEdBfYTY= =NWfT -----END PGP SIGNATURE-----

2 4

re: [gutvol-d] entire Gutenberg ebook collection on a flash drive
by Bowerbird＠aol.com 12 Sep '05

12 Sep '05

has someone put all the .txt files on a single d.v.d. yet? if not, why not? -bowerbird

3 2

What the Hack?! and other conferences next week
by Branko Collin 12 Sep '05

12 Sep '05

A dr. Newby from Alaska will be selling elixir, er, holding talks about search engines and culture preservation programs at the What The Hack hacker camp in the Netherlands next week. I know where I am going to be! http://www.whatthehack.org/ and http://program.whatthehack.org/ In the meantime a Juliet S. is going to be involved in an online conference on ebooks on Thursday, which everybody can follow at http://www.planetlibrary.info/lgleindex.htm after downloading a "small, safe plug-in". (Can I see the source code?) -- branko collin collin(a)xs4all.nl

4 5

Re: Re: [gutvol-d] ANNOUNCEMENT: XML has hit the PG archives! (gutvol-d Digest, Vol 13, Issue 19)
by Lee Passey 11 Sep '05

11 Sep '05

Joshua Hutchinson <joshua(a)hutchinson.net> wrote: > Lee Passey wrote: [snip] >> >> As Mr. Noring is always quick to point out, XML files can be viewed >> natively in both Firefox and IE6 when accompanied by appropriate >> style sheets, so I attempted to open this file directly in both of >> these browsers. >> > While this is true, our tei files are specifically meant as a master > document and NOT as a viewing document. They will NOT parse in any > browser "out of the box". As you've seen, you can jury-rig things to > the point where it is usuable, but that is not our intention. We > provide the HTML files directly for people that want to browse the > file in IE or Firefox. I understand that creating a file format which could be viewed without further processing was not your intention, but now that we have some evidence that suggests that it is a real possiblity is there any reason _not_ to pursue that possiblity, especially if it only requires adding three lines to the source (and making sure that all the dtd's are accessible)? [snip] >> I have no solution to this problem, except to suggest that named >> entities simply be avoided in favor of numeric entities, at least in >> the short term (I do note that the etext 16523-x.xml does not contain >> any named entities). >> > I personally prefer numeric entities, as well, but for the more common > ones, the conversion process will support named entities in the .tei > file. Most of them appear as unicode in the HTML, so it typically > isn't an issue in the final product. You are correct; so long as you are relying on conversion to HTML (or some other file format) before the file is used, there should be no problem (so long as the conversion utility can get to the correct .ent files). Use of named entities is only a problem if you are attempting to display the TEI-XML directly. [snip] >> It appears that the file is latin-1 encoded, despite the fact that >> the DTD claims that it is utf-8 encoded. This caused Firefox some >> grief as it tried to utf-8-decode some latin-1 accented vowels. >> > I may be wrong here (Marcello is my unicode guru), but I thought UTF-8 > was a superset of Latin1? Anyway, I know in this particular file > there are quite a few UTF-8 encoded characters (and a couple more that > should be that we found yesterday backchannel). UTF-8 and Latin-1 (aka ISO-8859-1) are both encoding methods. They share the same codepoints (the value of an acute 'e' is 233 in both encodings) but they use different encoding methods. Neither is a superset or subset of the other. Values from 0 to 127 are the same in both encodings, but values from 128 to 255 are encoded in a single byte in Latin-1 whereas those same values are encoded in two bytes in UTF-8. Values above 255 are represented in two or more bytes in UTF-8 (up to 6) where those same values cannot be represented at all in Latin-1. From an efficiency standpoint (which is not always the best way to look at things) if you have an English text which contains some few characters having values above 127, and which has as many above 255 as below, or if you have a text which contains a large number of characters with values above 255, UTF-8 is the probably the most efficient encoding (size-wise). If you have a western european text with a large number of characters above 127, but very few above 255 (French is a good example) Latin-1, with values above 255 expressed as entities (numberic or named) is probably the most efficient encoding. If you have a text where most of the characters have values above 1920 UTF-16 is probably the most efficient encoding (now we're really straying from the point). In any case, it doesn't matter which encoding is used, so long as it is not misrepresented in the <?xml ...> declaration. >> If you're interested, I'll start putting together a generic CSS file >> for TEI. > > > We aren't too interested in CSS directly for the TEI file (the css > file sitting beside the TEI file right now is a mistake ... that > should be changed later today). However, once I have a few more > documents posted and people seem fairly satisfied with the results, I > want to get alternate CSS files submitted by other people for the HTML > documents. Well, I might do it anyway for my own edification and enjoyment (and because I think you _will_ be interested at some point in the future ;-).) Some months ago I put together a couple of tables showing how HTML could be mapped to TEI-lite, and vice-versa. The goal was to create a mapping that could be used for round-tripping via XSLT; that is, a TEI-lite document could be used to create an HTML document which could then be transformed back into TEI without loss of markup. I will probably start from those tables in creating a tei.css file. They may also be useful to you in creating XSLT scripts (aka XSL style sheets). If you're interested they can be found at www.passkeysoft.com/~lee/xhtml2tei.html and www.passkeysoft.com/~lee/tei2xhtml.html. > > Also, if any industrious programmers out there know TEI conversions > and would like to tackle the job of preparing a conversion process for > other end formats (such as Palm files, Plucker, MS Reader, etc) please > let me and/or Marcello know. The conversion must run on Linux (our > server OS) and be open source (for future compatibility). You probably don't need anything more than someone with basic shell scripting capabilities, as all the software to do this exists currently. When you say Palm files, I am assuming you mean PalmDOC files, which are nothing more than text files converted into the Palm Database format. This conversion can be performed by the command line program "Makedoc". Source code is available at http://linuxmafia.com/pub/palmos/other-os/makedoc9.tar.gz. The shell script would be: PGTEI -> (via XSLT) -> .txt -> (via makedoc9) -> .pdb Plucker is a progam which encapsulates a bundle of HTML files into a single file which can be rendered on the PalmOS. The script for a plucker transformation should be very similar to the PalmDOC transformation (I'm certain Mr. Desrosiers could help you with the precise syntax): PGTEI -> (via XSLT) -> HTML -> (via plucker distiller) -> .pdb To my knowledge there are no known lit compilers that run on Linux (thus making them ineligble by your requirements). This is not really a big deal because most MSReader users who are familiar with Project Gutenberg are comfortable making .lit files from HTML themselves, so if you can serve good HTML they will be happy. What I would really like to see is an XSL script that could do a PGTEI -> RTF transformation. It probably wouldn't be very useful, but it would sure be interesting. Now on a separate note: As part of my CSS experimentation, I set the display setting for the <tei-header> element to "none", because while I think the data is important, I'm not particularly interested in seeing it when I'm reading. When I did this, I thought I lost the title of the book because it only appears in the <tei-header> element. I discovered later the title was repeated in the <front> element, identified as a <head>er. As I read the TEI spec, (and I am by no means well-versed) I believe that there should also exist a <titlePage> element which should be part of the <front>, and which should contain all the information traditionally found on the title page of a book. The main title should be marked as <titlePart type="main">, subtitles should be marked as <titlePart type="sub">, and the byline should be marked as <byline>. This would be in addition to the information included in the <tei-header> element, which may be formated differently (e.g. the author's name may be presented last name first for automated catalog processing). I also had some question about the difference between the <titlePart> element and the <title> element. Looking at the spec it seems that the <title> element is not to be used to indicate the title of the work, as would appear on a title page, but the title of _another_ work referenced in the main work (these are the titles we were taught to underline back in the days of single font typewriters). For example, if _The Kitáb-i-Aqdas_ made reference to the _Baghad-Vita_, it would be marked as <title>The Baghad-Vita</title>, and should probably be rendered with an italicised font. I also note that you encoded the glossary at the end of the work with <p> tags (naughty, naughty). Based on what I saw in the TEI docs I would have encoded it as follows: <div type="glossary"> <head>Glossary</head> <list type="gloss"> <label>'Abdu'l-Bahá</label> <gloss>The "Servant of Bahá", Abbás Effendi (1844-1921), the eldest son and appointed Successor of Bahá'u'lláh, and the Centre of His Covenant.</gloss> <label>Abjad<label> <gloss>The ancient Arabic system of allocating a numerical value to letters of the alphabet, so that numbers may be represented by letters and vice versa. Thus every word has both a literal meaning and a numerical value.</gloss> etc. </list></div> I hope you find this useful.

7 7

What's happening at copy.pglaf.org?
by Wallace J.McLean 10 Sep '05

10 Sep '05

I've been trying for over two weeks to raise a signal. What gives? Why isn't anyone returning my important email? I have a situation that needs resolution, and until it's resolved, I can't pick up any of the projects I had to leave off this spring. Where is everybody?

4 3

ok then, let's go to work (part 3)
by Bowerbird＠aol.com 09 Sep '05

09 Sep '05

and here's today's post... again, i'm talking about the scan-set that exists at: > http://www.openreader.org/myantonia those scans were subjected to o.c.r. by finereader v7, with the resultant text-file located here: > http://snowy.arsc.alaska.edu/bowerbird/myantonia1.txt one of the things you should notice about this text-file is that the original linebreaks were retained by the o.c.r. linebreak retention is very important when it comes to comparing the text-file against the page-scans because it makes that comparison many times easier to accomplish. one of the hassles with those original linebreaks, however, is with the words that were hyphenated at the end of a line. you can't just rejoin all words that have end-line hyphens, because some of 'em might be words that are hyphenated even when they're in the middle of a line. you _could_ use a dictionary, to determine whether the word is a compound, a better way is to use the actual text of this particular book, in case the author had some idiosyncrasies. so, in order to identify such words, we need a computer routine that finds all of the words that were hyphenated in the middle of a line. it's not hard to write such a routine, and the results from it -- for the "my antonia" text-file -- are appended to this post... once you have these "truly-hyphenated" words, you can then write a routine to correctly rejoin end-of-line hyphenated words. of course, you would only do this _after_ you are done proofing. (it is _here_ that you would do a compare against a dictionary, to make sure that you were doing the rejoins "correctly" and to "flag" any exceptions, so they could be checked more closely, to ensure that they were indeed idiosyncratic to the book/author.) you probably should also retain a copy with the original linebreaks, just in case it will come in handy for someone else down the line. another option would be to code the original linebreaks so that a viewer-app could keep them if the end-reader wants them, or delete them (i.e., convert 'em to spaces) if that's what's desired. after all, isn't the best general attitude to _retain_information_ that can't be reproduced, and not throw it away, if at all possible? there has been a growing realization that page-break info should be saved. why isn't the same realization extended to linebreaks? some people will want to reproduce the original p-book as closely as possible, and linebreaks would be an important aspect of that. distributed proofreaders keeps linebreaks up until the last minute -- because they understand that linebreaks help proofing immensely -- but then they throw them away, seemingly oblivious to the fact that they also could be extremely useful to someone else later on... don't throw this valuable information away! someone might want it! *** as your bonus for today, i've uploaded the .rtf file that finereader generates automatically out of the o.c.r. process. as i remarked wednesday, _styling_ information -- which is included in the .rtf -- can be very valuable in helping ascertain the book's structure... the .rtf can be found at: > http://snowy.arsc.alaska.edu/bowerbird/myantonia1.rtf for those who prefer a .zip, it's at: > http://snowy.arsc.alaska.edu/bowerbird/myantonia1-rtf.zip as illustration of today's lesson, however, i have uploaded an .rtf that does _not_ retain the original linebreaks. if you compare it to the .txt version -- to which it is identical -- you'll see just how difficult the comparison can be when that linebreak information is discarded, and all unnecessarily so, because it could easily be kept. *** so, was anyone able to "figure out" the heuristics that i was using in wednesday's routines to ferret out the headers in "my antonia"? speak up, folks, don't be shy, or you won't get any source-code... *** see you on monday, with part 4... -bowerbird p.s. here is the list of "truly-hyphenated" words in "my antonia": (this is the list before some unwanted one-of-a-kind exceptions have been culled fro it, such as the first two listed. a double-dash is an end-of-line hyphen that was part of an otherwise hyphenated word.) An'-ton-ee-ah An-tonia Ar-r-r-mond Good-bye Good-evening Good-morning High-School Indian-like Jake-y May-basket Preserving-time Snow-White Sunday-School Te-e-ach To-day Wash-day a-goin' a-going a-quiver absent-minded alligator-skin ashy-gray awkward-looking ball-gloves bar-room bar-tender barbed-wire bath-water bay-window be-frogged bed-ticking bee-bush black--and-white-check bluff-top boarding-house boarding-houses boot-tops bottle-tomatoes bow-legged box-couch box-elder brand-new bread-board breast-pocket bright-flowered bringing-up broad-backed broad-minded bronze-green buffalo-peas buggy-riding bull-snake bull-snakes bunk-bed butter-maker cabinet-maker cabinet-work cake-cut--ters candle-moulds car-window car-windows cared-for carpenter-shop case-hardened cattle-pond cave-house cheek-bones cheerful-looking chicken-bone chicken-house cigar-stand clap-trap clasp-knife close-clipped close-cropped close-growing cloud-hung coat-tail coffee-cake coffee-pot cologne-scented cone-flower cone-flowers cook-stove copper-red corn-knife court-plaster cow-pumpkin crab-apples cross-roads cuff-buttons curly-headed cutting-tables day-coaches deep-seated deep-seeing deep-set developing-room dining-room dog-town dog-towns door-bell door-crack down-feathers down-hearted draft-horse dragon-slayer draw-bank draw-bottom draw-head draw-side dressing-room dress--ing-room drug-store dry-goods dumb-bell dun-shaded dusty-smelling earth-owls easy-blowing easy-chair elder-hunting ever-falling every-day faint-hearted fair-skinned fancy-work far-away far-seeing farm-boy farm-work fashion-plates faun-like feather-bed feather-beds feed-bag fellow-countryman fellow-countrymen fifty-five finger-bowls finger-exercises finger-tips fire-box first-cabin flat-chested flat-topped forget-me-nots full-grown fur-worker gaming-tables garment-makers' ghost-moon gold-green gold-headed gold-washed golden-rod good-bye good-byes good-looking good-nature good-naturedly good-night goods-box grain-cars grain-sack grammar-school grape-arbor green-topped griddle-cakes ground-cherries ground-cherry grown-up guinea-hens half-buried half-ear half-fares half-grown half-mile half-past half-section half-story half-swooning half-window half-windows hand-painted hand-sheller hard-and-fast hard-worked hay-cave hay-fights head-first heart-broke heavy-odored hide--and-seek high-collared high-heeled hitch-bar hitching-bar hollow-chested home-coming hoo-hoo horse-collar horse-pond horse-sense house-cleaning husking-gloves ice-cold ice-cream ink-smeared iron-gray ironing-boards jack-rabbits kawn-tree kawntree-man ketch-on lady-teacher lard-pail large-minded level-headed liberal-minded lifting-up light-hearted lightning-flashes lining-silk living-room lodging-house lone--some-like long-winded low-branching lunch-basket merry-go-around merry-go-round merry-making mid-day mid-ocean milk--ing-pails mixing-bowl money-lender moth-like mountain-ash mournful-looking music-master natural-born natural-like nev-er never-ending newly-created non-existent nose-bleed nut-cake oil-can oil-cloth oilcloth-covered oil--cloth-covered old- old-fashioned old-maidishness one-night open-grazing orange-colored out-of-door over-considerate over-delicate owl-feather pagoda-like pale-blue pale-gold paper-hanger paper-rack passers-by pea-vines peck-measure picture-books pig-yards pillow-cases plough-handles plum-patch pocket-mirror pocket-money porch-posts post-office prairie-dog prayer-book prayer-meeting pump-handle quick-changing quick-footed rabbit-skin race-course raw-boned reading-table ready-made reap--ing-hook red-lined ripple-marks rocking-chair roll-call round-collared round-eyed round-shouldered run-over rye-field saddle-bags sand-blast school-teachers scroll-work scrub-oaks sea-coast section-lines self-asser--tiveness self-possessed self-possession self-sacrifice shaving-mug sheep-fold sheet-iron shirt-sleeves shooting-coat side-road silver-rimmed sing-song sitting-room sky-line slack-wire sleigh-bells sleigh-ride slim-waisted small-town snake-cane snake-skins snow-banks snow-blocked snow-covered snow-field snow-men snow-on-the-mountain snow-white soft-hearted stage-driver steel-gray stop-watch stopping-place store-boxes storm-win--dows straw-colored strong-minded sturdy-looking sugar-cakes sun-down sun-up sun-warmed sunflower-bordered supper-table sway-backed te-e-ach tea-kettle ten-dollar thick-set thoughtful-look--ing thrashing-machines title-page to-day to-morrow to-night train-crew tree-tops trom--bone-player trotting-horse trot--ting-buggy trout-fishing trumpet-blowing turning-lathe twenty-dollar twenty-four twenty-six un-wishful under-side up-bringing violin-teacher violin-teachf waffle-irons wagon-box wagon-seat wagon-tracks wagon-trail wall-paper war-times warm-blooded warm-hearted wash-basin wash-boiler watch-case watch-charm water-pipes water-spaniel water-tap water-tin weak-minded wedding-ring week-day well-behaved well-dressed well-fed well-grown well-made well-planted well-prepared well-set-up well-to-do west-bound wheel-rim wheel-ruts whinny-laugh white-handed white--and-gold wild-eyed wild-looking win-ter windmill-frame window-blinds window-shade wine-colored wine-stains wo-man wolf-cry women-folk work-basket work-horse work-horses work-room work-table work-team work-teams working-man working-slippers yard-measure yoke-mate

3 2