January 2006 - gutvol-d - lists.pglaf.org

Please help archive the Katrina disaster
by Jon Noring 03 Apr '06

03 Apr '06

[I'm forwarding this for Brewster Kahle at the Internet Archive. If you have any Katrina-related links, please submit them as detailed below. As Brewster requests, please forward the following message to other relevant mailing lists and online forums. Jon Noring, TeBC Administrator] The Internet Archive, a non-profit digital library, needs help in finding URL's of sites and blogs that contain documents of this major disaster. Please email links to sites and pages that should be saved for future research to katrina(a)archive.org. We have worked to archive events such as 9/11 and the tsunami with the generous help of volunteers finding and sending in links. We then save these digital works for the long term and create research tools (for example: http://www.loc.gov/minerva/collect/sept11/ index.html and http://web.archive.org ). As a library, we provide free access to those wanting to learn from these events-- we can only hope that we learn some lessons from disasters such as these. Again, please send email to katrina(a)archive.org with lists of URL's you suggest should be archived relevant to the Katrina Disaster. We are also looking for a couple of volunteers that can help orchestrate the crawl. If you are interested, please send a note to katrina(a)archive.org with "volunteer" in the subject line. Please forward this message to any appropriate lists or people. Thank you. -Brewster Kahle Digital Librarian Internet Archive ------------------------ Yahoo! Groups Sponsor --------------------~--> Get Bzzzy! (real tools to help you find a job). Welcome to the Sweet Life. http://us.click.yahoo.com/A77XvD/vlQLAA/TtwFAA/89EolB/TM --------------------------------------------------------------------~-> Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/archivists-talk/ <*> To unsubscribe from this group, send an email to: archivists-talk-unsubscribe(a)yahoogroups.com <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/

2 2

36,000 and counting
by Bowerbird＠aol.com 25 Mar '06

25 Mar '06

distributed proofreaders just hit 36,000 registered users. congratulations! -bowerbird

8 9

Early ebook history info wanted; "Alice"; Brown Corpus; Vannevar Bush; Asimov
by Jon Noring 10 Mar '06

10 Mar '06

Everyone, In doing some research on ebook history, which naturally will prominently include Project Gutenberg because of the obvious impact PG and Michael Hart has had on etexts and ebooks, I'm trying to reconstruct a fact database which includes the who, what, when, where, why and how of the various seminal events. So, I've just created a specialized Yahoo Group to collect/archive the snippets of facts that come up: http://groups.yahoo.com/group/ebook-history/ You're welcome to join and post any information you know on ebook history. Especially wanted is the pre-1990 period: commercial, academic, and public (free). We should collect this information before it is lost to the mists of time. ***** Anyway, in doing research on what is found in the early Google Groups archive on PG, Bowerbird recommended that I dig through textfiles.com, which appears to have archived a large number of ASCII texts that existed on various BBS systems of the 1980's and early 1990's (its coverage/completeness is unknown, however.) So, focusing on the first "modern" classic book that PG issued, "Alice's Adventures in Wonderland" (text #11, officially released 01 Jan 1991), I dug through textfiles.com to see what they had. The oldest copy I found there was "alice11.txt", Millenium Fulcrum Edition 1.1, dated 1990 (by copyright claim.) See: http://www.textfiles.com/etext/FICTION/alice11.txt That was issued in the very early era when PG was still affiliated with Duncan Research. As an aside, it is interesting to note the huge differences in that early text's boilerplate compared with the present boilerplate. PG has evolved quite a bit in handling the legal aspects (particularly copyright) of its texts, which is to be fully expected. So this text is a nice historical reminder of how far PG has come in the last 16 years. I was hoping, though, to find a much older text version of "Alice". Bowerbird stated his belief that Michael Hart keypunched Alice a lot earlier than 1990/91, but so far I have not found that version, assuming it was distributed and ended up in some BBS or online archive. So, Michael, if you're reading this, did you keypunch "Alice" well before 1990, where did you distribute it, and does a copy exist somewhere? That would truly be a historical work, especially if it was "digiscribed" in the 1970's or early 80's (BBS systems began maturing in the mid- to late-1980's.) A question for others reading this: where else should I search for information on digitized books placed on BBS in the 1980's? Were there others besides Michael who "digiscribed" public domain books and texts in the 1980's and placed them online? (I plan to dig through more of the textfiles area to see what book texts are dated in the 1980's, if any, and who did them.) ***** Another interesting thing I discovered in my research -- and which some of you undoubtedly know about -- is the "Brown Corpus": http://en.wikipedia.org/wiki/Brown_Corpus In the late-1960's, the partial/full texts of a variety of 500 works published in 1961 were keypunched for computer use (a maximum of 2000 words for each work), totalling a little over 1,000,000 words. The purpose was solely for lexicostatistics and not for direct reading. For this purpose the Brown Corpus is quite famous (enough to rate its own wikipedia article.) Only a few years later, in 1971 Michael Hart keypunched into a computer "The Declaration of Independence" for the purpose of electronic "distribution" and direct reading by others, so Michael is, as far as is now known, the first person documented to experiment with electronic distribution of readable, published digital texts. (I plan to contact the Brown Corpus people, if any are still alive, to see if there were experiments at Brown, or elsewhere, on this in the 1960's.) But nevertheless, to see major portions of published texts and books being keypunched and processed by computers in the 1960's is truly remarkable. ***** Another really cool thing, I found a Usenet message from 1987 which, in turn, is a fairly comprehensive description of an Atlantic Monthly article written by Vannevar Bush in July 1945, entitled "As We May Think". It is beyond amazing the insights Vannevar Bush had relevant to ebooks, to elibraries (like PG's) and the role of individuals and volunteers. Again, some of you have probably read Vannevar Bush's article, but for those who haven't... Usenet summary: http://groups.google.com/group/comp.sys.mac.hypercard/msg/660f72a6e3b5f7a2?… And the actual article is reproduced here: http://www.theatlantic.com/doc/194507/bush ***** Finally, Mark Bernstein, the founder of Eastgate, which in 1987 issued several contemporary hypertext fiction ebooks on floppy disk and CD-ROM, mentioned to me Asimov's mid-50's book "Foundation" where ebooks are implicit. Has anyone read this book and can comment on Asimov's 1950's vision for ebooks? Thanks! Jon Noring

7 18

Re: [gutvol-d] Language free version of guiguts?
by Dave Fawthrop 18 Feb '06

18 Feb '06

On Wed, 18 Jan 2006 20:19:33 -0500, Jim Tinsley <jtinsley(a)pobox.com> wrote: |On Wed, 18 Jan 2006 11:44:46 +0000, Dave Fawthrop <hyphen(a)hyphenologist.co.uk> wrote: | |>On Mon, 16 Jan 2006 15:54:37 -0500, Jim Tinsley <jtinsley(a)pobox.com> |>wrote: |> |>|On Mon, 16 Jan 2006 17:13:12 +0000, Dave Fawthrop <hyphen(a)hyphenologist.co.uk> wrote: |>| |>| |>|>I am told by the whitewashers that it is *essential* that all text for PG |>|>pass guiguts. Because this assumes that the language scanned is American |>|>it gives 90% plus false positive errors, on my books, which is totally |>|>unsatisfactory for any piece of test software. |>|> |>|>Is there a language free version of Guiguts? |>| |>|I'm not quite sure which question you're asking, and about which |>|checking tool, but I think there is some confusion somewhere, of |>|emphasis if not of fact, and I'm continually surprised by people who |>|don't know the origins of really quite recent procedures I remember |>|vividly, and I've had several threads recently about this general |>|subject of checking, so please bear with me while I regurgitate |>|history. I hope you'll find a satisfactory answer in here somewhere. |> |>I could only find one tool which shows on my Win XP computer that is |>guiguts. This as far as I can ascertain has various subroutines which |>are very badly tied together, and in no way at all follow the Windoze |>interface. |> |>|Anybody can use any programs they like to make texts, and different |>|people do use different tools, according to their own needs or the |>|needs of the individual texts. Considering that we get French and |>|German and Esperanto and Chinese texts, not to mention older English, |>|there is no one-size-fits-all solution for language. |> |>To get things past whitewashers one apparently must use this, or things get |>rejected. Your assertion is therefore clearly theoretically correct, but |>in reality absolutely wrong | |Now, this I can flatly deny. There is no such thing as a standard without a test to show that the test has been passed. Schools teach to the exam, which some think wrong, but it happened to me and judging by the media Brouhaha still happens in the UK. I was an Engineer, and there a draughtsman who failed to put a test (tolerance) on anything in a drawing, was exposed to public ridicule. If such a drawing got onto the shop floor the production departments would deliberately fail to follow any reasonable tolerance. | I can think of half-a-dozen people |offhand, regular producers, who don't use gutcheck in any form. |They don't need to. Their quality standards are high enough that |it won't find any real errors. I do run it on their texts, as a |matter of form, but I know in advance what the result will be. |For all I know, there are others, equally good, but I don't know |that they don't use gutcheck because the subject never comes up. |Most of us, of course, are not that good. You have just admitted that gutcheck is the standard on PG. | |Bill Flis, who wrote the GutWrench package, uses his own checkers |exclusively, and I know equally well that I won't find any errors |that can sanely be caught by automation in his texts either. You can |find them, if you're interested, at http://www.pgdp.net/tools/GW.zip | |>|Once, there were no checking tools at all, except for spellcheckers |>|built into Word Perfect and Word, which is what most people used, and |>|I could tell you some stories about having to convert those! |>| |>|David Price and Martin Ward and I made checkers that we used for |>|ourselves. There may have been others, but those are the ones I'm |>|aware of. Everything else was Mark One Eyeball. |>| |>|I had done a lot of cleaning-up work on a lot of texts for various |>|people, and I would then send those on to Michael for posting. They |>|would commonly take hours of work each. In self-defense, I wrote a |>|checker I (later) renamed to gutcheck. When the WWs were formed in |>|2001, I brought gutcheck with me, and we all used it to find errors |>|quickly in incoming texts. |> |>But gutcheck gives 90% plus false positive errors, many hundreds on my |>texts in Yorkshire Dialect, mostly poems. It enforces the American |>language, and American punctuation conventions. It objects to most |>Yorkshire abbreviated words such as t' which occur dozens of times |>in the poems I work on. It also objects to non standard punctuation |>which occur in my texts as an example "? whereas American convention |>apparently is ?" . |> |>Writing as one who has designed, written and sold language software for |>some 20 years (see my web site). The *first* stage in the design of any |>software involving language is how other languages will be treated. This |>is usually done by putting all the features of one language, in a specific |>data structure(s) and/or subroutine(s) which can be used or not as |>required. |> |>All I asked for was a copy of gutcheck with the features specific to |>American removed which should be a very short editing and recompiling job. | |I'm not sure how you define "American", but ALL gutcheck features are |language-specific, one way or another. You really appreciate this when |checking Hebrew or Tagalog! Even the relatively familiar French, |German and Spanish have various punctuation features quite |incompatible with gutcheck's assumptions. I'm talking with various |LOTE producers about language-specific versions, but have not yet |decided to take any action. Then gutcheck should be modified to have versions for many languages. If you read the Subject of this thread, you will find: "Language free version of guiguts?" | |>Worse the only way to view output is on a screen. Copy does not work so it |>is impossible to copy the output to a text file and edit the repeated false |>positives out of the list. It is totally unacceptable to distribute a GUI |>program where the standard Copy and Paste functions do not work |> |>Worse still and absolutely ***unforgivable*** in any GUI program the |>settings places the settings file on ***THE DESKTOP***. Deleting it loses |>all settings. |> | |I can't comment on GuiGuts. As a command-line guy, I don't use it all |that much, except sometimes, when I find some specific feature |invaluable. If you want to comment, the appropriate place is in the |GuiGuts thread of the Tool Development forum at DP, which Steve reads |and answers questions and requests in. |http://www.pgdp.net/phpBB2/viewforum.php?f=13 I have asked the question here. I do not do forums. |>|Up till then, there was really no difference between DP and Other |>|texts, though because the people who mostly submitted from DP were |>|experienced, and because DP favored simple texts |> |>DP is by its nature not suitable for my texts, because the language is as |>different from American as say French. A non Tyke (yorkshireman) as has |>been shown in the past, has extreme difficulty understanding the text. | |Well, considering that they regularly do several languages, I doubt if |Yorkshire dialect would stand out much. Right now, in round 1, I find: |English, German (math, with LaTeX), Finnish, French with Scots, Middle |English, Middle French, Portuguese, English with Ancient Greek, |Spanish, Italian, Dutch, German, English with Breton, French, Tagalog, |Latin, and I just know there's some Esperanto around somewhere. I know |they've also done Irish (sean-litriú), because I had a hell of a time |finding all the correct characters for the UTF-8 version (and I'm |still not convinced about Tironian-et). Of course, if you want real |variety, you need to hit the European DP. | |>|And GuiGuts and gutcheck have accreted features ever since. If you |>|have GuiGuts, then you have gutcheck, since Steve bundles it with |>|GuiGuts -- and you also have a large number of other tools that may |>|or may not be useful for the particular text you're working on. |>| |>|There are many other checkers available as well, and I'd love to |>|ramble on about them, but this is too long already, and it doesn't |>|bear on your question. |> |>|This is how it comes -- by evolution, not by fiat -- |> |>Untrue! |>I am *forced* to use guiguts/gutcheck by the Whitewashers. | |I say again: not everyone does. Just eradicate all mistakes and nobody |will ever know what you used. | |>Gutcheck does not work on Windoze. | |It runs in a Win32 command prompt, but it doesn't have a GUI on any |platform. "You have to be joking MAN" | |>| that incoming |>|texts are checked with _several_ tools, according to what seems |>|appropriate for the text, but most commonly with gutcheck and/or |>|GuiGuts. |> |> |>Finally guiguts is as it stands unusable on my texts. No doubt I will find |>other equally drastic problems |> |>As all my work goes on my own web site, and gets copied from there onto |>many other sites, PG is just a nice add on and could be ditched if it were |>to take too much effort. |> |>The text which WW objected to so strongly has been on my site for a couple |>of years, and absolutely *nobody* has noticed the ?errors? People read it |>for the dialect, not the punctuation. I have however had several |>appreciative emails. | |Well, I'm very familiar with that condition, but that's a whole |'nother argument. A text does not have to be perfect to be valuable. |We have many older texts, especially, that have many errors. That |doesn't make them useless. I handle most of the errata reports for PG, |and nearly all of then express appreciation for the availability of |the text, along with their handful of reported errors. I may find |another hundred or so problems when I check the text out, but these |readers never noticed them. Two million downloads a month, with (I |estimate) about one million errors among 17,000 books, and we get |about one errata report per day. | |And there are many people who do want to make etexts but don't want |to live within the constraints of PG -- some don't want the |quality-checking, some complain that we don't quality-check enough, |some don't want to work in plain text, some don't want to go through |the clearance procedures, and so on. | |We have 40 to 60 submitted texts in the average week, and three WWs |active to take them at the moment. If everything in an incoming text |is perfect, one of us will spend about an hour on it. Plus a load of |time on other activities. We can't accommodate everyone on everything, |and there is no doubt that the quality gets higher as time goes on, |because of the processing that we do. This is what we have to do, |to keep the operation moving and the quality high. Not everyone is |going to be happy with the process. Some will choose not to send their |texts to PG. I'm sorry about that. | |>|(Now tell me that all you wanted was the -t switch. :-) |> |>I am not going back to the bad old Unix days, when each program had to be |>learned individually. Come back Bill Gates. All is forgiven. | |Well, I say again, if you don't want to use it, you don't have to; |not everyone does, and especially not everyone does for all texts. |It's essentially a collection of regexes, selected to give, on |average, the best results for the most common type of PG files. |Many |DPers who work on other types of texts just put together their own set |of regexes, and run them through GuiGuts or GutWrench or from a *nix |command line, whichever they prefer. I do not do windoze programming. You are essentially saying that a non programmer can work for PG. :-( Did you really mean that? You have agreed with me above that gutcheck is the standard which must be passed to get. I am just trying to find a version of that standard which will run on my machine, with the text As I understand it that answer to a perfectly reasonable request see Subject from PG was: ****************** ***GET STUFFED.*** ****************** I will look for a workaround. -- Dave Fawthrop <dave hyphenologist co uk> 17,000 free e-books at Project Gutenberg! http://www.gutenberg.net For Yorkshire Dialect go to www.hyphenologist.co.uk/songs/

6 9

re: [gutvol-d] blah blah blog
by Bowerbird＠aol.com 01 Feb '06

01 Feb '06

michael said: > I read the PG eBooks with all sorts of plain text viewers > and have no problems with inconsistencies, much less in > the various browsers that have a wider range of options. the inconsistencies are ones that a person "wouldn't notice", but which trip up any automated processing by a program... an obvious example would be that most section-headings (e.g., chapter headings) are preceded by four blank lines, but the occasional one might have three, or five, instead... nobody would claim that, in terms of a human reader, this inconsistency is meaningful -- it's not -- but when it comes to a program analyzing the file, it might make a difference... if there's only one level of header, then 3 or 4 or 5 blank lines might be equally good at signaling that there is a new section. but if a book has three different levels of headers, as some do, then you could use 5 blank lines to indicate the major sections, 4 to indicate regular sections, and 3 to indicate the subsections. if the number of blank lines isn't consistent, the program has to become much more sophisticated (and thus prone to failure) to try and determine the _actual_ level of each header. another example involves lines which should not be rewrapped, such as the lines in a table, or the lines in a letter's address-block. if these are consistently prefaced with one or more leading spaces, then a rewrap routine is easy to _write_ and easy to _comprehend_, and a programmer can spend time on more productive pursuits that add value and functionality, not ones that just resolve inconsistencies. lots of programmers have _started_ programs for the p.g. library. the vast majority of them have given up before long, in frustration. the inconsistencies in the formatting are the main source of difficulty. someday someone will set up a shadow version of the p.g. library where all the inconsistencies are resolved, and you will see then how much value is added by the ingenuity of programmers who are able to take consistent formatting of the e-texts for granted... -bowerbird

4 7

re: [gutvol-d] Ebook reading devices
by Bowerbird＠aol.com 01 Feb '06

01 Feb '06

ian said: > Fair enough - then the iRex device > might be a better bet. And less proprietry. might be. except i heard it will be $300-$500. with no deep pockets to subsidize and back it. way too much for a dumb terminal in this age... next? -bowerbird

9 11

Anyone recomend a good free wisywig html editor to PG standards
by Dave Fawthrop 01 Feb '06

01 Feb '06

Just turning my Hartley books into html. The generate html function of guiguts did not do a bad job of generating html, but the poetry needs a lot of tweaking. But it expects me to insert flags by hand :-( I gave up doing that in the bad old nroff days. <shudder> way back in OMG 1986. Anyone recommend a good free windoze html editor to PG standards? I normally use an old copy of M$ Front page for html. -- Dave Fawthrop <dave hyphenologist co uk> "Intelligent Design?" my knees say *not*. "Intelligent Design?" my back says *not*. More like "Incompetent design". Sig (C) Copyright Public Domain

2 1

!@!Re: [gutvol-d] blah blah blog
by Bowerbird＠aol.com 01 Feb '06

01 Feb '06

mikey said: > Please keep comments on issues > raised in bowerbird's "blah blah blog" > off this list. sorry to disturb your sleep, mikey. but you can go back to bed now... *** bastien said: > Can we see the code? no. > Can we use/test/improve it? i'll release an app sooner or later that you can use. as far as the routines themselves, creating them took a _persistent_ application of common-sense and a lot of elbow-grease. that's all. which means you can generate them yourself, if you really want. i'll also be willing to give you some pointers if you should happen to hit a wall at any point, once you have shown me you are willing to take on the job. or, if you prefer, you can take the shortcut, and buy my source code. the price is in the 6 figures. i can't afford to give away something that valuable. if you can, i suggest you buy it and give it away... but really, what i want you to take away is that you _can_ do this, and it's really not even all that _hard_. for instance, i've talked about using the number of preceding blank lines as an indicator of heading level. assuming that such consistency has been maintained, even a beginning programmer could write the code to ascertain the structure of a book. care to try it? write it in pseudo-code first, and then in a language, any language with which you are comfortable, be it perl or python or ruby or basic or whatever you like. it might be better to take it over to gutvol-p, so we don't put a coding session in all the gutvol-d boxes, but if you show me you're willing to do some work, i'll be more than happy to show you what will pay off. if you're not, then i'll refer you to the fables of aesop, specifically the one about the little red hen... *** michael said: > There's nothing wrong with > what such people are asking, > other than that they are asking > someone else to do it for them, > free of charge. i know that michael is talking about "such people" as a general class, and i'm fairly confident he doesn't lump me in that class. but other people here might. so let me be absolutely clear here for them... i'm not asking anyone to do anything "for" me. and i am more than willing to do the job myself. i said so, directly, on my blog, and i repeat it here. what i am doing is suggesting you make these changes for _yourselves_ and for _your_readers_, simply because consistency in the e-texts leads to greater functionality... my experience is that this greater functionality would benefit you with increased efficiency in the preparation of the e-texts, and benefit your readers in greater total usability of the e-texts. > Those of us who have been on this list, and others, > for very long these days have no trouble remember > any number of people who have had variety upon variety > of requests that Project Gutenberg should be run in > such a fashion as to meet with their demands. again, i make no "demands". just giving you tips. for your own good. > The response is always to invite the creation of some examples, > along a suggested pathway for future efforts, to be accompanied by > your request for others to assist you in making such future efforts. it is also the case that i _am_ putting examples online. i've already pointed to some, and more will come soon. > http://snowy.arsc.alaska.edu/bowerbird/alice01/alice01/alice01.zml > http://snowy.arsc.alaska.edu/bowerbird/alice01/alice01/alice01.html > http://snowy.arsc.alaska.edu/bowerbird/alice01/alice01/alice01.pdf > http://snowy.arsc.alaska.edu/bowerbird/alice01/alice01/alice01b.pdf other examples will be "my antonia", "books and culture", "free culture", "the universe or nothing", "the secret garden", and a handful of books by cory doctorow, for starters. as z.m.l. requires just a smattering of changes to the markup-free version of an e-text, hundreds of examples will follow before long, and thousands once my processes are refined... if i don't lose interest, i might have most of the library converted in a year. from these plain-text .zml "masters" will emerge automatic .html versions -- from which a plethora of other formats will be able to be generated -- and automatic creation of .pdf versions according to user specifications... other sweetnesses might follow too, like ipod versions and p.s.p. versions. and last but not least, the z.m.l. viewer-app will create a kick-ass powerful high-functionality electronic-book experience using these .zml "masters". enough so that you'll wonder why you ever thought you needed "markup". (there's more -- scansets and banana-cream -- but that's enough for now.) > Describe how these examples are better than previous examples have done, and will continue to do, as appropriate times arrive... -bowerbird

2 1

Re: [gutvol-d] Language free version of guiguts?
by Dave Fawthrop 01 Feb '06

01 Feb '06

On Tue, 31 Jan 2006 19:30:55 +0000, Cori <hiddengreen(a)gmail.com> wrote: |On 1/31/06, Dave Fawthrop <hyphen(a)hyphenologist.co.uk> wrote: |> |Indeed :) Catching a dozen real errors is a definite win! |> |> 12 in 500plus is a resounding failure. | |I think there might be a misunderstanding about the purpose of the |Gutcheck ... if a text checking tool was provided that never gave me |any false errors, I'd be convinced that it wasn't catching all it |should be. Spellcheckers, or the barrage of regex checks that DP has |developed, all flag up false positives on my books, but they couldn't |be made much more effective without personalising them to each and |every text -- which would take more time than just clicking through |the false alarms..? The point of all these checks is to (hopefully) |be over-sensitive to problems, rather than under-sensitive (thus |leaving errors.) | |Or have I missed something in turn..? Do you have text checking tools |that only ever signal real errors..? Can they be shared..? With my other hat on I write "intelligent" language software, Low 90% correct is very bad, above 99% correct is acceptable. For a voluntary organisation I would be accept 50% correct. -- Dave Fawthrop <dave hyphenologist co uk> "Intelligent Design?" my knees say *not*. "Intelligent Design?" my back says *not*. More like "Incompetent design". Sig (C) Copyright Public Domain

2 1

coold nothav e
by Bowerbird＠aol.com 31 Jan '06

31 Jan '06

mihceal siad: > So, what you are telling me hre, ist hat > while a human can muddle through ok, > it takes a computer to really maess things up. i coold nothav esaidit bettter myyyself. :+) -bowrebrid

1 0