- gutvol-d - lists.pglaf.org

Re: [gutvol-d] XML version of some books of PG (and other formats)
by Joshua Hutchinson 04 Dec '04

04 Dec '04

I'm curious to see if your script can handle tables. That is our current biggest bugaboo when it comes to transforming to PG TXT format. Josh ----- Original Message ----- From: "Sebastien Blondeel" <blondeel(a)clipper.ens.fr> To: gutvol-d(a)lists.pglaf.org Subject: [gutvol-d] XML version of some books of PG (and other formats) Date: Fri, 3 Dec 2004 05:28:39 +0100 > > Hello, > > I hacked some scripts doing the following: > > RTF -> XML > > RTF: from Word, using a (very) simple stylesheet: just > paragraphs, 3 title levels, footnotes, and italics > Meta-information is in the properties of the document. > My script can extract images too, if wanted. > > XML: using a personal and simple DTD (embedded), probably easy to port > to any more complete DTD, such as TEI > > This is the hard part, and I am never quite sure it will not break in > case the Word file is weird. > > > From that, I then did other (proof-of-concept) scripts to produce: > > XML -> PG TXT > XML -> (LaTeX) -> PDF, DVI, PS (with hyperlinks) > XML -> valid HTML 4.01 (probably useless) > XML -> XHTML 1.0 Strict with some CSS (embedded) > > The programming is very defensive, so when all transforms finish I am > confident enough the stuff is right. > > You can find examples of those formats at > http://www.eleves.ens.fr/home/blondeel/ebooksgratuits/ > (most of the books there don't have the meta-info properly set up, > so don't worry too much about that). > > My scripts also clean up small typography mistakes (they are specialized > in French rules but can of course be taught any thing). They will be > used to help give PG nicer formats from the ebooksgratuits team (until > now their Word macros could only produce PG TXT, which is not very sexy > to read for the end user). > > Regards, > _______________________________________________ > gutvol-d mailing list > gutvol-d(a)lists.pglaf.org > http://lists.pglaf.org/listinfo.cgi/gutvol-d

3 2

Re: [gutvol-d] XML version of some books of PG (and other formats)
by Joshua Hutchinson 04 Dec '04

04 Dec '04

Anyone have a link to w3m in a windows executable (command line is fine, I just don't have access to a way to compile the source where I'm at right now)? This definitely looks interesting. Josh ----- Original Message ----- From: "Sebastien Blondeel" <blondeel(a)clipper.ens.fr> To: "Project Gutenberg Volunteer Discussion" <gutvol-d(a)lists.pglaf.org> Subject: Re: [gutvol-d] XML version of some books of PG (and other formats) Date: Fri, 3 Dec 2004 21:12:28 +0100 > > On Fri, Dec 03, 2004 at 12:07:31PM -0500, Joshua Hutchinson wrote: > > A web browser will generate line breaks within cells so that the table > > will end up looking very similar to the above. I haven't tried w3m > > ... will it handle the above scenario? I've tried lynx dumping to a > > You should have. Yes it does: > > -=-=-= > $ cat /tmp/toto.html > <table border="0"> > <tr> > <td>Data label that is extremely long and is broken up accordingly over > multiple lines.</td> > <td>Now we have a column of data that is also very long and broken up > over multiple lines.</td> > <td>Now we have a column of data that is also very long and broken up > over multiple lines.</td> > </tr> > </table> > > $ w3m -cols 72 -dump /tmp/toto.html > Data label that is Now we have a column of Now we have a column of > extremely long and is data that is also very data that is also very > broken up accordingly long and broken up over long and broken up over > over multiple lines. multiple lines. multiple lines. > > $ w3m -cols 48 -dump /tmp/toto.html > Data label that Now we have a Now we have a > is extremely column of data column of data > long and is that is also that is also > broken up very long and very long and > accordingly broken up over broken up over > over multiple multiple lines. multiple lines. > lines. > -=-=-= > > (Note: for the 72 columns version I don't know why there is an extra > space between columns 1 and 2. Probably a bug of w3m: it was already > there in the base-ball example. This is easy to detect and fix I guess: > use ``border=1'' and clean out the frames: > > $ w3m -cols 76 -dump /tmp/toto.html > +-------------------------------------------------------------------------+ > |Data label that is |Now we have a column of |Now we have a column of | > |extremely long and is |data that is also very |data that is also very | > |broken up accordingly |long and broken up over |long and broken up over | > |over multiple lines. |multiple lines. |multiple lines. | > +-------------------------------------------------------------------------+ > ^^^ ^^ > > You can detect those useless empty columns and remove them (or decide to have 2 > or 3 blanks between columns). Doing this without frames is more dangerous, and > the columns more difficult to detect: > > $ w3m -cols 72 -dump /tmp/toto.html > Data la el that is Now we have a column of Now we have a column of > extreme y long and is data that is also very data that is also very > brokenx p accordingly long and broken up over long and broken up over > over mu tiple lines. multiple lines. multiple lines. > ^^^ > this is not a column break. > > > text file and IE/Mozilla dumping to a text, and they all fail > > miserably. > > w3m is better than lynx for tables (and many other things: it is able to > display images in console mode and inside xterms!). links is good too. > > As for the example given in a later message: > > -=-=-= > $ w3m -cols 72 -dump teioutput.html | head > 150. Ptolemy publishes his geography. > 230. The Peutinger Table pictures the Roman roads. > 400-14. Fa-hien travels through and describes Afghanistan and India. > 499. Hoei-Sin said to have visited the kingdom of Fu-sang, 20,000 > furlongs east of China (identified by some with California). > 518-21. Hoei-Sing and Sung-Yun visit and describe the Pamirs and the > Punjab. > 540. Cosmas Indicopleustes visits India, and combats the > sphericity of the globe. > 629-46. Hiouen-Tshang travels through Turkestan, Afghanistan, India, > > $ perl ~/work/PGDP/ebooksgratuits/Fmt.pl 72 teioutput.html | head > <table border="0"> > <tr><td>150.</td><td>Ptolemy publishes his geography.</td></tr> > <tr><td>230.</td><td>The Peutinger Table pictures the Roman > roads.</td></tr> > <tr><td>400-14.</td><td>Fa-hien travels through and describes > Afghanistan and India.</td></tr> > <tr><td>499.</td><td>Hoei-Sin said to have visited the kingdom of > Fu-sang, 20,000 furlongs east of China (identified by some with > California).</td></tr> > <tr><td>518-21.</td><td>Hoei-Sing and Sung-Yun visit and describe the > -=-=-= > > Note: sometimes the columns in w3m are, weirdly, unbalanced. I don't have an > example right here but I guess you can help with percentage-width attributes in > the columns (if that is possible at all in TEI). I am using an old version of > w3m, too (Debian stable). > > With links: > > -=-=-= > $ links -dump teioutput.html | head > 150. Ptolemy publishes his geography. > 230. The Peutinger Table pictures the Roman roads. > 400-14. Fa-hien travels through and describes Afghanistan and India. > 499. Hoei-Sin said to have visited the kingdom of Fu-sang, 20,000 > furlongs east of China (identified by some with California). > 518-21. Hoei-Sing and Sung-Yun visit and describe the Pamirs and the > Punjab. > 540. Cosmas Indicopleustes visits India, and combats the sphericity > of the globe. > 629-46. Hiouen-Tshang travels through Turkestan, Afghanistan, India, > > $ links -dump toto.html > +------------------------------------------------------------------------+ > | Data label that is | Now we have a column of | Now we have a column | > | extremely long and is | data that is also very | of data that is also | > | broken up accordingly | long and broken up over | very long and broken | > | over multiple lines. | multiple lines. | up over multiple | > | | | lines. | > +------------------------------------------------------------------------+ > > $ links -dump toto.html # without border > Data label that is Now we have a column of Now we have a column of > extremely long and is data that is also very data that is also very > broken up accordingly over long and broken up over long and broken up over > multiple lines. multiple lines. multiple lines. > -=-=-= > _______________________________________________ > gutvol-d mailing list > gutvol-d(a)lists.pglaf.org > http://lists.pglaf.org/listinfo.cgi/gutvol-d

2 1

Re: [gutvol-d] possible fix for overwide tables in PGTEI text (was:XML version of some books...)
by Joshua Hutchinson 04 Dec '04

04 Dec '04

----- Original Message ----- From: "Marcello Perathoner" <marcello(a)perathoner.de> > > The new forthcoming version 0.3 of the PGTEI converter already does this. > (with a little help from the markup: you have to manually specify the width of > the column.) > I saw that in the preliminary document you sent me, Marcello. I haven't had a chance to really dig into it yet. (US Thanksgiving holiday has put me behind) Dumb, off-the-top-of-my-head question: Would it be possible for the converter to assume that if no manual width is specified, that it should just divide the table width by the number of columns and apply that value automatically to each column? That way, if a quick-and-dirty table will work, no further markup is needed. But if special formatting is needed (perhaps for a really big, complex table?), the etext preparer can take the time to do so. Josh

2 1

Update 2 on the eFictionWise
by Gutenberg9443＠aol.com 04 Dec '04

04 Dec '04

Hi all-- My eFictionWise 1150 arrived this morning, and I have been fiddling with it ever since. Ten seconds after I got my Rocket out of its box, I was in love with it. If my first experience with a dedicated ebook reader had been an eFictionWise 1150, I would never have even tried to use a dedicated ebook reader again. It is nicely packaged. It feels good in the hand. The problem of having to point the down arrow to go up and the up arrow to go down has been remedied. BUT-- (1) There seem to be only two type sizes available, larger and smaller. Although the Rocket has the same built-in capacity, it also has a way to download other font sizes from the RocketLibrary. (2) With my vision problems, I need black on white. What I'm getting is grey-beige on white-grey-beige. The controls do not allow me to create black on white. I will be able to read the books in bed, because they are backlighted, but I'm not going to be able to do it without glasses, as I can the Rocket. (3) I still haven't figured out how to do most things. The biggest problem is that three of the four on-screen icons don't mean the same as they do on the Rocket. All kinds of pulldown menus are located in very illogical places. (4) I think I have figured out how to highlight passages, but not how to insert bookmarks. (5) The write-and-draw feature does not work. I have not checked out the keyboard yet, but it's pretty useless if I can't get back to the notes I have made. That's why I need bookmarks. (6) The method of downloading books is a total, utter, and complete nightmare. When I am downloading a Rocket or Microwave Reader from FictionWise.com, I pay for it and then I download it onto my screen and then, if it's a Rocket, I goes directly to the RocketLibrarian, which then asks me if I want to put it into the Rocket eBook reader. If I say yes, the job is done in a few seconds. If I say no, the file sits right there in my RocketLibrarian on my computer until I need it. If I want to import a personal file, from PG or any of several other sources, I make sure that I have saved it in .txt or .htm. Then I go into the Rocket Librarian, press a button and import the file, name the file, and then do exactly what I do for a book that is already .rb. Now, read the following eBookWise version of this task: I had put two ebooks in my eBookWise cart a couple of days ago, When I went back today and tried to buy them, the program didn't let me. Instead, it told me that I already had them. In order to try out capabilities, I picked out another ebook in which I was mildly interested and bought it. I had to jump through hoops to identify myself and my online "bookshelf." Then, in order to get it onto my eBookWise reader, I had to use a special heavy telephone cord to get it from the "bookshelf" to the reading device. Then I went to load one of my personal books onto the eBookWise. In order to do this, I had to go to my computer and then upload the file into my "bookshelf" online. This required me to jump through a few more hoops. I am allowed only 10 mg of "personal" stuff. Then I had to go back to the reader and play telephone games a while longer. I'll try it out again when the new program allowing USB uploads onto the eBookWise, and allowing the unlimited library that the Rocket allows, arrives. In the meantime, I'm using my Rocket. My own recommendation is that eBookWise take a VERY good look at a Rocket and then rewrite the abominable software for this device. I'm sorry. I wish I had better news. Anne

1 0

Re: [gutvol-d] XML version of some books of PG (and other formats)
by Joshua Hutchinson 03 Dec '04

03 Dec '04

The problem I've always run into is where the table tries to grow beyond 80 characters wide. For instance, say that one row looks like this in the original book. Data label that Now we have a column Now we have a column is extremely long of data that is also of data that is also and is broken up very long and broken very long and broken accordingly over up over multiple lines. up over multiple lines. multiple lines. Most automated text converters will put each cell on one line with no line breaks. A web browser will generate line breaks within cells so that the table will end up looking very similar to the above. I haven't tried w3m ... will it handle the above scenario? I've tried lynx dumping to a text file and IE/Mozilla dumping to a text, and they all fail miserably. Josh ----- Original Message ----- From: "Sebastien Blondeel" <blondeel(a)clipper.ens.fr> To: "Project Gutenberg Volunteer Discussion" <gutvol-d(a)lists.pglaf.org> Subject: Re: [gutvol-d] XML version of some books of PG (and other formats) Date: Fri, 3 Dec 2004 17:53:49 +0100 > > On Fri, Dec 03, 2004 at 10:52:25AM -0500, Joshua Hutchinson wrote: > > The hard part is getting the table info within PG text 80 column width. > > It is not always possible of course. > > > FYI, this table becomes this in TEI markup (NOTE: I made the second > > That looks simple enough. > > > Club column just continue under the first for simplicities sake): > > Change it to HTML: > > -=-=-= > <table border="0"> > <tr> > <td colspan="4" align="center">THE RECORD OF 1875.</td> > [...] > -=-=-= > > then replace: > row -> tr > cell -> td > > then "w3m -dump table.html" gives: > > $ w3m -dump table.html > THE RECORD OF 1875. > Club. Won. Lost. P.C. > Boston 71 8 .809 > Athletic 55 28 .756 > Hartford 54 28 .639 > St. Louis 29 39 .574 > Philadelphia 37 31 .544 > Chicago 30 37 .448 > Mutual 29 38 .426 > St. Louis Reds 4 14 .222 > Washington 4 22 .156 > New Haven 7 39 .152 > Centennial 2 13 .133 > Western 1 12 .077 > Atlantic 2 42 .065 > > (the star after St. Louis has disappeared). > > If you need it embedded in a program I can try to code the algorithm, > depending on the programming language you want (Perl should be easy). > > Then you can detect cells with just numbers in them should be > right-aligned, etc. > > It should also be easy to translate this to LaTeX for PDF/DVI/PS output. > _______________________________________________ > gutvol-d mailing list > gutvol-d(a)lists.pglaf.org > http://lists.pglaf.org/listinfo.cgi/gutvol-d

2 1

possible fix for overwide tables in PGTEI text (was: XML version of some books...)
by Jussi Kukkonen 03 Dec '04

03 Dec '04

Joshua Hutchinson wrote: > I'm curious to see if your script can handle tables. That is our > current biggest bugaboo when it comes to transforming to PG TXT > format. > Now that you mentioned it.... I've been playing with PGTEI, and encountered this problem (too wide tables) also. If anyone is wondering what we're talking about, please search for string "1271-95." in http://koti.welho.com/jkukkone/geo/teioutput.txt Oh, feel free to see the html version also while you're there: http://koti.welho.com/jkukkone/geo/teioutput.html (warning for modem users - some images might still be pretty large). So, I spent some time with Groff* and Tbl** manuals and I think I found a fix for this. Currently Tbl input tables look like this (3 rows, 2 columns): *** 1873. Livingstone discovers Lake Moero. 1874-75. Lieut. Cameron crosses equatorial Africa. 1875-94. Élisée Reclus publishes his _Géographie Universelle_. *** Tbl _can_ be instructed to wrap lines when needed by changing the input to this: *** T{ 1873. T} T{ Livingstone discovers Lake Moero. T} T{ 1874-75. T} T{ Lieut. Cameron crosses equatorial Africa. T} T{ 1875-94. T} T{ Élisée Reclus publishes his _Géographie Universelle_. T} *** According to my tests it works suprisingly well. Sometimes Tbl does wrap too eagerly, but I saw nothing that wasn't acceptable. Marcello (or anyone with some authority on PGTEI), I can try and come up with a patch for tei2nroff-common.xsl, if that's wished for. Let me know. - jussi *, ** For those not familiar with the more obscure unix tools: Groff, or GNU Troff, is a document typesetting tool used in PGTEI to produce TXT and PDB versions. Tbl is a table formatting tool used by Groff -- Jussi Kukkonen

2 1

Re: [gutvol-d] XML version of some books of PG (and other formats)
by Joshua Hutchinson 03 Dec '04

03 Dec '04

I didn't think you could have a <head></head> inside a <table>... So I went back and looked at the TEI-Lite. It doesn't mention it, so I thought I was right. Then, just to be sure, I checked the full spec. There, it does mention <head></head> as a viable element inside a <table> So, you are right. <head></head> markup would be the more correct route. Josh ----- Original Message ----- From: "Marcello Perathoner" <marcello(a)perathoner.de> To: "Project Gutenberg Volunteer Discussion" <gutvol-d(a)lists.pglaf.org> Subject: Re: [gutvol-d] XML version of some books of PG (and other formats) Date: Fri, 03 Dec 2004 18:17:00 +0100 > > Joshua Hutchinson wrote: > > > <table rows="16" cols="4"> > > <row> > > <cell cols="4" role="label">THE RECORD OF 1875.</cell> > > </row> > > Shouldn't that be > > <table rows="15" cols="4"> > <head> > THE RECORD OF 1875. > </head> > > ? > > > -- Marcello Perathoner > webmaster(a)gutenberg.org > > _______________________________________________ > gutvol-d mailing list > gutvol-d(a)lists.pglaf.org > http://lists.pglaf.org/listinfo.cgi/gutvol-d

1 0

XML version of some books of PG (and other formats)
by Sebastien Blondeel 03 Dec '04

03 Dec '04

Hello, I hacked some scripts doing the following: RTF -> XML RTF: from Word, using a (very) simple stylesheet: just paragraphs, 3 title levels, footnotes, and italics Meta-information is in the properties of the document. My script can extract images too, if wanted. XML: using a personal and simple DTD (embedded), probably easy to port to any more complete DTD, such as TEI This is the hard part, and I am never quite sure it will not break in case the Word file is weird. >From that, I then did other (proof-of-concept) scripts to produce: XML -> PG TXT XML -> (LaTeX) -> PDF, DVI, PS (with hyperlinks) XML -> valid HTML 4.01 (probably useless) XML -> XHTML 1.0 Strict with some CSS (embedded) The programming is very defensive, so when all transforms finish I am confident enough the stuff is right. You can find examples of those formats at http://www.eleves.ens.fr/home/blondeel/ebooksgratuits/ (most of the books there don't have the meta-info properly set up, so don't worry too much about that). My scripts also clean up small typography mistakes (they are specialized in French rules but can of course be taught any thing). They will be used to help give PG nicer formats from the ebooksgratuits team (until now their Word macros could only produce PG TXT, which is not very sexy to read for the end user). Regards,

1 0

Re: [gutvol-d] interim report on eBookWise reader, info about Rocket
by Gutenberg9443＠aol.com 02 Dec '04

02 Dec '04

In a message dated 12/2/2004 9:10:38 AM Mountain Standard Time, joshua(a)hutchinson.net writes: The SmartMedia card is only supported up to 128MB and it lists for $30.11 at Amazon (which means you can probably find it cheaper other places, but that's a good ballpark figure). Thank you. I wasn't sure about that. It's $35.18 at CompUSA, plus shipping, so the Amazon price is probably about the best. Well, 128 MB will hold about 256 average texts, so that plus the built-in storage for another 20 books should suffice most people for the average airline trip or emergency room wait! The only time I read more than one complete book in either of those situations was the first time I flew after 9/11. I had to get to the airport two hours early and then the plane was two hours late. I spent most of the time sitting on the floor with my Rocket plugged in so I wouldn't drain the battery. Anne

1 0

Re: [gutvol-d] interim report on eBookWise reader, info about Rocket
by Joshua Hutchinson 02 Dec '04

02 Dec '04

Good review... Just one quick quibble. The SmartMedia card is only supported up to 128MB and it lists for $30.11 at Amazon (which means you can probably find it cheaper other places, but that's a good ballpark figure). Josh ----- Original Message ----- From: Gutenberg9443(a)aol.com To: gutvol-d(a)lists.pglaf.org Subject: [gutvol-d] interim report on eBookWise reader, info about Rocket Date: Thu, 2 Dec 2004 10:52:00 EST > > Hi all-- > > One quick note on the Rocket: If your power cord has gone out, call Fox > International at 1-800-321-6993 for a replacement. If you're in the US it will > cost about $33 including shipping. I don't know whether you can even get it > outside the US. Don't try to go online to get it so you'll have a printout of > the order; they'll just tell you to call the phone number. > > Now to the eBookWise reader: > > Judging from the info I have so far, its footprint is slightly larger than > that of the Rocket, but it weighs less--about a pound, whereas Rocket weighs 22 > oz. It's still slightly smaller than a trade paperback, though, and weighs > considerably less. Some of its controls seem counterintuitive to me, but that > may be just because I've used the Rocket so many years. > > There are thousands (about 7,500 to be precise) of commercial books for > those who can afford to buy them; they're at eBookWise.com, which is a > subsidiary > of FictionWise.com, and is presently engaged in converting all, or almost > all, of its content into an eBookWise format. Some of them are new books, > often > bestsellers, and others are classics that are not yet out of copyright. If I > had it to spend, I could spend a thousand dollars there in two shakes of a > puppy-dog's tail. Alas, I don't have a thousand dollars. > > Although loading your own content is right now rather clunky, involving > upload to a server and then download through a telephone line, it is doable. > You > can keep your own bookshelf at the server, and download by telephone, so you > don't have to take your computer on the road with you to change the books in > your reader. They have their software engineers working on a direct USB > download program, the kind Rocket has, but it isn't ready yet. This close to > Christmas, they made the decision to make the device available now and fix the > software later. > > This is a quotation from eBookWise's propaganda sheet: > > "In addition, the eBookwise-1150 can display your own personal content in > the following file formats: plain text (.txt), rich text format (.rtf), > Microsoft Word documents (.doc), HTML (.htm or .html), and Rocket eBook > Editions > (.rb)." > > So that means that just about any free online book is readable on this > reader, except that far under 1% that are available only in PDF. So--PG, PG Oz, > Blackmask, _http://www.sacred-texts.com_ (http://www.sacred-texts.com) , > Phoenix-Library.com and umpteen dozen other sites are now your oyster, in a > reader > you can fit into any briefcase or backpack and most large-size handbags. > (Phoenix-Library has good language-to-language dictionaries in a surprisingly > large number of languages.) > > It will hold only about 20 texts at a time, which was also normal for the > Rocket unless you bought the extra large storage device when you ordered your > Rocket. I did, and my Rocket holds about 50 texts. However, it has a slot for a > SmartMedia card; I checked, and you can get SmartMedia cards holding > anything from 4 mg up to half a gig. (The half-gig card costs about $250.) > > It does not do much with illustrations, as it is grayscale only. However, it > appears that at least some small illustrations can be put into it. > > If you want one right now, go to eBookWise and create an account. Put $110 > into your account. You can immediately order your machine, which will use 10 > cents short of the entire $110, but it will then give you $20 in book credits. > Wait to buy them until your device arrives, but you can go through and select > them and put them into your cart now. Of course, you can deposit as much > money as you want to into your account, and put the books into your shopping > cart, to purchase when your device arrives. > > This is not the best possible ebook reader, but it is the best presently > available for anybody who is not content to read ebooks only on a computer or a > PDA. > > Anne > > > _______________________________________________ > gutvol-d mailing list > gutvol-d(a)lists.pglaf.org > http://lists.pglaf.org/listinfo.cgi/gutvol-d

1 0