- gutvol-d - lists.pglaf.org

How new technology scares away users
by Marcello Perathoner 19 Oct '04

19 Oct '04

This is a video that turned up in the TEI list about the problems users have with all those new-fangled publishing technologies. Its in Danish but you still get the gist. http://homepages.nyu.edu/~mz34/helpdesk.WMV -- Marcello Perathoner webmaster(a)gutenberg.org

1 0

re: [gutvol-d] Why Bowerbird is a kook
by Bowerbird＠aol.com 19 Oct '04

19 Oct '04

once again, because i guess i haven't said it enough, the beta-test for my viewer-program is now open... you can join in and see the elusive software yourself by e-mailing: zml_talk-subscribe(a)yahoogroups.com as soon as i have enough people to get that test going, i'll be able to move on to that class-action suit against the computer overlords for wasting our time with x.m.l. i think the settlement on _that_ is gonna be _huge_! ;+) -bowerbird

2 1

[ANN] Project Gutenberg IRC channel
by David A. Desrosiers 19 Oct '04

19 Oct '04

I've offered this before, and maybe it got lost in the chaff of previous messages and threads, so I'll offer it up again. If anyone wants to feel free to discuss issues related to Project Gutenberg, ebooks, conversions, tools, bugs, the weather, or whatever else in "real-time", they can join our network to do so. I run an irc network that is dedicated to developer-related support issues on various projects, and I would like to extend that invitation to include Project Gutenberg as well. Feel free to join the channel #gutenberg or #project.gutenberg on our network... (irc.sourcefubar.net) and talk about any of the current, past, future, or whatever events you'd like, related to PG and other similar projects and products. We have redundant tri-coastal links, so the network will not "split" or go down like (cough) other similar networks. We also offer ssl-only ports, for those who wish to make sure their conversations are "secured". Many project teams use our network for exactly that purpose, in private, secured channels to discuss issues related to their projects. If you join the same network on port 994, and configure your client appropriately (making sure to get the validated cert from cacert.org) you will be able to talk securely from client-to-server, without the feer of snooping. These are exciting times, and I hope to see everyone there! Can someone with authority publicize this on PG's website? Welcome aboard! David A. Desrosiers desrod(a)gnu-designs.com http://gnu-designs.com

2 1

re: Re: [gutvol-d] re: poofing and tarking
by Bowerbird＠aol.com 19 Oct '04

19 Oct '04

anne said: > Why can't you just vanish, > if such is your preference, > without being obnoxious about it? i don't think my posts are "obnoxious", anne, or i wouldn't write them. but if you think they _are_ "obnoxious", anne, then why do you read them? -bowerbird

1 0

re: Re: [gutvol-d] re: poofing and tarking
by Bowerbird＠aol.com 18 Oct '04

18 Oct '04

david said: > Quite a bit more than "a little", but yes, thats me. that's what the winkey-smile was all about... :+) > What other efforts are you alluding to? > Why not help those people who insist on > reinventing a fleet of new wheels, to collaborate > with existing projects that have similar/same goals? well, mainly because amazon is looking for any "help". and my guess is that google will be equally isolationist. moreover, the publishers they are both coddling do not share the "let's share" mentality that drove michael hart. stanford has talked about a huge digitization effort, but i'm not sure how far they are, or if they've even started. but it is safe to say they've got enough money that they won't be on the lookout for any volunteers to assist them. and i'm not sure what's up with the million-book project, or even who is in charge of that effort, but their objective seems slightly different than the one guiding things here; they're seem to make scan-books, not cleaned-up e-texts. (they _are_ doing o.c.r., but they're not proofing the stuff.) and the specific area that's of _most_ interest to me is the one comprised of the d.i.y. authors using cyberspace to connect directly to their audience, sidestepping the clutches of the middlemen who were necessary before. this is _new_ content, so i think it'll eventually eclipse the public-domain that is the thrust of project gutenberg... > And this is exactly what the Distributed Proofreaders project > proposes to solve, and they've been pretty successful thus far, IIRC. um... distributed proofreaders _might_ solve the problem of inconsistent formatting _sometime_ down the line, if their policies settle into place. but if you've looked at their output throughout their history, you'll know they have not done it yet. but, honestly, that shouldn't even be their job in the first place. project gutenberg should have had solid formatting rules down long before distributed proofreaders even came into being... > I've had a lot of luck stepping out of the box, and > analyzing the text based on the "style" of the text, > versus the actual content itself. I was approached by > someone who is doing a paper and his PhD thesis on > this exact kind of approach. Basically (with my expertise > and help) he's taking the bulk of Gutenberg, importing > every word from every work into a database, and > then running his own algorithms across the entire collection, > to pull out the styles by known authors. well, that sounds interesting. :+) but "authorial style" is fairly irrelevant to the matter of formatting the text in the way that a typographer does it, or an e-book requires, which is actually the task at hand... > And I assume you've done this? yep. > And your routines are made public somewhere, > so others can improve and correct them to continue to be better? nope. if i turned 'em loose now, the tarking naugshlocks here could just pick them up for their own nefarious purposes, and i have _no_ intention of letting my hard work be used that way. i came here to the project gutenberg listserves in the first place because i intended to share, but that intention has been squashed. besides, all the negative feedback i've gotten here has convinced me that people out in the world think that what i've done is impossible. so i figure there must be a couple bucks in it, and why give them up? i don't have a day-job, and my girlfriend deserves some nice things... > And where is his code? Where are his "routines"? i don't believe that moynihan has ever made his macros available. he _did_ offer them to project gutenberg, but i guess they declined. i can't express to you just how stupid i think _that_ decision was... > ALL of the talk of how "easy" this is, is completely irrelevant, > if nobody wants to actually contribute that knowledge back > so others can improve and benefit from it. if somebody thinks something (which would be very valuable to them) is impossible, and you know that it isn't, don't you think that you should _tell_them_? i do. that's why i'm here. i'm not gonna do it _for_ them, because they've abused me so -- are you in the habit of helping people who mistreat you? -- but, since i do believe so passionately in electronic-books, i feel i have an obligation to _try_ and make them wake up. that's why i've stayed here for so long, nearly a year, and taken all the abuse that they have dished out to me. because i believe in e-books, and i admire michael hart. (michael, by the way, has been very supportive of me.) but i'm about to give up, because they just won't listen. nonetheless, i feel _good_ about the fact that i _tried_... > If you're not willing to do this, then our conversation stops here. ok. no problem. i believe in sharing, and told you what you could share with me, if you want me to share my work back with you, but if that's not acceptable to you, then i'm cool with that. i'll go my own way... > What was that "bizarre reason"? you'll have to ask the people in charge. > If you want someone to support "your" format, > then you'll probably have to take that first step by > justifying and documenting it. i've done that. i've posted it several times on this listserve. and i will send it to you backchannel. 11 dirt-simple rules. > The only page I could find describing the format was here: > http://czt.sourceforge.net/zml/ > And I assume thats not your project or code. you're right, that's not it. > What kind of errors? Incorrect hyphens? Broken paragraphs? > Missing end quotes? (this is common) all of those, yes. and many more. every kind imaginable. and some that you never would have been able to imagine. > Impossible to regain, unless you have the original work in-hand, > to see if there were actual CAPS used, or not. right. > Maybe the "errors" were intentional. Many authors use > poetic license to express their thoughts, and sometimes > those things break the rules of grammar and spelling. that's not it. the all-caps convention dates back to the days of keypunch machines, when computers had no lower-case characters. (there is a rumor that i started that michael hart actually entered "alice in wonderland" on a keypunch machine. don't know if it's true.) > How do you mean? You mean 1.jpg 1.jpg 1.jpg appearing in > three places, but intended to represent 3 _different_ images? no, i mean 1.jpg being used in 3 different _e-texts_. which means that you can't dump those e-texts into the same folder without experiencing a filename crash. which means that you need to rework all the filenames in the library if you want 'em to be unique, which is something that you do really want, if you value your sanity... > How do you express a Cyrillic text in 7-bit ascii? You can't. right, that's the problem, though 8-bit e-texts have become common. many mostly-english texts, though, do have foreign words in them where an 8-bit diacritic was chopped down into a 7-bit character. some of these can be automatically replaced. but then you are running the risk of turning what _was_ a non-diacritic into one. (for example, burkey cites the change of "role" to "role" with a hat on the "o". but you know there are plenty of plain "roles" out there.) > Such as? many large works have been split up into smaller sections, but then also "collected" into one e-text as well. there are also "collections" of certain authors, and so on. you'll want to cull out this redundancy... > Are you revealing the "problems" in a condescending way? > Or in a constructive way? i know of no more constructive way to reveal a problem than to diagram the code that will fix it and volunteer to write the app. i've done that, and had shit heaped at me. you be the judge. > The way you approach the "Hey, this is broke" process > is very telling as to how you will be received and > responded to for same. and the way i am "received and responded to" is very telling as to whether i will _continue_ to offer my code that will help fix the problems... > which is exactly why you should have > your own mirror of Gutenberg, > or a subset of it as you work on the pieces. i was just letting you know that piece of information. otherwise, you might think that you could get the d.v.d. of the e-texts, and simply work on the e-texts from that. odds are some of those files have already been replaced... > If you are "moving on", then it behooves you to try to contribute what you've learned (in terms of knowledge, code, or "routines") back to those who will continue to contribute and learn. i've left a ton of messages, detailing the problems and laying out exquisite details about the fixes i suggested. feel free to mine that, if you can plow through the flack. (the vast bulk of my posts are over on the u.n.c. archives; i'm not sure if those have been brought to this list, or not. you should also be aware that the .html conversion program over on the u.n.c. machine was faulty, and many of the threads are cut off in midstream. the full thread is in the .html source, so you would have to recover the missing messages from there. that's a good example of how a conversion program can mess up.) > We're only here to help the next generation learn and improve. > If we're not leaving anything here by which others can > remember us and grow themselves; if we're not teaching others > as we learn ourselves, then what is the point? oh, don't get me wrong, david... :+) as i outlined above, there are plenty of arenas in the e-book world, project gutenberg is just one of them. all the others need help too. so i don't intend to stop speaking, or to stop working on e-books. i've been doing continuous work on e-books for over 25 years now. to the contrary, i intend to _continue_to_speak_, and _loudly_, and to finally _go_to_work_ and get some of my things finished, so e-book authors out there in the world can start _using_ them. the difference is, rather than speak here _quietly_ and _privately_ with the project gutenberg folks here _behind_the_scenes_ on their own listserves, trying to get them to pay attention to the problems in their library, i will instead speak _publicly_, using my new blog, making noise about the many problems that are being ignored here, so at least the rest of the world learns -- and grows -- from them. so instead of working to make the "people in charge" here smarter, which has essentially meant banging my head against a brick wall, i can instead spend some time _productively_ by making programs that people can use to spread e-books out into the world at large. my time, thoughts, and work-product are too valuable and important to continue wasting them here on people who do not appreciate them. yours probably are too, but maybe you're more "diplomatic", and maybe they won't ignore you or badger you when you say something that they desperately _need_ to hear, even if they don't _want_ to. oh yeah, and eventually i'll even come back and clean up the e-texts, when i've automated my various procedures to the fullest extent and implemented them in code, and the people-in-charge here have made a mess of the library in the process of trying to make x.m.l. work. because michael hart deserves better than that... -bowerbird

1 0

re: Re: [gutvol-d] re: poofing and tarking
by Bowerbird＠aol.com 18 Oct '04

18 Oct '04

anne said: > I don't know why it was spurned, but > I do know that he's posted a lot of > stuff that is still in copyright and > sooner or later lawyers are going to > eat his lunch. As long as nobody could > figure out who owned the copyright, > that was okay, but now that courts have > ruled on the owner, it's another matter. i don't know anything about that. i'm very supportive of people who will have the guts to publish something and take a chance at being dragged into court, if they are making that thing _available_ when it was an orphan out of circulation. but again, i don't know about blackmask, so i don't know if that applies, or not... -bowerbird

1 0

Re: [gutvol-d] re: poofing and tarking
by Joshua Hutchinson 18 Oct '04

18 Oct '04

----- Original Message ----- From: "David A. Desrosiers" <hacker(a)gnu-designs.com> To: Project Gutenberg Volunteer Discussion <gutvol-d(a)lists.pglaf.org> Subject: Re: [gutvol-d] re: poofing and tarking Date: Mon, 18 Oct 2004 16:34:59 -0400 (EDT) > > I've had a lot of luck stepping out of the box, and analyzing > the text based on the "style" of the text, versus the actual content > itself. I was approached by someone who is doing a paper and his PhD > thesis on this exact kind of approach. Basically (with my expertise > and help) he's taking the bulk of Gutenberg, importing every word from > every work into a database, and then running his own algorithms across > the entire collection, to pull out the styles by known authors. <snipped some good stuff> You've really intrigued me by your description of what you're working on. Is there anywhere I can read up more on it? Sounds very promising. Josh

1 0

Re: [gutvol-d] re: poofing and tarking
by Gutenberg9443＠aol.com 18 Oct '04

18 Oct '04

In a message dated 10/18/2004 1:54:55 PM Mountain Standard Time, Bowerbird(a)aol.com writes: there are a handful of tarking naugshlocks here _so_ unworthy of my help they made me decide to decline to do any work for project gutenberg, in spite of its great historical importance and my highest regard for the genius of michael hart. i'm sure others, like you, will cover my absence, while i will be happy grazing greener pastures... Why can't you just vanish, if such is your preference, without being obnoxious about it?

1 0

Re: [gutvol-d] re: poofing and tarking
by Gutenberg9443＠aol.com 18 Oct '04

18 Oct '04

In a message dated 10/18/2004 1:54:55 PM Mountain Standard Time, Bowerbird(a)aol.com writes: but -- as far as i know -- his gift was spurned, for some bizarre reason i'll never be able to grasp. I don't know why it was spurned, but I do know that he's posted a lot of stuff that is still in copyright and sooner or later lawyers are going to eat his lunch. As long as nobody could figure out who owned the copyright, that was okay, but now that courts have ruled on the owner, it's another matter. Anne

1 0

re: poofing and tarking
by Bowerbird＠aol.com 18 Oct '04

18 Oct '04

craig said: > you poofing friggleschnitz! mommy, mommy, craig is starting a flamewar! he called me a poofing friggleschnitz, mommy! i demand that he be banned! ;+) *** david said: > One and the same, yes. great! :+) and you did a little bit of work on... what was it?... plucker, right? ;+) if you are prepared to tackle the i.m.d.b., 15,000 e-texts should be a piece of cake... > ...much like bringing 5 friends into a video store, > and trying to agree on one movie for everyone > to watch. Not going to happen ;) i wish things were that inconsequential. but the project gutenberg e-texts make up the most important e-library, historically. although even now it's starting to be dwarfed by other efforts, it would be a proper tribute to michael if it were to be well-maintained... > That being said, I'd be interested in seeing > a list of the tools people know of, or are > working on, or have worked with in the past, > that can be used to take a 7-bit ascii text > PG work, and convert it into other formats. "convert" is a rather loose and unspecific word, wouldn't you say? :+) nonetheless, i'll cut to the chase... the main problem with the e-texts is their formatting is _so_ inconsistent. so before you can do anything useful with them, you must write routines that can resolve their inconsistency. the inconsistency is very maddening, because it's so pointless. although some is understandable, considering how many hands created the e-texts, the sadder truth is that much of it could have been prevented; however, mr. newby and company simply fail to grasp the negative consequences of the inconsistency, and thus never made it their priority to minimize it. the good news is you _can_ write routines that will fix the problem. it is _not_ impossible, just thorny; the biggest expenditure of time is a quality-control check to make sure that you knew every inconsistency. their variety will amaze and astound. subsequent conversion to any format is straightforward once you have done the job of resolving the inconsistency. you don't even have to do that job, if you don't want to, you can just go to david moynihan at blackmask and get his files, as he has edited out almost all the inconsistency, which is what then allowed him to make a half-dozen versions of most e-texts in the entire library. if you're looking for explicit info, ron burkey did a converter called "gutenmark", and his website at http://www.sandroid.org/gutenmark does a good job of documenting the inconsistency he faced on the way, before he gave up the effort, saying: > the more perfect my > automated conversions became, > the farther (in my own mind) > I seemed to be from > having a perfect conversion. i think that's a nice way of saying that the more he learned about the e-texts, the more he found out how bad they are, from the standpoint of consistency... there is also some basic information at: palmdigitalmedia.com/dropbook/converting but i'd guess that at this point in time, moynihan will have the most expertise about the problems you would be facing. much of it might be inside his noggin, but i do know he has a _lot_ of macros that undoubtedly embed gobs of wisdom. and, more to the point, david has shown, incontrovertibly, that mass conversions to a plethora of formats is fully possible. recently, david even _offered_ his files to project gutenberg, but -- as far as i know -- his gift was spurned, for some bizarre reason i'll never be able to grasp. oh yeah, i've written some routines that squash out most of the inconsistency, and there's a way you could pry 'em out of me -- namely, if you got support for my z.m.l. (zen markup language) built into plucker. it's a simple rule-set; you could probably have it up-and-running in a couple days... backchannel me if you're interested. :+) once you've vanquished the inconsistency, there are other concerns, which might or might not be a problem to you, including: 1. errors in the e-texts, lots of them. 2. styling lost or converted to all-caps. 3. information about images discarded. 4. image filenames are often not unique. 5. accents lost in many foreign e-texts. 6. a confusing redundancy of some books. 7. attacks levied if you reveal problems. oh yeah, also make sure that you are always working with the freshest e-texts available, as i'm not sure if they make an announcement whenever they make corrections to an e-text; they just quietly substitute in the new file... *** i would welcome you here, but i am on my way out the door _very_ soon... :+) there are a handful of tarking naugshlocks here _so_ unworthy of my help they made me decide to decline to do any work for project gutenberg, in spite of its great historical importance and my highest regard for the genius of michael hart. i'm sure others, like you, will cover my absence, while i will be happy grazing greener pastures... at any rate, have a nice day... ;+) -bowerbird

2 1