February 2012 - gutvol-d - lists.pglaf.org

A New Approach : Forward
by Keith J. Schultz 06 Feb '12

06 Feb '12

Hi Everybody, I am moving forward on my suggestion "A New Approach". First let me say I personally do not care wether you what kind of mark up you use. I do not care about semantic mark or the use there of! I do not care what was/is done on DP already. This is a new approach. PG lack any kind of consistent specification of what is expectable or gives any guidance towards developing tools and tool chain. I will try to develop such a specification. NO, it will not be a professional specification. It might be along the lines of Guide. It will not be technical in language, though most tech-talk I hope to be in understandable English. It will be geared towards what any format that is used should contain. If you need want more go ahead, but keep in mind that this specification will (if adopted) be the basis of the structures of a master format and any features you use will have to map to these! Why? Well, that will become obvious as we go along. Most important of all it is to also be collaborative and hopefully adopted be PG in one form or another. IF WE do a good job I hope it will. On the other side I will be re-iterating the obvious and what is pretty much in common use, yet not specified. Though there is always room for improvement. regards Keith

1 0

yagni, yagni, yagni.
by Bowerbird＠aol.com 06 Feb '12

06 Feb '12

greg said: > The basic feature list we'll need for a complete tool is > fairly short, and might build upon multiple existing tools: > - able to authenticate users > - able to handle different user authorizations, > at grainularity finer than "whole site" (i.e., someone > might be committer for one eBook, but not all eBooks) > - can display differences, merge trivial results, handle > forks, and similar source code revision practices > - keeps history; can display older versions of files > - can be plugged into a transformation system > - can use structured metadata (DC, XML, MARC...) > to populate information about books > - can handle our 38,000 titles in ~1.5M files > - can handle books (i.e., "projects") with > multiple source files, as is found in HTML+images > - a documented API > - can work with multiple file types > > Conceptual versus functional requirements: > - well documented in English > - runs on a Linux server > - active development community > - open source > - able to accept code contributions, track feature requests, etc. > > I would add these as desirable, but not critical: > - exposes the back-end filesystem, for use with other tools > - some sort of rating system for most popular items (or, > if not, it is required that we can build one upon the tool) > - integrate wiki or bug tracking or other features > in support of the effort and then, for good measure: > This is just a quick listing, and is probably not complete. wow. yagni. yagni, yagni, yagni. you ain't gonna need it. period. good thing too, because the complexity of it would bury you. it amazes me how you can take something that is _so_simple_ and infest it so fully with such a confusing mess of complexity. here's a copy of pride and prejudice. > http://zenmagiclove.com/prapr/prapr.html as you probably know, it's a pretty popular book. it's been considered to be a classic for some time now, and i know it must get a lot of downloads, from all of the e-book websites. and it's really simple. as you can see. plus, it ain't gonna change. not too much anyway, not in the near future, or the distant future. now, there might be errors in this, but -- once you found 'em -- pointing to them would be a pretty simple matter, and then the correction procedure would also be a very straightforward thing. here's a plain-text version: > http://zenmagiclove.com/prapr/prapr.zml as you might imagine, i generated the .html version, up above, from this .zml file. it was easy. really easy. because there is very little markup required by this book... _remarkably_ little... like any typical book, it consists mostly of plain old paragraphs. there are some italics sprinkled about. and some blockquotes... and then of course there are the chapter-headers -- 61 of them. and the title-page and table-of-contents required some markup. and a few small-caps, which i discarded as "decorative" elements. but that's it. so this is another "baby" book, like the ones jim did, which requires almost no markup at all. so it's just a bit _ironic_ that marcello used this book to demonstrate his markup prowess. i can't imagine any reason in the world why you might think that you would require some kind of complex system to "handle" this. you put 'em on a web-page, and that's pretty much the end of it. you'd want to put up some other copies of it as well, for users... like an .epub version, and a .mobi version, and a .pdf, and so on. like i just posted in the a-b-c's of first-aid for project gutenberg. and "the crap formats you serve". and a million other prior posts. and i could use the plain-text version to generate those formats, with a click of one button, using some simple conversion routines which i programmed myself... and marcello will be all to happy to inform you what a crappy programmer i am. yet... i made it work. and you can do the same thing with 94% of the books in your library. and probably more, but percentagewise, let's start with the first 94... that's if you do what needs to be done. but if you keep wasting your time and energy on fantasy schemes for some "sophisticated system" promising to do a bunch of crap you don't even need to do, then you won't get a damn thing done. and you know what's funny? :+) what's funny is that you're gonna choose the stupid path. again. yagni. yagni, yagni, yagni. you ain't gonna need it. you ain't... -bowerbird

1 0

the a-b-c's of first aid for the project gutenberg patient
by Bowerbird＠aol.com 06 Feb '12

06 Feb '12

greg said: > I'm still pursuing the theme of > an existing sophisticated code base to enable supporting > multiple versions & formats of our eBooks > (user-contributed, curated, edited, masters and revisions). you don't need a system to handle "user-contributed revisions". because you don't need any "user-contributed revisions" at all. that'll just add more "snowflakes" to the avalanche burying you. what you need is very simple. you need to _improve_ your converter routines with a few small changes that'll immediately end 85% of the complaints you get. this is the equivalent of clearing airways and stopping bleeding. then you need to apply some elbow grease to fix the remaining glitches, in the derivatives, and curtail regenerating any of 'em, so as not to rewrite over the fixes which you just had to apply. this is the equivalent of controlling for shock; quite important. then you need to re-do the _input_ for those problem-children, so you can resume the regeneration at the earliest opportunity. this is the equivalent of making splints for all the broken bones. finally, you need to test all new submissions, to make sure that you are not allowing any new problem-children to be admitted. this is the equivalent of protecting the patient from new injury. eventually you will offer a handful of formats for every book -- such as plain-text, .html, .mobi, .kf8, .epub, nook, and ibooks. (yes, nook and ibooks officially use .epub, but you'll be able to create versions of an .epub file specially customized for them.) and none of the formats will need to be "generated" on-the-fly. they will all exist as solid versions, both known and dependable. the only time you will regenerate them is when you fix an error. and way far down the line, you'll be able to customize output to individual preferences, but walk first, before you go tryin' to run. because today, you've got broken bones throughout your body... -bowerbird

2 1

Existing software/platforms
by Greg Newby 06 Feb '12

06 Feb '12

I'm still pursuing the theme of an existing sophisticated code base to enable supporting multiple versions & formats of our eBooks (user-contributed, curated, edited, masters and revisions). Last week I had mentioned looking at TRAC. That is a tool that layers over version control systems like Mercurial or git. I've also been looking at DSpace, which I'm in the midst of rolling out for my campus (i.e., day job), and I will try to ingest all of the files & metadata shortly. However, DSpace does not yet have versioning capability, which I think will remove it from consideration. Two others I'm taking a dive into are Notre DAM (http://www.notredam.org/) and ResourceSpace (http://www.resourcespace.org/about.php) Although these both seem more geared towards photos than eBooks, they do have a suite of transformation capabilities, and accept user contributions. I found a recent comparison of such "digital asseet management" systems here: http://www.opensourcedigitalassetmanagement.org/ It would be great to hear of anything people know of these or other existing systems. At the same time, I know people are looking at using other existing tools that are more general. -- Greg

4 16

and all you yap dogs? the archives mock you.
by Bowerbird＠aol.com 06 Feb '12

06 Feb '12

i said: > just suffer in your super-stupidity... at least greg suffers in silence. that's "dignified". all you noisy yap dogs, nosily yapping on this list? you are just as stupid, and a whole lot more noisy. the archives of this listserve mock you. they do. the cyberarchaeologists who mine this listserve will laugh at the fact that you yapped in circles for years while the solution was sitting right in front of you... here it is, reduced to one line (disregarding .pdf): .zml -> .html -> .epub -> .mobi -> .nextbigthing or, to put it another way: .zml == .html == .epub == .mobi == .nextbigthing for any chunk of text in the book, you can trace it through all the conversions, from master to slaves. and yes, i could back-translate from slave to master, because i can put the necessary info in each format. in which case it's just specifying a reg-ex to change markup from one system to the markup of the other. but of course, since one is using the .zml as "master", one always has the .zml hanging around, so one really doesn't have any need to ever do the back-translations. so yap all you want, noisy yap dogs. yap yourselves happy. but the archives mock you; mock you and your stupidity. but hey, enjoy the super football game... :+) -bowerbird

2 1

severe and irreparable damage
by Bowerbird＠aol.com 06 Feb '12

06 Feb '12

greg said: > Nothing that Marcello or anyone else did > is deployed to www.gutenberg.org yet. geez, greg, please, when you say stuff like this, which is so easy and obvious to prove _demonstrably_ false, you do severe, irreparable damage to your credibility. > http://www.gutenberg.org/ebooks/1342 > http://www.gutenberg.org/ebooks/1661 and lest anyone think these uploads are restrained to the "generated" formats only, take a look at the utf-8. you'll see that it's the version derived from his .rst, as marcello knows _that_ is what the repackagers grab... that's cheating. on a big scale. and it's totally unfair. it is also cheating because who announced that those two books were the first ones that we would be doing? how come we all didn't get to begin at the same time? that's cheating too. on a smaller scale. but still unfair. -bowerbird

1 0

the crap formats you serve them (a historical review)
by Bowerbird＠aol.com 06 Feb '12

06 Feb '12

greg said: > This is because we get frequent requests > to add such files into the PG collection, and > have no good, standard, scalable way to do so. > Plus, complaints about shortcomings > of existing mobile formats. you have it backwards. first people realize your mobil formats are crap. so they rework your files, so they won't be crap. then they figure they might as well offer you the reworked files, so other people won't have to eat the crap formats you seem to want to serve them. that's the sequence of events, the way it happens. however... if you _didn't_ serve people crap in the first place, nobody would ever _bother_ to rework your files... so you'd get no complaints. and zero files offered. you'd just have tons of happy users. end of story. nobody re-did david moynihan's black-mask files. because he did a good job on 'em in the first place. and he put them on machines that were _primitive_, so it doesn't behoove you to use that as an excuse. and nobody reworks the files from feedbooks.com. because they're "good enough". doesn't take much. even the relatively crappy files from manybooks.com, available in 289 various formats, don't get reworked. (and again, they targeted a lot of _crude_ machinery.) because all of those places were _responsive_ to the wants and needs of the users. they _served_ people. you gave people .txt, even after they wanted .html, and .prc, and .mobi, and .lit, and all that other shit. they wanted it, but you wouldn't give it to them, so you opened a door for all those other sites to do it. then you started doing .html, but you did it _wrong_, because you allowed producers to make _snowflakes_, even though you got frequent and extreme warnings -- from me -- that you were making a bad mistake... now, you are feeling the consequences of your error. and i'm saying "you know, i told you so, i really did." but you didn't learn, did you greg?, didn't learn at all. you now want to compound your snowflake problem by "solving" it with even _more_ snowflakes. sheesh! *** i also told you what you _should_ have been doing... all along, i told you, and now i will tell you _again_... use a light-markup format like .zml as your master, from which you generate _good_ versions of all the formats your users desire, and everyone gets happy. for at least 5 years now, i have demonstrated that a robust text-to-html conversion is possible, and once the kindle came along, i showed that it was merely a matter of generating the auxiliary files to support that format, and the same with .epub... and it's not like this was some secret process, either. that's how all of those other sites generate their files. indeed, after deciding to drop public-domain e-books, because he was no longer making much money off 'em, david moynihan offered to project gutenberg the scripts he had created to generate all of his derivative formats. and you guys spurned his offer! i was totally astounded! meanwhile... i also coded and showed editors and authoring-tools, both offline and online, and systems for collaborative editing, proofing, formatting, and error-correcting... i've even written tutorial series on such programming, and -- quite recently -- explicitly _shared_the_code_. so a solution has been sitting right in front of you, for years and years, but you decided to choose to willfully not see it, to intentionally refuse to look... and you continue to refuse, right up to this day, super-bowl sunday with the giants vs. the pats... the solution still sits there. and you refuse to see it. and i have told you, and told you, and _told_you_... but you know, greg, i'm getting tired of telling you, since you haven't ever listened before, and there is no reason for me to believe that you will listen now, and -- like i said -- i'm getting tired of telling you... so, you know, just suffer in your super-stupidity... but hey, enjoy the football. it, too, will be super... -bowerbird

2 1

Article in Innovation News Daily
by Greg Newby 05 Feb '12

05 Feb '12

DP got consulted for this. The article talks about the value of people in the loop for transcription: Volunteer Transcribers Put Millions of Pages Online http://www.innovationnewsdaily.com/846-volunteer-transcribers-put-millions-… -- Greg

1 0

Where are the RST manual sources?
by traverso＠posso.dm.unipi.it 05 Feb '12

05 Feb '12

What happened to the master file of the PG-RST manual, PG 181? I can find the manual in several final form, but not in RST. Consultintg these sources is sometimes the only way to understand how to use some feature (and often even this doesn't help). I have copies, but I wanted to send a link to a proofreader that might be interested to try RST. Carlo

3 2

strongly resent, even flatly reject, getting dumped
by Bowerbird＠aol.com 05 Feb '12

05 Feb '12

al said: > And then what happens to their text/HTML files? > Do they get dumped? I can think of at least > four independents (one of whom is me) that > would strongly resent, even flatly reject, > that being done to their files. well, that probably varies from one producer to the next... for instance, david widger probably won't be bothered that he was whitewashed out of marcello's "pride and prejudice". after all, david has his name in _lots_and_lots_ of p.g. books. he probably won't even notice that it was removed from one. and jose menendez probably won't mind too much that his name was erased from marcello's .rst of "sherlock holmes". jose hasn't done many books, so it's probably "no big deal". plus, after all, their work is still available, in the "old" folder, under the "more files" tab, which people peruse all the time. you're familiar with that "old" folder, mr. haines, aren't you? so, you know, al, maybe you're just being "overly sensitive". maybe you should be a bit less "protective" about your work, learn how to be more magnanimous, less needy for "credit". still, i bet you're right that you could find 4 such producers... *** greg said: > I'd like to go ahead and make a way to get these > back into the collection, replacing the originals, > *en masse*. (Actually, we keep the originals, > in an 'old' subdirectory.) I don't anticipate > opposition to this idea, assuming we're tweaking, > not redoing the look and feel crafted by the submitter. oh, wait, al, greg doesn't think there will be any objections. so, see?, i guess we're all good... -bowerbird p.s. this is all so complicated because we can't just _talk_ to people, we have to read their minds, it makes it so hard.

3 2