January 2012 - gutvol-d - lists.pglaf.org

Re: [gutvol-d] the idea of using it
by Bowerbird＠aol.com 20 Jan '12

20 Jan '12

andrew said: > Just because you were not aware of RST being used on PG > does not meean that it has been "hidden" from you. > The idea of using it came from some volunteers from pgdp. really? "the idea of using" light-markup as a master-format "came from some volunteers from pgdp"... really? well, let me congratulate those far-sighted pioneers. and you, andrew sly, you are one sly devil, aren't you? trying to rewrite history, right here, right in front of me. well, yeah... nice try, mr. sly, but no dice. the archives clearly prove that it was _one_person_ who introduced the idea of light-markup as master-format... a full 8 years ago. _and_ that a boatload of rocks were thrown at me for it. for years -- literal _years_ -- people threw mud at me... look at the archives if you don't believe me, people. if you don't _remember_ it, or if you were not here. because i remember it well, and quite vividly. i do. and i think you will too. unless you're repressing it. because lots and lots and lots of shit got pitched at me. mud and rocks, and insults and taunts, and i took it all. i took it, from a vicious pack of wolves out on an attack, and i withstood it, and i prevailed, and that's a fact, jack. and the person who threw the most rocks of all, even if he had such a bad aim that not a single one ever hit me, was marcello. the asshole. he was toothless, as a wolf, but he tried his hardest to be as viscous as he could be. perhaps you've seen the hate-site he put up about me? anyway... for years he insisted that my idea was stupid, that light-markup could never work on anything except the most simplistic documents, nothing that was "hard". so i let him bellow his ignorance for a good many years. until i felt he had wagered _all_ of his credibility on it... and then i set the trap. i simply mentioned in a post that the python community had been using a light-markup -- restructured-text --- for its documentation ever since the language went big... it was pretty hard for marcello, who was using python, to deny the reality that geeks were using light-markup. for documentation that was far from being "simplistic". so that shut him up, pretty much. but wait, there's more, because that wasn't the trap... so, once the kindle came out, marcello quickly realized that this was gonna be a big deal for project gutenberg -- when you look at the traffic stats, it's very obvious -- but he had to have a solution that would actually _work_, as opposed to the t.e.i. bullshit he had been advocating. bingo. he stepped in my trap, and picked up restructured-text. game, set, match, championship and trophy ceremony. did marcello ever admit he'd been defeated? course not. he's a class-one asshole who'll never admit he is wrong. did anyone here on this list ever admit they were wrong? course not. way too many assholes here to admit that... indeed, every once in a while, the wolf-pack still tries to mount an attack. but they're impotent -- merely flaccid. does that change that fact that i was right? not one bit. does that change the fact that i pioneered the idea here? no it doesn't. so, as much as you'd like to rewrite history, there ain't nobody here who's gonna buy your recycled lies. so if you ever want the proof, just go look at the archives. -bowerbird p.s. the delicious irony is that, for project gutenberg needs, restructured-text is still twice as complicated as it could be, and, given marcello's "charisma", thus fated to be albatross. fly, albatross, fly! i don't know why this albatross won't fly... p.p.s. for irony dessert, not that andrew can't spell "mean". he can _be_ mean, but can't spell it. what does that mean?

2 1

Re: [gutvol-d] book of james -- 016
by Bowerbird＠aol.com 20 Jan '12

20 Jan '12

james said: > The URLs ending with .py are not Python programs. well, technically, a python program lives at that address, but what _you+ will see, in your browser-window, will be the output which that python program is sending to you. > The "dummy" file does seem to be > doing the "compose+'+S" correctly ok, that's interesting, on some level... but you'll still need to fix the 777 file. > and I do have to change the browser to UTF-8 to see it. i suspect that, since it's labeled as a .txt file, it's being sent out as ascii, rather than utf8... probably some switch i can toggle at my i.s.p. > If you want me to "think like a programmer" > then at a minimum I'd want to have: i want you to think like a d.i.y. programmer who cannot require things "at a minimum"... :+) > 1). The file containing diacritics > that you used to extract the words. there is no such file. i read in your text-file, and extracted any word containing a diacritic. for the record, there's under a dozen diacritics. > 2). The Python program(s) you used > to extract them and make this list. i wrote that program in basic, not python, so you'll need to settle for english pseudo-code, which is what i just told you, back at point 1). > 3). The Python program you wrote > to make the word substitutions > and put the diacritics back in. that wasn't a python program, not originally, just a macro of global changes in a text-editor. easy enough to do it in basic, though, or python, which is what i did to make "waxon" and "waxoff". off the top, so untested and maybe even wrong: > s=thebookfile > swi=switchfile.split("\n") > for x in range (0,len(swi)): > dan=swi[x].split(" -- ") > dia=dan[0].strip() > non=dan[1].strip() > s=re.sub(non,dia,s) c'mon, james, that's a 7-line routine. you can d.i.y. > I do have your file with misspelled words by page. that list is now outdated. > Many of the words in the diacritical file are familiar enough > to me that I won't need to look them up. but some are not, i take it. won't hurt to check _all_ of 'em against a scan, especially if you just check a few occurrences... that routine might run about 25 lines, james. do you want me to write that one for you too? > My only concern is the "Compose+'+s" characters. all you need to do is enter it correctly in the file you edit. > I just wonder how they got that way to begin with well, we know it happened in my hands. so who knows? it could have happened any number of ways, all of which are now immaterial. don't get hung up on the irrelevant. just take the "777" .html file, edit it appropriately, and then re-mount it somewhere only where i can get to it... -bowerbird

3 2

Re: [gutvol-d] book of james -- 016
by Bowerbird＠aol.com 19 Jan '12

19 Jan '12

james said: > One thing I spotted was the use of i don't need details. just show me the utf8. :+) > However, I thought you should be aware that > your word extraction program is doing this > and it is wrong. i know it's wrong. that's the whole point. don't explain. show me the right version. > I have a thought on looking up the words. > PDFs and DjVus from archive.org > have text contained in them. > I should be able to put in > a questionable word from the right column > and see what it should be on the left, > then fix it. you're a programmer, right? start thinking like one. i don't know exactly what you mean by "put in a questionable word", but it sounds uncomfortably _manual_. ditto with doing "find" in a .pdf or .djvu. you have a list of the bad words in a file. and you have the actual e-book in a file. with pagenumbers pointing to the scans. so... think like a programmer, and write code that _automates_ the process for you, so you just have to click a button or two and maybe -- in the extreme case -- edit text in a text-field by using your (ick) keyboard. think like a programmer. i _will_ repeat this. if i _have_to_ repeat it. but james, i don't want to have to repeat it... -bowerbird

2 1

Re: [gutvol-d] the idea of using it
by Bowerbird＠aol.com 19 Jan '12

19 Jan '12

i said: > p.p.s. for irony dessert, not that andrew can't spell "mean". and -- for your after-dinner cognac -- note that bowerbird can't spell "note"... not that it matters... ;+) -bowerbird

1 0

book of james -- 016
by Bowerbird＠aol.com 19 Jan '12

19 Jan '12

ok, james, it's nose-to-the-grindstone time... :+) today we attack your diacritics, most of 'em on names... *** james said: > You list of words and names have no accents > or diacritical marks, and so does the HTML. > Even with global search and replace > that's not going to be fun to put back. there must be something you don't understand... but that's ok. that's why i'm here... :+) in a nutshell, changing between versions of the file which _do_ or _do_not_ have the diacritics is merely a matter of converting all of 'em with a single script. or, if we arrange it another way, clicking a button... click it once to replace non-diacritics with diacritics, and click again to change diacritics to non-diacritics. back. and forth. now you see 'em; now you don't. i won't try to make the claim that clicking a button is "fun", but i'd certainly dispute any claim it's "difficult", which is what i believe you were trying to imply above. maybe you thought you'd be doing 'em one-at-a-time? think again. we're doing things the easy way now... *** first, take a good long look at the diacritics you did: > http://zenmagiclove.com/bhaga/diacritics777.html i pulled those out of your edited version of the file... *** next, here's the demo code for changing to diacritics: > http://zenmagiclove.com/bhaga/waxon.py *** and likewise, the demo code for removing diacritics: > http://zenmagiclove.com/bhaga/waxoff.py *** notice that, to develop the script, i had this demo code actually perform its conversion _on_ the file containing the _data_ which it uses, the file containing the words... you can find that dummy file here: > http://zenmagiclove.com/bhaga/waxer.txt (if that file doesn't display properly, you will need to force your web-browser to recognize the file as utf8.) *** of course, eventually, we'll have it use the _real_ file: > http://zenmagiclove.com/bhaga/diacritics777.html but first, james, you're gonna need to clean that file... there are a couple of different ways you could do that, and i don't know which of them will be easiest for you. can you recognize the correct version of each word by just looking at it? or do you need to look at the scans? if you can do it by just looking at the list, that's great. if you need to look at scans, i'll code a routine for you. *** but... either way, before you start, let's look at the file... (an important note's at the end of this, so don't skip it.) there are some 777 words on it, give or take a few, but i'd guess a couple hundred of them are _errors_, for instance, up top, we have two similar ones: > Akuti > Akriti those might be two different names. or one could be a scanno for the other, in which case it must be fixed. there are lots of these cases. further down, we have: > Vyasa > Vyana again, could be two names. or one could be a scanno. pay attention, as some of them look very close: > Surasa > Surama ... > Sinivati > Sinivali ... > Sinhaya > Sinhaka ... > Sayujya > Saynjya ... > Satarupa > Satarapa *** there are 25 cases that particularly need attention. you can locate these cases by searching for "yyy"... for instance, here's the top one: > Avirmukhiyyy > Avirmukhizzz the "yyy" and "zzz" were added by me, as markers... in such cases, you'll see that the non-diacritic versions are exactly the same (or would be, without my markers). however, the diacritic versions _differ_, in some way... i take it that these are _not_ different names, but rather that you did their diacritics differently. perhaps that is because you made a mistake, or maybe the p-book had it different, maybe because they made a mistake, or... but whatever the case, if these names are supposed to be _the_same_person_, then they should be made the same... (unless i misunderstand something about diacritics, which is totally a possibility, as i claim to know zilch about them.) *** one last thing, james, and it's extremely important to note. when you make these corrections, you must do the edits to the _diacritic_ side of the pair, not the non-diacritic side... (the non-diacritic side of the pair is what's in the file now.) however... although the _edit_ must be made to the diacritic _side_, the actual edit _can_ be the correct non-diacritic version. this will work because we'll be running the converter twice. so the first time through, the incorrect non-diacritic will be changed to the correct non-diacritic. then in the second run, the correct(ed) non-diacritic is changed to the correct diacritic. if you have questions about this, feel free to backchannel me... -bowerbird

2 1

that "blue p.d.a." cover will go down in e-book history some day
by Bowerbird＠aol.com 19 Jan '12

19 Jan '12

a generic cover is one thing. many places create a "cover" for e-books which is just text giving the name of the book, and maybe the author as well. kind of like the generic products at the store, labeled as "corn", or "cereal", or "beans", or "soup", or "bean soup". it's not all that pretty, but it works. i mean, really, "alice's adventures in wonderland" _should_ have a psychedelic cover, but "alice in wonderland" works... at least we can tell it apart from "war and peace" or "flatland". so yeah, a generic cover is one thing. *** but project gutenberg e-books? well, they take "generic cover" to a whole new level, baby... because _every_darn_book_ gets the _same_ generic cover! it's as if every product in the store got the _same_ label -- so that _everything_ got labeled as "supermarket product". imagine if peanuts were labeled as "supermarket product", and butter was also labeled as "supermarket product", and _peanut_butter_ was also labeled as "supermarket product". make it pretty hard to tell 'em apart, wouldn't it? but every book in the p.g. library gets the exact same cover. it's as if every book is given a cover that says "book". : +) alice in wonderland? "book" war and peace? "book" flatland? "book" because, you know, they're pretty much interchangeable, right? if you have read one of them, you've read them all. so let's give them all the same cover -- "book"... except... it would be one thing if it was an actual image of a _book._ but no! no sir! that'd be too obvious... too skeuomorphic. so project gutenberg uses a picture of a _p.d.a._ and not just _any_ p.d.a., mind you, but one that looks to be right out of 1993. you'd think it would be hard to tell, from a mere image, but this graphic actually _screams_ "clunky"... you can tell this was an early p.d.a., one of the ones that was big, and _heavy_, and cost an arm and a leg when it was new. it probably had 640k of memory (because who needs more?), and i bet it could run _a_full_3_hours_on_a_single_charge_... i guess project gutenberg wants a generic cover that will say "old school" -- befitting its status as e-book pioneer -- and this graphic does that, baby, it does that _in_spades_... because nothing says "e-book" like a picture of a p.d.a. *** yep, that "blue p.d.a." image will hang in a museum someday. so i hope you appreciate its historic significance _right_now_... -bowerbird

2 1

General question on download totals
by James Simmons 19 Jan '12

19 Jan '12

I don't know who to ask this but I'm puzzled about how download totals work on the PG site. Are they cumulative, that is every download since the book was posted? Or are they something else? I ask this because I check the totals for books I've donated and the current numbers are MUCH lower than I remember them being. Ancient Manners, for instance, was more popular than War And Peace the first week it was available. Now it shows 125 downloads. It got many more than that the first day! I'm just puzzled. I use the download counts as motivation and now I don't know if I should trust them. James Simmons

3 2

Re: [gutvol-d] book of james -- 014
by Bowerbird＠aol.com 18 Jan '12

18 Jan '12

james said: > if I seem a little dense if you seemed "dense", i wouldn't bother... > it's because you reveal what you're doing > in bits and pieces, hints, and old email threads it's a big ocean that i'm boiling here, james... be patient, and drink one cup of tea at a time. > I assumed that the page breaks were going away > until I saw your work product > where the TOC actually had page numbers > and linked to those pages. > That made me wonder. ok, let's use this as a teachable moment. (it's wonderful that you're a good learner, since i always end up being pedagogical.) first of all, nothing in a _work-product_ will necessarily be in the _final_ product. so that "wonder" shouldn't even come up. indeed, what you _should_ be thinking is "why is this aspect in this work-product?" and "what need is it serving in this case?" second, and of an enduring importance, is the fact that _navigation_ is one of the essential qualities of an electronic-book. by "navigation", i mean 3 specific things: 1. knowing where you are in the e-book. 2. knowing exactly where you want to go. 3. being able to go there quickly and easily. pagenumbers help to keep you grounded; they make it easy to know where you are... they also often make it easy to know where you want to go. they serve as sign-posts... in the electronic text environment, _links_ are the things that make it easy to _jump_ -- i.e., go somewhere quickly and easily... so navigation is why we keep pagenumbers in our "master" file -- because they are _so_ essential in our editing -- and often even in an output-format created from that master. (but only if they won't interfere with usage. still, we keep 'em up until the last moment.) navigation is crucial in a work-product too. which is why i included the links in that file. they are tremendously useful, at times, and you can just ignore 'em the rest of the time. another thing that you can note about this is that pagenumbers help us to formulate _an_intelligent_file-naming_scheme_to_use. notice that i use the prefix of "bhaga" on all of the files associated with this book... (that's what defines it as a coherent entity within my system, which has many books; that 5-letter label also names its _folder_.) after "bhaga" is "f" or "p", depending on if it's a forwardmatter page or a regular page -- i.e., either roman numerals or arabic -- _then_ the pagenumber (padded to 3 digits). so the scan for page 123 is "bhagap123.jpg". and the .html filename is "bhagap123.html". the text filename would be "bhagap123.txt". and the html anchor is likewise "bhagap123". this intelligent naming-scheme means that the book has a dependable predictability, so it's easy to create .html anchors, and links, and to call images, and so on... thus, those pagenumbers in a p-book table-of-contents can be _effortlessly_ turned into .html links, with a minimum of fuss or muss or coding. it's like falling off a log. so it's something that's simultaneously powerful and simple, great navigability at practically zero cost... so the question is: "why _wouldn't_ i do it?" and the answer: "i'd be stupid not to do it." all the links in that huge table-of-contents? i accomplished them with two lines of code: the first one created each of the anchors... the second made the links to those anchors. and did you catch the results of a third line? each of the page-separators is a _link_ that will load the image-scan for that page into a _named_ window, which also means that subsequent clicks on other page-separators load their scan in that same named window. so if you wanted to refer to the page-image to resolve any of those paragraph questions, it would have been extremely easy to do so. this is a little trick which you and i will use later, james, so make sure you play with it. and, again, all of that with one line of code. so these are great examples of the way that it can be fun and easy to code books which are powerful, fast, easy, functional, and fun. and _that's_ what you should be noticing, james, not complaining about "these links which i do not want". there's all kinds of gold injected into the samples i give you, and you should really be panning for it... not all of the nuggets will be as big and obvious and easy to explain as these, but ounce for ounce, they're just as valuable... *** that's page _numbers_. page _breaks_ are another animal. they are useful less often, so most of the time we will not retain them. (but again, we keep 'em as long as we can.) it's probably also important to note that we have a way to "rejoin" paragraphs even while retaining the pagebreaks and pagenumbers. (for instance, that's important when we do the check for balanced doublequote-marks, where the paragraphing exerts an impact.) > A design is beginning to emerge > from everything you've told me. > Looking at your ZML file helped. good. glad to hear that. :+) you have to also realize that there are hundreds of subscribers to this listserve, and you're just a vehicle for this thread, where i'm actually talking to all of them, not just to you individually... so i have to make sure this is still interesting to them, even though they've heard the whole story. (well, at least they _believe_ that they have.) *** the other thing is, i love life. i laugh a lot. i'm always smiling when i write these posts. i have a good time. i enjoy the interaction. so i'm having fun. even when i pretend to complain... (although it does get tiresome when i must repeat myself too frequently!) you don't have to fret over offending me... -bowerbird

1 0

book of james -- 015
by Bowerbird＠aol.com 18 Jan '12

18 Jan '12

james- i wanna make sure that you and i don't get sidetracked by this discussion that's going on about guiguts and .epub... i think it's important to stay focused because -- for one -- you recommended guiguts in your "enlightenment" book, and -- for another -- you've said here that _you_ use it... let me say, right at the outset, that i do respect guiguts... thundergnat -- the original programmer -- did a service for distributed proofreaders that very few will ever match. people who've worked on it since steve, including hunter, have helped as well; guiguts has been invaluable to many. and i mean all of that praise quite sincerely, i really do... at the same time, however, guiguts is long in the tooth... and it never was too user-friendly; it's a _bear_ to install. but the worst thing about it is that it was molded out of the d.p. workflow, which is _not_ something to emulate. so unless you're working inside d.p. -- and maybe even not then -- you definitely do _not_ want to use guiguts. still... as far as the current _discussion_ goes, join in! hunter is _asking_ for advice, so give him some, and he might just be able to improve the resultant output. i'm not too hopeful about that, for reasons i'll append, but i'm not opposed to change, and it could happen... but even if guiguts does "improve", in terms of input for the .epub converter, you still won't want to use it. all of the useful things it does -- and it does many -- are things that you can write your own scripts to do. and scripts you write will work much better, for you. this _specifically_ includes the text-to-html routine... you might or might not decide to adopt my .zml, but even if you don't, i can _guarantee_ that you will have seen enough of the light to realize you can d.i.y. this. so join the fun of the current discussion, and see if you can learn anything from it, but understand that -- in the long run -- guiguts will not matter to you. -bowerbird p.s. so why i am not optimistic about this discussion? 1. we discussed it a million times; it never mattered. 2. marcello doesn't care a whit what anybody thinks. 3. the d.p. postprocessors don't care what you think. 4. postprocessors got recruited as snowflake makers. 5. everyone wants the e-books made to _their_ taste. 6. too many cooks spoil the soup; so then it's spoiled. 7. 3 d.p. committees worked on this; they all gave up. 8. .epub is a moving target; tomorrow, apple moves it. 9. it's impossible for a single .epub to please everybody. 10. end-users will soon demand an ability to customize. so... have fun. but i've got better things to do... :+)

2 1

Re: [gutvol-d] book of james -- 014
by Bowerbird＠aol.com 18 Jan '12

18 Jan '12

james said: > I just wanted to be certain that > Bowerbird had some plan to do that. thanks for giving me a chance to prove i'm not stupid. -bowerbird p.s. i have studied e-books _intensely_ for 3 decades. if something is bloody obvious, you can probably just assume, safely, that i have stumbled across it before...

2 1