- gutvol-d - lists.pglaf.org

giants passing
by Bowerbird＠aol.com 06 Oct '11

06 Oct '11

first michael hart. and then one month later, steve jobs. so sad... i've appended the article that recalls my fondest image of michael. -bowerbird ========================================= Hart of the Gutenberg Galaxy by Denise Hamilton Wired magazine, Issue 5.02 -- February 1997 What kind of a man wants to put the 10,000 most important books online by 2002 and make them available for free? (Hint: the kind of man who puts sugar on his pizza.) I am sitting with Michael Hart at Garcia's, a pizza place near the University of Illinois at Champaign-Urbana, where we are having a perfectly normal conversation about digital libraries and are preparing to tuck into our dinners. Hart is an e-text visionary and cofounder of Project Gutenberg, whose aim is to put copies of the world's greatest books on the Net for free. Hart is also a world-class eccentric with the social skills of a cranky 2-year-old. As I look up from my glistening pepperoni, Hart, who is 50, does the damnedest thing I've seen done to a pizza. Tearing open two dozen sugar packets, he sprinkles them methodically over his deep-dish pie until it is layered with white crystals. Then he digs in. "This may seem weird to you," he confides between mouthfuls, "but it's the only way I can get enough fuel to keep working. This right here is 2,000 calories, and it should keep me going for a while." Then the sugar begins coursing through his system, and Hart takes on the bug-eyed look of someone who has just hoovered up a few fat lines of Peruvian flake. The words tumble out. Sweat beads his upper lip. "Aaaah," he says, leaning back in the wooden booth. "I love working till I drop. I love looking back on the day and knowing I put another book online. That's what I'm here for. If I feel like I'm crashing, I eat. If that doesn't work, I sleep. I live by the nanosecond and burn the candle at both ends. With a blowtorch." For more than a quarter century -- eons, in Net time -- Hart has made Project Gutenberg his mission in life. Or, more accurately, his fervid, manic obsession. While other geeks have grown rich mining the silicon motherlode, he's been sitting like a troglodyte in his Urbana basement, tending a utopian vision named for the 15th-century German printer whose amazing invention first made books accessible to the masses. As the 20th century wanes, Hart is using the Internet to try to finish what Johann Gutenberg started, pumping into cyberspace digitized copies of Rebecca of Sunnybrook Farm, Moby Dick, and Orlando Furioso and the works of Henry James, Plutarch, and Dostoyevsky -- basically, any and everything that's ever been published between hard covers. It's one of civilization's oldest dreams: the universal library. And in an era of ever slicker Web graphics, blinking GIFs, and Java-powered billboards, he's doing it with militantly plain ASCII text files -- just the words, please -- downloadable anytime anywhere in the world by anyone with a functioning computer, a phone line, and a modem. No ads, no credit card numbers, no charge. Even by the Net's generous standards, Hart is a wild romantic. He refuses to keep usage logs. He's not a big fan of copyright extension laws. He considers the Web an appalling waste of bandwidth -- Gutenberg's main files reside on an FTP site. What passes for organization is a loose network of volunteer typists, scanners, editors, and proofreaders, plus a couple of pro bono attorneys, all held together by email. Other than that, Hart operates with something pretty close to no strings attached -- political, institutional, or otherwise. Not that a lot of the groups that share his aims would have him. Since Hart first typed the American Declaration of Independence into a U-of-I mainframe 26 years ago, Gutenberg has put about 1,000 titles online, on a budget smaller than some big-time electronic publishers allocate for market research. His self-imposed goal is to reach 10,000 books by the end of 2001, Gutenberg's 30th anniversary. And with 9,000 to go, the questions rise like the greasy steam from our pizzas: Is Hart a deluded crackpot, a high tech saint, or just another Net eccentric? And what is a project with seriously historic ramifications doing in the hands of someone who puts sugar on his pizza? To find out, I'm pulling up outside the big, century-old, ivy-covered brick house that Hart bought years ago with a modest inheritance. In my honor, he's wearing slacks but quickly changes into the uniform he will sport for the rest of my three-day visit: black bicycle shorts that emphasize his pot belly, a red T-shirt, a baseball cap, and Nike Air Jordans two sizes too big that cost US$1 at a garage sale. Beneath the cap is short, thinning hair. He has a booming laugh and a large, clean-shaven fleshy face with graying sideburns. When I ask to use the bathroom, Hart gestures into the basement, past peeling plaster and ancient caulking. "Don't bother flushing," he hollers down. "I've got a garden hose, and I'll just run some water through it." I begin to realize that this is a man with clearly defined priorities. "Other than to redesign democracy, I can't think of anything more important than Gutenberg," Hart tells me when I return. "It's the Archimedes lever: Give me a place to stand, and I'll move the world. Well, that's what I'm doing. Our goal is to give away 1 trillion e-text files by December 31, 2001. That is 10,000 titles each to 100 million readers, which is only 10 percent of the present number of computer users. Since that number will double by 2001, we will meet our goal if only 5 percent of all users download a file." Easy as that. Hart calls himself an electronic Johnny Appleseed, sowing the Internet with books for the global proletariat. He foresees the day when, armed with the trashiest laptop and a modem, tribesmen in Borneo's rain forest will be able to click onto Gutenberg and download texts. But why would anyone want to read a book on a computer when they can hold a bound copy in their hands, I ask. Because it's fast, convenient, and free, Hart shoots back, along with a list of the people who are using Project Gutenberg already: Kids doing research for classes. People in foreign countries who want to practice English. Grandmothers turning their computer-savvy grandkids on to The Jungle Book. Folks looking for rare Robert Louis Stevenson volumes -- Gutenberg has the author's entire 30-plus volume oeuvre -- not carried by their local library or bookstore. And people like Jo Churcher of Toronto, who is blind. She downloads Gutenberg texts and runs them through a speech synthesizer that reads them out loud, like books-on-tape, only free. "Gutenberg has finally allowed blind people to begin building up libraries of their own," says Churcher, who likes the project so much she has helped scan in 12 books, including The Pickwick Papers, for Gutenberg's files. We move into Hart's living room, a dimly lit cavern piled with modems, computer cards, hard drives, videocards, and power supplies, all leaning drunkenly against each other. There are hundreds of boxes of software; thousands of records; CD-ROMs and CDs; moth-bitten beaver, mink, and monkey coats; hippie op art; what Hart says is an original Modigliani; a gaily painted child's hobbyhorse; a Cable piano from the 1910s; a Thomas Edison Dictaphone from the '30s; and rooms filled with books, including 50 complete dictionaries, three copies of Chaos by James Gleick, and nine of Mistral's Daughter by Judith Krantz. "She's usually a ditz," he explains, "but this was a good book." Hart is a garage sale fanatic, hitting up to 100 each weekend as he criss-crosses the leafy suburbs of Champaign-Urbana on a bicycle (he doesn't have a car). He also forages daily among neighborhood dumpsters, where U-of-I students routinely discard stereo components and new floppy disks. But his favorite dumpster is located behind the U-of-I computer lab. We drive there, and Hart hops right in, wallowing thigh-deep amid pizza cartons and plastic milk bottles because he knows the school often throws out old equipment, stuff he can use for Gutenberg or trade at the weekly "geek lunch" he attends with other tech-heads in the U-of-I orbit. Today, he fishes out a set of Unix manuals, an 8mm tape cartridge, and two copies of Mathematica, for Sun's Solaris operating system. Slim pickings. Nothing like the time he scored an entire ATT 7300 minicomputer. Hart doesn't need high-end stuff to run Gutenberg -- it's just nice to have around. Every few years, he gets a computer from Apple, NeXt, IBM, or Hewlett-Packard. Bell &AMP Howell once donated a $50,000 scanner, to help volunteers input books quickly. And he's thinking about setting up an email server, so he can offer free accounts to Gutenberg volunteers. Just talking about the idea gets him giggling with glee. As a child, Hart's idols were Peter Pan and Albert Einstein. In adulthood, the two have twined like a double helix to define his personality so that within one conversation, he can go from visionary brilliance to bluster to bumble. "Michael is one of the smartest guys I've met in my years," says Greg Newby, an assistant professor and assistant dean at the School of Library and Information Science at the U-of-I and a senior research scientist at the National Center for Supercomputing Applications. "He's also one of the least mature. That makes for an interesting combination." Hart grew up in Tacoma, Washington, where his parents worked as government codebreakers during World War II. In peacetime, his father was a Shakespeare professor and a CPA. His mother was a math and education professor and ran a women's clothing store. Dyslexic but precocious, Hart was messing with algorithms while other kids were fumbling with Lego. He plowed through the U-of-I in two years, graduating first in his class with a self-created major in man-machine interfaces. Hanging around the university's computer lab after graduation, he dreamed of a universal digital library and posted an e-text manifesto that grew into Project Gutenberg. With the US bicentennial looming, he chose the Declaration of Independence as his inaugural text -- he also happened to have a copy of the text in his backpack. Next he did the Bill of Rights, then the Gettysburg Address. But the project moved sluggishly because Hart inputted everything manually after his day job of selling stereos. In those early mainframe days, Hart had to operate within the impracticable bounds of a 10-Kbyte donated storage space. But as computers got faster and smaller, Gutenberg began to look more like a real possibility. Things really broke loose in 1988, when Hart typed in his first full novel, Alice in Wonderland. Mark Zinzow, a senior research programmer at the U-of-I who met Hart that year, remembers thinking that the project -- which at the time had 10 books online and a 1200-baud connection -- was completely harebrained. "But it was also a noble goal," says Zinzow, "and I thought, 'If he wants to tilt at windmills, I'll help him get on his horse.'" Although Hart no longer had a formal affiliation with the university, Zinzow gave him access to email and mailing lists, and more important, an FTP server. Soon, he was watching in amazement as Gutenberg pumped out a gig of data each day. Around the same time, Hart was invited to place his digital collection under the official auspices of the Benedictine University, a tiny Roman Catholic seminary in nearby Lisle, Illinois. The symbolism did not escape Hart: monasteries had once been repositories of knowledge in the Dark Ages, copying and preserving books for posterity. Wasn't Gutenberg doing the same thing electronically, preserving e-texts long after paper and microfilm would crumble into dust? The monks went even further, naming Hart an adjunct professor of electronic text and giving him a $12,000 annual salary. They kicked in some extra money for living expenses, most of which eventually came from sales of a Gutenberg CD-ROM that has hit 100,000 copies. (Regularly updated, the current edition boasts more than 500 titles, on a single disc.) As the project grew, Hart's management style stayed laid-back. He refused to draw up a master list of books for the Gutenberg pantheon, preferring to let volunteers input their favorite tomes. Initially, Gutenberg got the usual suspects -- the Bible, Virgil's Aeneid (in English and Latin), and Hamlet. Then came more unusual fare: The Book of Mormon, Herland (a 19th-century feminist novel), and Flatland (science fiction, about 4-D travel). Deadlines were flexible. There was no follow-up. Sometimes the e-texts arrived months or even years late. Other times, they just dropped into limbo. What keeps things going is Hart's fanaticism. A DOS man and a neo-Luddite when it comes to GUI interfaces, he says he's never even used the World Wide Web -- he doesn't like the graphics. But the five machines that make up Gutenberg Central boot so fast you can barely see the start-up box flash by. And when a fresh new book from a volunteer comes in on a winter night, he'll wake at 3 a.m. and sit before the glowing screens in a sweatshirt resembling a hooded monk's cowl he got from the Benedictines. In summer, he works in nothing but biker shorts, blasting classical music or classic rock. To get his Vitamin D and avoid rickets, Hart relies on a full-spectrum lamp that mimics the sun's rays. Nearby is a rumpled mattress where he sleeps when felled by exhaustion. When I visited, he was proofing The Violet Fairy Book -- a 1901 collection of stories edited by Andrew Lang -- and checking that the format, spacing, and margins conform to official Gutenberg style. After scanning for errors, he adds a header -- "Welcome to the World of Plain Vanilla Electronic Texts. Readable By Both Humans and Computers, Since 1971" -- includes some boilerplate legal information, and updates his directory. Then he hits Enter. Bing -- the book is in an FTP site on PrairieNet, a community access computer system in the Midwest. From there, it will ripple around the world, to interested individuals, literary Web sites, and other digital libraries. Hart has to keep working, or he'll drown in data. Each day, he receives up to 400 pieces of email, chats with dozens of volunteers, and works on upcoming books. Besides running Gutenberg, he's also the central figure behind "Ask Dr. Internet," a free service run by a Gutenberg-like group of tech-heads that has evolved into its own full-time job. Even the most clueless AOL newbie gets an answer -- often with a liberal icing of Hart's own curious views about things like the Web and graphical interfaces. With Gutenberg, he gets help from 750 volunteers around the world. Lawyers work pro bono, researching a book's copyright to ensure it has entered the public domain. Tech-heads like Zinzow provide sysadmin consulting and tend computers held together with gum and baling wire. Scholars input and proofread e-texts. A group of 50 Russian academics, for instance, recently did Webster's Unabridged Dictionary by hand. The 45 million keystrokes took them six months, for which they were paid $5,000 by one of Hart's financial supporters. Some texts are labors of love, by volunteers who tap along for years to finish one title. But most of the heavy lifting is done by a hardcore group -- mainly academics -- with access to state-of-the-art laser scanners. Geoffrey Pawlicki, a longtime supporter who put Shakespeare's Antony and Cleopatra online, recalls meeting Hart in 1980: "He had files in a backpack and was always running around hooking people up with modems. Initially he was dismissed as a crackpot, but the same could be said of Ted Turner." Not that Project Gutenberg's founder and a media mogul are likely to ever be confused -- for starters, Hart's totally uninterested in tracking Gutenberg usage. "I don't care where a book goes," he says, "I just want it to sprout legs and run." He knows that 10,000 files are downloaded daily on the U-of-I server, but that doesn't give the whole picture since the archives are mirrored on hundreds of sites the world over and heavily redistributed. (The New Zealand Digital Library, for instance, boasts 492 Gutenberg titles in easy-to-use HTML.) Many users post Gutenberg texts on their own Web sites, so Hart's work can be found on Tarzan sites, Shakespeare sites, and The Jungle Book sites, to name just a few. The only thing Hart asks is that anyone who uses a Gutenberg text tack on a "small print" header that says in part: "Why is this 'Small Print!' statement here? You know: lawyers. They tell us you might sue us if there is something wrong with your copy of this e-text, even if you got it for free from someone other than us, and even if what's wrong is not our fault. So, among other things, this "Small Print!" statement disclaims most of our liability to you. It also tells you how you can distribute copies of this e-text if you want to." One result is that Gutenberg texts reach people who have no idea what an FTP server is, let alone how to use one. Hart gets a lot of email from the United States, Britain, Canada, Singapore, and Germany, so he assumes those are big markets. Many people comment on the CIA World Facts Book, so he thinks that's his most popular book. (It's updated annually, and he can put it online immediately because government publications are public domain.) Other favorites seem to be the Bible, Alice in Wonderland, and the collected works of Shakespeare. But even broaching the idea of doing more to find out what people might want from Project Gutenberg makes Hart see red. He imagines the sinister implications of tracking folks who want to download, say, Salman Rushdie's The Satanic Verses in a country like Iran. And besides, any effort to "steer'' the project misses an essential point. "Once we put a book out," he says, "it goes everywhere there are computers and readers. That's unlimited distribution. And in a world based on competition for everything, that's the biggest threat." So in one sense, Hart is an electronic David, striking a literary blow against an establishment Goliath that tries to control information through restrictive copyright law, downloading fees, and red tape. And for that, he has a lot of fans. "Michael's one of the few people I know who's not motivated by greed," says Zinzow. "He's trying real hard to do a good deed for the world, and he's really underappreciated. If Michael wasn't here, access to books on the Internet would be only for the rich. He's an electronic Robin Hood, keeping the Sheriff of Nottingham from creating a monopoly." Newby agrees. "When you have Microsoft buying up the right to thousands of art images, and the federal government taking away copyright, going backward and copyrighting things that weren't copyrighted before, it's not just paranoia; he's trying to fight the evil forces." Hart is happy to agree. "Some people say, 'I am the most powerful because I have the most power.' I say, 'I am the most powerful because I give the most power away.'" And there's no reason to doubt that his dogged pursuit of that ideal is one of the things that has enabled Gutenberg to survive all these years. The project has doubled each year since 1991, when he had 12 books online, and the way Hart sees it, Gutenberg should continue to do so. That means 1,600 books for 1997, 3,200 for 1998, and so on. But there are limits to what he can do without institutional backing and only a handful of hardcore volunteers. Even supporters like Newby say the man can barely handle his current workload and will be hard-pressed to take on more unless he delegates authority. Lately Hart's been trolling the Internet for someone to pass the torch to.But it's hard to imagine him letting go. "No one wants to be a supervisor, so I have to do it all myself," he says plaintively. Hart also refuses to waver from his initial vision, and this has limited his scope. If he made some concessions, it's possible Gutenberg might have 100,000 books online today -- a respectable library -- instead of 1,000. Indeed, Hart says, various academic institutions and even some Texas oil interests have offered to bankroll Gutenberg over the years, in exchange for control. One university offered him a six-figure salary, he says, to bring the project to their campus. He turned them all down flat. "Almost everybody out there wants to charge for books, and they want real control over which books we do and which edition comes out," Hart grouses. "They want a bit in my mouth. I don't trust them." He's right to worry. In recent months, complaints about Gutenberg's use of over-taxed U-of-I computing resources led to an ultimatum from school officials: find a formal university sponsor or get off. Hart tried the library, the School of Library and Information Science, and a computer network run jointly by the Big 10 football schools. No luck. "The library, which would be the logical place, wasn't interested in sponsoring it -- they might have been concerned about the project's academic credentials," says Bob Penka, an associate director at the U of I's Computing and Communication Services office. "Whether that's snootiness or what, I don't know." Hart has lost his U-of-I email account, but so far the project is getting a stay of execution -- at least until the next round of complaints. Is Gutenberg too important to be left to a solitary eccentric? Maybe it should be organized by the Library of Congress, the National Endowment for the Humanities, or some such organization. But herein lies another problem. Hart isn't the only one staking out literary dibs in cyberspace. The Georgetown Center for Text and Technology counts more than 300 online library projects in nearly 30 countries, including the Dartmouth Dante Project, with 600 years of commentary on Dante's Divine Comedy, and the Oxford Text Initiative, which charges users to download its scholarly publications. And for all its high-minded goals, Gutenberg's simplicity works against it. It lacks the bells and whistles, the flashy graphics and sophistication that grant-givers love. Instead, Hart plods along in ASCII, doesn't give a hoot about market research, and wants to give his product away. The lack of recognition rankles. "There are literally a billion dollars of grant money out there, and I'm never going to get any of it," he says. And those 300 competitors? "All those projects will never produce even one mainstream book that you or I will ever see." Digital library experts say he's got a point. "Funders tend to be interested in projects that use state-of-the-art technology or advance knowledge, or that create something new and exciting," says Ann Bishop,a coprincipal investigator with the U of I's Digital Library Initiative project, which is working with commercial publishers to put academic journals online. "Projects that aren't high tech tend to get lost in the shuffle. Perhaps if he played it up as something vital to national education, he would get more funding. But I don't know if he's interested in twisting it that way." Ultimately, Hart's biggest impediment may be US copyright law, which pretty much prohibits Gutenberg from publishing anything written after 1920. The law now protects a work for 50 years after the author's death, which means that Hemingway, Genet, and Garcia Marquez won't be online anytime soon. Even The Odyssey and Plato's Republic are off-limits if Gutenberg wants to post a translation published after 1920. One exception is when an author gives special permission, such as cyberpunk author Bruce Sterling, who approached Hart to release an e-text of his novel Hacker Crackdown. The copyright problem isn't getting any easier. The first US copyright law covered 14 years, with a possible 14-year extension. A 1909 amendment doubled that to 28 years, and in 1976, the law was extended again. Now Congress is considering going to life plus 70 years. "Do you realize that under the proposed law, the blueprint for the Wright Brothers' airplane would still be under patent?" asks an incensed Hart, who has testified in Washington against the new bill's passage. "If they win, we'll have to get a certain number of titles and then quit. Unless we want to do the Robin Hood thing. And I'm too old to be a revolutionary." A glint appears in his eye. "But I'll do Gone with the Wind on my deathbed," he vows. "The book and the movie. It should be out there." I leave Urbana feeling that Hart just might meet his short-term goal. If he were any less obsessed, he would have given up a long time ago. Instead, he is doubling each year. But I also wonder how long the project can keep expanding exponentially. Unless Hart can draft reinforcements or hook up with a sponsor, eventually it probably will stall, and that's a shame. As the plane takes off, I recall Hart's description of the mad rush that comes from sitting in his basement, watching the newly transcribed titles come in, working on them, and then launching books into cyberspace. His words echo in my head: "Bennett Cerf never published a book a day, but I'm doing it. Do you have any idea of how powerful that makes me feel? But sometimes I go to bed at night, and I don't know what I'm going to do in the morning because no one's sent me anything. That's kind of scary." ----------------------------------------------------------------- Denise Hamilton is an obsessive reader who writes regularly about business and culture for the Los Angeles Times, New Times, and other publications. ----------------------------------------------------------------- this article was found at: http://www.vivanet.com/~akp/index.html akpgsh(a)rit.edu or akp(a)vivanet.com "All that is gold does not glitter, Not all those who wander are lost." -- J.R.R. Tolkien

1 0

Re: [gutvol-d] do you want to look backward? or move forward?
by Bowerbird＠aol.com 05 Oct '11

05 Oct '11

ricky said: > I happen to disagree, but > love to hear the counter view the shelf-life on this topic expired about 10 years ago... the next step in that little dance was the question asking "what about the deluge of content that will be inevitable?", the answer to which was "collaborative filtering systems". then there were all the user-generated-content systems, the ultimate "failure" of which i recently documented in a post discussing the problems within wikipedia and d.p. the thrust: power freaks grab power and ruin everything. then all that was followed by the rise of myspacefacebook, where people are busy chatting up their high-school peers, and twitter, the 21st-century telegraph where people who are too busy and superficial for more than 140 characters have found that they too have an ability to do "networking". but, again, i refer you to the subject-header of this post... for those who want to look ahead, the interesting questions are ones like "where do we go from here?" and "how can we use the tools now available to do something irresistible?", and "what is the next form of art that will be _captivating_?" -bowerbird

1 0

Career opportunity inside
by gutvol-d＠pglaf.org 05 Oct '11

05 Oct '11

Hello, We have an excellent opportunity for an apprentice applicant to join a rapidly expanding company. An at home Key Account Manager Position is a great opportunity for stay at home parents or anyone who wants to work in the comfort of their own home. This is a part time job / flexible hrs for Canadians only,This is in view of our not having a branch office presently in Canada, also becouse of paypal and ebay policies wich is prohibit to work directly with residents of some countries. Requirements: computer with Internet access, valid email address, good typing skills. If you fit the above description and meet the requirements, please apply to this ad stating your location. You will be processing orders from your computer. How much you earn is up to you. The average is in the region of US$650- US$750.00 per week, depending on whether you work full or part time. If you would like more information, please contact us stating where you are located and our job reference number - 06731-354/5HR. Please only SERIOUS applicants. If you are interested, please reply to: Norbert(a)firstjob-market.com

2 1

Re: [gutvol-d] do you want to look backward? or move forward?
by Bowerbird＠aol.com 05 Oct '11

05 Oct '11

ricky said: > I'm not sure. i'm sure. i'm absolutely positive. > As content creators, it's in publishers' best interest except publishers are _not_ "content creators". _writers_ create the content. always have; always will. the publishers have traditionally taken that "content" and made it the subject of large press-runs so as to make affordable the print-books they then placed in brick-and-mortar bookstores. they are _middlemen_, and all the services they rendered were a function of the former nature of the book as _a_physical_product_, and they extracted the lion's share for those services... but now writers can turn their own content into e-books -- using a few dozen lines of code, as i've just shown -- that are easily uploaded and then available world-wide, without necessarily giving up _any_ cut to a "publisher". the old days will soon be history, and publishers know it. indeed, they are _sure_ about it; even absolutely positive. the only question is how long they can ride it until it dies. -bowerbird

2 1

these grapes are sweet -- lesson #18
by Bowerbird＠aol.com 04 Oct '11

04 Oct '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** again, here's spellcheck run against grapes006.txt: > http://zenmarkuplanguage.com/grapes115.py > http://zenmarkuplanguage.com/grapes115.txt this tells us what words we still need to add to our "specials" dictionary, so we'll overlay that file now: > http://zenmarkuplanguage.com/grapes-006-specials.txt and we'll run a the spellcheck using that full "specials": > http://zenmarkuplanguage.com/grapes116.py > http://zenmarkuplanguage.com/grapes116.txt this shows us we've attained a null "notfound" list, so we can create a custom unique dictionary for the book. and here's the newest version of the text-file, which now includes the formatting and styling information: > http://zenmarkuplanguage.com/grapes007.txt i also modified the pagenumbers and runheads so they would take the form z.m.l. uses "traditionally". -bowerbird

6 19

these grapes are sweet -- lesson #23
by Bowerbird＠aol.com 04 Oct '11

04 Oct '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** again, here's our z.m.l. "master", the latest version of our text: > http://zenmarkuplanguage.com/grapes007.txt so now we're generating e-books from this "master" .zml file. it's important to recognize that we're building all our e-books from this single text-file, minimizing our file-handling chores. in our last lesson, we created a .mobi e-book, for the kindle. today we do the code that generates material for the .epub... .epub is more complicated -- _not_ "more sophisticated", just more _complicated_ -- because we have to "chop up" the book into chunks that are no bigger than 300k, because some of the .epub viewer-programs were evidently coded by sick baboons, and can't handle any more text than 300k or they will choke... this includes the viewer-app from adobe, which incorporates d.r.m., and is often required by corporate publishing houses, so sometimes you have no option to use some other viewer... so now people all over the world have to chop up our e-books, because the coders at adobe are incompetent at what they do... hurray for .epub! aren't you glad we finally have "a standard"? i find this whole idea of chopping up the book very distasteful. it reminds me of the way you have to chop up a piece of meat into teensy-tiny pieces so grampa -- who's lost all his teeth -- can gum it successfully. life shouldn't need to be this difficult. at any rate, here is the code that creates a folder of .html files: > http://zenmarkuplanguage.com/grapes123.py > http://zenmarkuplanguage.com/grapes123.txt and here's the subdirectory where those files were created: > http://zenmarkuplanguage.com/grapes123-folder this folder of files can then be dropped on "epubzip", which is a little app that zips up the folder in the way .epub requires... (they made it so you couldn't use just _any_ old zip-program, because that would have been far too simple and made sense.) once you've dropped the folder onto "epubzip", you'll get this: > http://zenmarkuplanguage.com/grapes123-folder.epub you'll then wanna drop that .epub onto "epubcheck", to ensure it validates your parking for you. love that validated parking... *** again, as before, this code is _not_ "hardened", nor does it handle anything over and above which was in this p-book -- entirely headers and paragraphs save for a few blockquotes -- because it is the minimal set needed to do _this_book_... so it's nothing more than "proof of concept". but it _is_ that. *** so we've got a kindle and an .epub. now we just need a .pdf... -bowerbird

1 0

Re: [gutvol-d] these grapes are sweet -- lesson #21c
by Bowerbird＠aol.com 04 Oct '11

04 Oct '11

roger said: > Are you showing off or trying to advance the art? first, i don't know what happened to grapes120.py. i fully intended to put that one up, so i'll dig it out. it basically just said, "your dictionary is complete". and no, i'm not releasing scripts in the 200-series. for reasons that should become very clear shortly. *** ok, here's the deal, roger... :+) if i give you the code that creates the viewer, then you'll say "...but we can't edit the text..." and if i then give you code that lets you edit, you'll say "...but this just saves out each page, when what we wanna do is view the diffs, and cumulate the pages into a book, and...", etc. i don't wish to participate in one-way giving... in the past, i haven't shared _any_ of my code. during the "babelfish" thread, from last year, i invited people on this listserve to collaborate with me, as i guided a process of writing code, but nobody contributed anything to the thread -- absolutely _nothing_ at all, from _anyone_ -- so i took down that thread before it got going... so this time around, a year or so later, i decided to take the thread through to its full conclusion, and post the code that i wrote in the process of it, despite the fact that i would get zero cooperation. but that doesn't mean i'm willing to just hand over _everything_. i decide what i give, and what i don't. just like you do. and just like everyone else does. i am boiling a bunch of lobsters in a kettle, roger... and every six months or so, i turn up the heat a bit. so yeah, probably in six months, i'll give y'all the viewer-code. six months later, the editing code. and six months after that, the cumulation code... or maybe not. as long as i give people applications that work, i feel like i have contributed to the cause, even if i keep my source-code to myself, thank you. so you really need to be appreciative for what i give, rather than trying to spotlight what i _haven't_ given. because that kind of focus is just gonna piss me off. and i can tell you why... see, roger, i've been boiling these lobsters for years. literally _years_. it started off with me telling them that they could turn project gutenberg's plain e-texts into beautiful e-books, using "zen markup language". the lobsters scoffed. they told me that i was crazy. i knew i wasn't crazy, because i had _working_code_ on my hard-drive that _proved_ that i was correct... but they scoffed. said the only way to get e-books was to mark the text up with _x.m.l._... well, it was plainly obvious to me (and anyone with half a brain) that their x.m.l. was _too_complex_ to be a solution -- at least one that depended on p.g. volunteers -- so i persisted, said my proof was on my hard-drive. and they persisted, _daring_ me to prove 'em wrong; they said i _lied_ about what was on my hard-drive! well, again, as i said, i had working code that proved that i was right and they were wrong, so i could have revealed all, right then and there, and been finished. but this was too much fun. they were actually _betting_ their own _credibility_ about what i had on my hard-drive. how _stupid_! they couldn't know what i had on my hard-drive, so why would they be so stupid as to make a bet on it? if i was lying, i'd know it, so i wouldn't take the bet. so if i was gonna take the bet, that was meaningful. so it was dumb for them even to _propose_ the deal; it's like betting me if i can _miss_ a basketball shot. of course i can _miss_ it -- just toss the ball away... why would they _ever_ make a bet like that? stupid! so i played them like the ignorant asses they are... i let 'em get all cocky, thinking they'd defeated me, turning 'em into lobsters who crawled in my kettle. so then i started turning the heat up, little by little. first i showed them one thing. then another thing. the next thing, and the thing that came after that... and every time they did their "yeah, but...", i would refute their point with solid proof to the contrary... they gradually stopped attacking, and grew sullen. after a couple years, it finally dawned on them that they'd been taken. but it was far too late by then... then a couple years later, marcello had to admit _total_defeat_ by using a light-markup approach. the .tei/.xml route wasn't gonna put p.g. e-texts on the kindle or the ipad, and marcello knew it... meanwhile, i continue to enjoy turning up the heat. i am doing it to this day, roger, proving them to be stupid asses who bet, and lost, their own credibility about what i had in my pocket. they were so dumb. i understand that you want to just _eat_ the lobsters. but, frankly, i'm having too much fun cooking them. *** but anyway... let me be very clear in speaking to you, roger... you're not one of the assholes i talk about above. you weren't here back then, or at least not posting. it is the case, however, that you _have_ said some untrue and unflattering things about me at times, and never apologized. it's ok, i can take the heat. but you're really not in a position to upbraid me... having said that, though, i don't hold a grudge, and i respect a lot of the work that you've done. this especially applies to the coding you've done, making tools, and setting up a proofing website. but it's precisely because of this that i'm perplexed. what is it, exactly, about the viewer-script which is something that you are not capable of coding? why do you ask me for code that you can write? indeed, for code that you have _already_written_? (your proofing site needed code exactly like that.) but even if you hadn't already written such code, the viewer-script is little more than summoning the text for a page plus the scan for that page... (the other big part is the logic of the html-form, but that is largely contained in the .html itself, so simply viewing the source will reveal all that.) so i am curious about why you'd make a big deal about the viewer-script; it's rather primitive code. indeed, the _search_ code, which i _did_ provide, is more advanced, and more thoughtful as well... and -- just to put the finishing touch on this -- if you go back, you'll see that the search code included the option of showing a specific page, thus creating a foundation for the viewer-script. i'm a little surprised you didn't recognize that it was the key that would unlock that door for you. anyway, roger, if you wanna discuss this stuff, i'm open. you're the only guy around who can talk about it at a level where i just might learn a thing or two from a conversation with you... -bowerbird

1 0

these grapes are sweet -- lesson #21c
by Bowerbird＠aol.com 04 Oct '11

04 Oct '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** i said: > i left them in, because i was curious to see if anyone noticed. > evidently, nobody paid enough attention to notice. oh well... of course, i also presented the _errors_ incorrectly, with the "error" and "fixed" labels reversed, and nobody commented about that, either, so it appears nobody is paying attention. but that's ok, folks. i keep myself mightily amused with it all. this is just good fodder for those future cyber-archeologists, so they realize we weren't _all_ stupid idiots about e-books... for the record, i've appended the correct version of the chart. *** now, let's have a little more fun with the cleaning dimension. if you're new here, you might've noticed something "curious" about this last stage of the clean-up job, where i _evaluated_ how well we'd done by comparing the text to a "clean" copy... and you might have thought to yourself, "well, that's great, but the problem is that you usually don't _have_ a clean copy, which is why you are cleaning the text to begin with, so that does not work as some kind of 'general solution' for us". and you would be right. or at least half-right. for at least as far as you went... but there are some interesting wrinkles just beneath the surface. because it ends up that we can use another _incorrect_ copy of the digitized text to help ourselves find the errors in our work; i.e., the "comparison" copy doesn't necessarily have to be clean. here's the strategy: you compare the second digitization, which let us say (for the purpose of illustration) is another set of o.c.r., with the current digitization, on a line-by-line basis throughout. for each pair of compared lines, there are 5 possibilities: 1. the lines match, and each of them is correct. 2. the lines match, and each of them is incorrect. 3. the lines don't match, and your current copy is correct.4 4. the lines don't match, and the second digitization is correct. 5. the lines don't match, and neither digitization is correct. if o.c.r. is generally correct, and we know that yes, it is indeed, the first scenario is the one that will happen most of the time... for the time being, we'll put aside the second scenario. for the remaining 3 scenarios, we can compare the line in our current digitization to the pagescan, and correct it if required (which will be in the cases which fall under scenarios 4 and 5). in the case of scenario 3, we'll look at the line, see it's correct, and leave it as it is. it was a little work, but no harm was done. the secret of this approach is that a _difference_ between the two lines in our pair will flag that line so that we examine it... whether it's right or wrong is immaterial, since we _look_ at it. and since the number of nonmatching pairs of lines is _small_ (relative to the number that match because they're both correct) this approach is relatively good at finding the _incorrect_ lines... indeed, the only case that gives us any pause at all is scenario 2, where the lines match, but each of them happens to be incorrect. now, i can't blame you if you think scenario 2 will be common... it kinda makes sense that if one o.c.r. makes an error on a line, that a second o.c.r. pass would make that identical error again... and in such a case, they'd both match, but they'd both be wrong. but how about if we do an empirical test of that assumption, eh? indeed, how about doing _lots_and_lots_ of empirical tests of it? i've done just that -- lots of empirical tests of that assumption -- and my research has shown very strongly that it is _not_ the case. and i can demonstrate it once again, by using our current book... *** if you look, you'll see there's another digitization of this book done by internet archive. it's from the university of california, whereas the first copy was scanned at the university of toronto. here's the term to search for in order to find the second copy: > booksandculture00mabirich here's the landing page for it: > http://www.archive.org/details/booksandculture00mabirich and here's the o.c.r. text for it: > http:// ia700303.us.archive.org/10/items/booksandculture00mabirich/booksandculture0… now, there are a lot of things to dislike about that second copy. first and foremost, you'll notice that it's missing its em-dashes. i'm serious; the em-dashes are missing in action, gone entirely! it's _also_ missing many of the end-of-line hyphens, and some apostrophes as well, but let's just focus on the em-dashes now. look at page 30 to see all three of the missing-in-action cases: > http://z-m-l.com/go/mabie/mabiep030.jpg this is a very common problem with books from internet archive. it's a serious problem -- a fatal problem, really -- and yet it is a problem that has infested far too many books in their library... it's downright embarrassing... i would be _ashamed_ to put out books that were missing their em-dashes like that, but evidently the people at internet archive do not have any problem with it... because i informed them about these glitches many _years_ ago. and -- when they ignored me -- i _continued_ to inform them... and when they _still_ ignored me, i finally started to pester them. and i pestered them for _years_, until finally they agreed to _fix_ those em-dash problems... and so i waited for them to do that... and i waited. and i waited. and i waited some more. until finally, i got tired of waiting, and i asked 'em why they hadn't fixed them. and i was ignored. and brushed off. and they called me names, and asked me why _i_ was being so difficult. they blamed _me_! hey, i wasn't the person who was losing their damn em-dashes... so i continued to pester them... until finally they _banned_ me... and now, years and years and years later, some of their e-texts -- like this one we're viewing right now -- have no em-dashes... they should be ashamed of themselves. they should be disgraced. but instead, they are putting out press-releases, and trumpeting how wonderful they are, now that kindle is grabbing their books. evidently, they don't expect those kindle people to actually _read_ their e-books, and discover what a big load of crap their o.c.r. is. and i would say it directly to their face: you should be ashamed. *** anyway... though i wouldn't advocate using this second copy of the book for a full-on comparison, since you'd have to "work around" all the missing characters, we can certainly take a quick look at the 16 lines in our first book which had errors in them, to evaluate... we find 12 of the 16 -- _75%_ -- were correct in the second book. here are those 12: > plete expressions, in that concrete > The latest of them, Count Tolstoi s > The reality of this element of per > all they saw and knew a part of them > process, and culture and genius stand > there has always been, not only a > stories and constantly touched upon > exploration, travel, and discovery ; he > There are, it is true, a few men and > from the days of the earliest Greek > sion, thought, impulse, which never > grance, and growth which lie enfolded in other words, most of the time, the two sets of o.c.r. did _not_ have matching errors. one got it right -- the other got it wrong. so this comparison could've reduced our error-count for this book to _4_ errors, out of 279 pages, which is an extremely good payoff for a proofing procedure that -- unlike distributed proofreaders -- did not require a painful word-by-word comparison page-by-page. which is _not_ to say a "smooth-reading" wouldn't be a good thing. because it would be a good thing. but let's have people who _want_ to read this particular book be the ones who do the smooth-reading. *** so, all in all, this _comparison_ methodology is far superior to the word-by-word method now employed by distributed proofreaders, providing benefits which are comparable but at a _far_ lower cost. this is a methodology that can _scale_ to the required dimensions, unlike the d.p. way, which has stalled out at 2,500 e-texts a year... and yes, i told this to the people over at distributed proofreaders. you can still find it, still sitting there, in a thread on their forums: > http://www.pgdp.net/phpBB2/viewtopic.php?t=24008 what did d.p. do? they banned me too. seemed to be going around. -bowerbird ===================================== p.s. all in all, the results were very good. recall the book is over 200k, and 279 pages. *** 16 lines with o.c.r. errors going uncaught, most of which were involving _punctuation_. 2 were on a name, which we shoulda checked. wrong> plete expressions. In that concrete fixed> plete expressions, in that concrete chstr> =================^=^=============== wrong> and exhausts while he instructs, the fixed> and exhausts while he instructs; the chstr> ===============================^==== wrong> bought Gary's crib, and took it with fixed> bought Cary's crib, and took it with chstr> =======^============================ wrong> read my Gary's Plato. It so hap- fixed> read my Cary's Plato. It so hap- chstr> ========^======================= wrong> The latest of them. Count Tolstoi's fixed> The latest of them, Count Tolstoi's chstr> ==================^================ wrong> mass the facts about any given period, fixed> mass the facts about any given period; chstr> =====================================^ wrong> The reality of this, element of per- fixed> The reality of this element of per- chstr> ===================^---------------- wrong> ail they saw and knew a part of them- fixed> all they saw and knew a part of them- chstr> =^=================================== wrong> process, and culture and genius stand, fixed> process, and culture and genius stand chstr> =====================================^ wrong> there has always been, not only a fixed> there has always been not only a chstr> =====================^----------- wrong> stones and constantly touched upon fixed> stories and constantly touched upon chstr> ===^^------------------------------ wrong> exploration, travel, and discovery; ha fixed> exploration, travel, and discovery; he chstr> =====================================^ wrong> There are. It is true, a few men and fixed> There are, it is true, a few men and chstr> =========^=^======================== wrong> from the days of the earliest Greek, fixed> from the days of the earliest Greek chstr> ===================================^ wrong> sion, thought. Impulse, which never fixed> sion, thought, impulse, which never chstr> =============^=^=================== wrong> grance, and growth which he enfolded fixed> grance, and growth which lie enfolded chstr> =========================^^----------

3 2

these grapes are sweet -- lesson #22
by Bowerbird＠aol.com 03 Oct '11

03 Oct '11

this thread explains how to digitize a book quickly and easily. search archive.org for "booksculture00mabiuoft" for the book. *** here's the latest version of our text, all cleaned and formatted: > http://zenmarkuplanguage.com/grapes007.txt so now we wanna start generating e-books from this "master". ok, here's code that kicks out an .html file which _kindlegen_ -- amazon's free .mobi generator -- can use to create a .mobi. (you can download kindlegen from the amazon.com website.) > http://zenmarkuplanguage.com/grapes122.py > http://zenmarkuplanguage.com/grapes122.txt the .html file that's generated is here: > http://zenmarkuplanguage.com/grapes122.html that .html file can be input to the kindlegen app, which will turn it into a .mobi that can be read just fine, but it will _lack_ a few niceties, including a cover, a table of contents, and ncx (which is another table of contents, because i don't know why). so, because we don't want to be second-class in any fashion, here are the other files you'll need in order to be first-class: > http://zenmarkuplanguage.com/grapes122.jpg > http://zenmarkuplanguage.com/grapes122.opf.txt > http://zenmarkuplanguage.com/grapes122.ncx.txt so download/save all three... the .jpg is straightforward, but you'll remove the ".txt" extension from the .ncx and .opf files -- the ".txt" is so those files will display in a web-browser -- and then move all of them (plus the .html file, of course) into the folder where your kindlegen program is located, and then execute the kindlegen program by pointing it to the .opf file... you should get out a mobi that looks like this: > http://zenmarkuplanguage.com/grapes122.mobi considering the fact that we defined no styles, i think the .mobi looks astoundingly good, except the headers are not centered. of course, since you've got the code, "customize as desired..." and no sir, that code won't extrapolate to anything other than other books which are dead-simple. you'll need to harden it, so it becomes robust to the exceptions which inevitably occur, and expand it, so that it can handle challenges which pop up... but that's not my aim here. i'm just showing you it's possible. *** our kindle work is done. in the next lesson, we'll do .epub... -bowerbird

1 0

the importance of creating an infrastructure
by Bowerbird＠aol.com 02 Oct '11

02 Oct '11

another important thing which i should've mentioned in my last response to keith is that both amazon and apple have created ecosystems which surround their machines. it's not just that they have millions and millions of users, but that this broad base creates an entire infrastructure, because amazon and apple concentrated on that aspect. a lot of the time, this is condemned as "a walled garden", where the criticisms are aimed at the "walled" part of it, but it's also important to understand the "garden" part... the other companies trying to deliver hardware expect us to live in a barren desert, or at best a weed-infested patch. but if/when there is an _infrastructure_ around machinery, it attracts focus and resourcefulness which create synergy. here's an example of that, a browser plug-in which sends the contents of a web-page to your kindle viewer-program: > http://sendtoreader.com/ > http://kindleworld.blogspot.com/2011/03/want-to-try-out-free-web-app.html -bowerbird

2 1