On 10/29/2011 02:18 PM, Greg M. Johnson wrote:

... reasonable speculations about 200 years from now, ...

Hilarious. Could a person from 1800 reasonably have foreseen portable devices that could communicate with every person on earth and ebooks delivered over cellular networks? (In 1800 they used gas for illumination, and electricity, thou known to scientists, had no practical application. Radio telegraphy came a hundred years later.)

Microsoft Word 1995 format (if there were such a thing) is probably out. WordPerfect for 3.1 Windows format is probably out.

The works worth preserving -- those that age slower than the substrate they are written on -- were preserved by copying them again and again to new substrates. Every file format will age and become unreadable eventually. The only hope for preserving the file's contents will be timely conversion into a newer format. Plain text lacks some important information about the text structure and does not convert gracefully to any other format. Many have tried, all have failed. By the time OCR software will recognize more textual features than plain text, plain text will be dead. Plain text sucks as archival format because it forgets too much. Plain text sucks as distribution format too. It displays well on an 80-column teletype (some 20 people still have one in the basement) and nowhere else (some 6 billion people use something else). "Reasonable speculation:" in 100 years, machine OCR will be so accurate that it will find and correct typos in the printed book, it will infer semantics and mark them up, and, while PG plain text files can still be found in the Google cache, nobody will use them, as it will take about 2ms to OCR any scanned book from the IA.

But I am wondering who wins the elimination round between: i) PG's current format for "TXT" with 80 chars ii) a format that is still all ASCII but removes all the end-of-column CR's, and only has double returns at the end of paragraphs.

So you think that TXT *without* hard CRs is better than text with hard CRs. After all that pomp about "objective truth" and "200 years from now" I expected something more thoroughly enlightening. -- Marcello Perathoner webmaster@gutenberg.org

On 10/29/2011 6:18 AM, Greg M. Johnson wrote:

But I am wondering who wins the elimination round between: i) PG's current format for "TXT" with 80 chars ii) a format that is still all ASCII but removes all the end-of-column CR's, and only has double returns at the end of paragraphs.

This is a hard question to answer, because you haven't told us about your assumptions regarding rendering software (and all rendering requires /some/ sort of software). In true ASCII, for example, the way to make an "enyé" (e.g. the ñ in mañana ) is 110 (e) 8 (backspace) 126 (~), or as interpreted, "strike the paper with the print head in position 110, move the print head back one character, then strike the paper with the print head in position 126." To start a new line, you would use 13 (carriage return) 10 (line feed) or "move the print head to the far left position, then roll the platen up one line." I'm going to make a possibly unjustified assumption that these ASCII characters designed to control the printing hardware (form feed to advance one page, bell to activate the bell, XOff to stop the rs-232 feed while I'm busy, XOn to start the feed up again, etc.) are out of bounds for you, as you're not trying to control a mechanical device; I'm assuming when you say ASCII, you mean /printable/ ASCII, or ASCII values between 32 and 126 (inclusive). But on the other hand, you've mentioned carriage returns, so at least /one/ control code is acceptable. Of course, simply moving the cursor (the CRT equivalent of the print head) to the left side of the screen would simply result in a line of text being printed over and over again in the same spot, so I'm assuming you want to reinterpret CR to mean "move cursor to left hand side and move it down one line." Or does your system require both the carriage return and line feed to move to "the beginning of the next line on the screen?" (Note that in the *nix world, this function has been assigned to the line feed ASCII value, such that it is common to see it (10) referred to as "new line" instead of "line feed".) Should I assume that "move the cursor to a new line on the display, scrolling the display when the cursor is already at the bottom of the screen" can be represented by a carriage return (13) /or/ a line feed (10) but that the two of them together must be merged into a single command? That is, CRLF (two characters) is a single new line, but CRCRLF (three characters) would be two new lines? Are there any other control codes that you would exempt from the "printable ASCII" rule? I'm starting to feel very uncomfortable making all these assumptions. Now if I'm assuming that the software you have in mind can't handle strike-backspace-overstrike, then I have to assume it can't handle much more sophisticated functions either, like word-wrapping; we're going to need something like CR/LF/CRLF to tell the hardware where to put the cursor, otherwise we'll end up with a screen full of text perfectly right-justified, but with most of the words split at odd places when they occur at the end of the line. So we must insert this vaguely-defined "new line" at the beginning of any word that would be divided at the edge of the display. But to know where to insert the new line characters, we have to know the width of the screen, in fixed-size characters. PG made the /assumption/ that the width of all screens was 80 characters; the notion of TXT70 was that once the line length exceeded 70 characters, you should insert a new line character at the next occurring whitespace, so you should never see a line that ends funnily. But you're talking about /really/ old computers, which includes the Commodore, the Atari, and, most importantly, the Apple II. All of those computers had screens that were only /40/ characters wide, so to satisfy your requirements, what we really wanted is TXT40, where no line exceeds 40 characters in length. Of course, if you're willing to allow software smart enough to do word wrap, all bets are off, because with software I can display HTML just as easily, and with much better results, than pure printable ASCII. By now you should be saying, "Man, this is absurd." And you'd be right. You have posited a question with exposing all relevant assumptions or providing all relevant requirements. We're not starting on the same page at all. Everyone (well, almost everyone) wants simple answers, magic bullets. "We could easily fix today's economy if we would only just [insert favorite simplistic solution here]." But the simple answer is that there are no simple answers. Life, and text preservation, is more complex and nuanced than that. This is not to say that the answers do not exist, it just means that if you want a solution you have to closely examine the parameters of your problem, and deal with the required complexity of the solution.