- gutvol-d - lists.pglaf.org

Re: save those pagenumber references
by Bowerbird＠aol.com 21 Mar '10

21 Mar '10

al said: > My personal practice is and therein lies the rub. the p.g. e-texts are rife with "personal practice". and the d.p. e-texts are soaking in it right now... one thing you have to know about pagenumbers is some people need 'em and other people hate 'em... which means you have to have 'em, and you have to give people a way to shut them off... _totally_ off... the only way to do that is to establish a convention, so viewer-app developers can make everyone happy. most "personal practice" implementations try to walk the tightrope between the two sides, and fail _both_, in the sense they don't do the _full_ job that the "pro" people want pagenumbers to do, but yet aren't nearly as non-invasive as the "anti" people reasonably want. if a hundred different digitizers do it a hundred ways -- or a thousand digitizers do it a thousand ways -- nobody is gonna end up happy; we'll all be miserable. and face it, if both sides are going to end up unhappy, you might as well flip a coin and make one side happy. the only way to make it work is to do it _one_way_... so developers can target the convention successfully. michael isn't going to prescribe this remedy for p.g. even if he tried, he probably would not succeed, and he has made it clear that he doesn't even want to try and do things like that, as per his basic philosophy... nobody else has a remote chance of success with p.g. so alas, it is not to be. but perhaps it doesn't matter. because it's becoming increasingly clear that the only cyberlibrary that's going to matter is the google one, and -- after a few missteps at the very beginning -- google has gotten pretty smart about pagenumbers... so whatever conventions they establish will stick. *** but, to answer the question... gardner said: > Is there any suggestion what > a formatted text-only book that > retains page numbers should look like? > Is it reasonable to just sprinkle them > into the text, maybe something like this: there's nothing difficult about the issue, technically. you wouldn't want to "sprinkle them" thoughtlessly, but any number of _well-specified_conventions_ can handle the tiny number of wrinkles that do crop up... (tell me if you want me to dredge my memory-bank to catalog them, but there seriously aren't too many.) -bowerbird

3 2

save those pagenumber references
by Bowerbird＠aol.com 21 Mar '10

21 Mar '10

ok, on the "good news" front, it appears that rfrank has finally decided to start naming his files more wisely, so big respect to the people who steered in that direction. there seemed to be some uncertainty from roger about how to go about coding apps with those new filenames, so i'll talk a little bit about that and hope it filters back... but the initial info can be used by other people as well! sure, if you're scanning your own books, you can name the files intelligently from the get-go, and never worry. (but, um, if you _are_ scanning your own books, please ask me for advice on filenaming, and don't just do what d.p. did when they tried to implement smart filenames, because they got some of the "details" badly mangled.) but sometimes, from other people, you might get files which were named badly, and you'll have to rename 'em. even some of the big scanning projects -- umichigan and the internet archive and google (well, not so much google, not any more, they wised up pretty quickly) -- have been known to adopt some fairly stupid filenaming conventions, so if you use their stuff, you'll have to clean up their mess. so it behooves you to know how. first things first: get yourself "twisted", the dkretz program. > http://code.google.com/p/dp50/downloads/list the initial impetus for this program was precisely this task of renaming files intelligently, and it works very well for it. so that's really all you need. but i'll tell you a bit more... let's say you're doing preprocessing. one of the things that d.p. does is it strips the pagenumbers out of the .txt files... that is just asinine! do not do that, folks. that is the info that you _need_, so -- obviously -- do _not_ throw it away! rfrank discards the pagenumber info from his .txt files too. sometimes, though, for some books, the pagenumber info sidesteps deletion. one such book was the "sitka" one that jim and i have been working on. you can find the file here: > http://z-m-l.com/go/jimad/sitka0-ocr.txt you can see, at the bottom of each page, the pagenumber, enclosed in brackets. and oh what a lovely sight they are! because they tell exactly what the file _should_ be named! for instance, go down to the start of chapter 1. you will see that it occurs in the file rfrank named "011.txt". but, as shown by the pagenumber at the bottom, it's page 7, and _should_ be named "007.txt" or (better) "sitkap007.txt". (in case you're wondering why chapter 1 starts on page 7, it's because the _foreword_ starts on page 1, and runs to page 5. page 6 is a blank verso that is opposite chapter 1.) so we know the file "011.txt" should be "sitkap007.txt". great! but remember the another wrinkle too -- the pagescan filename. so if we know that "011.txt" should be named "sitkap007.txt", we also know that "011.png" should be named "sitkap011.png". now we're cooking... *** so, to find out the pagenumbers in each of the text-files, you can run a little perl program i've put up on the site: > http://z-m-l.com/go/jimad/doglobal.pl that program is a simple "find" program that pulls out any line with the string ".txt" in it, or a right-bracket (i.e., "]"), as shown: sitka0-ocr-001.txt -- [Illustration][**fine print verified by CP] sitka0-ocr-002.txt -- sitka0-ocr-003.txt -- sitka0-ocr-004.txt -- [Illustration: Lovers' Lane, Sitka.] sitka0-ocr-005.txt -- sitka0-ocr-006.txt -- sitka0-ocr-007.txt -- [3] sitka0-ocr-008.txt -- [4] sitka0-ocr-009.txt -- [5] sitka0-ocr-010.txt -- [Blank Page] sitka0-ocr-011.txt -- [7] sitka0-ocr-012.txt -- [8] sitka0-ocr-013.txt -- [9] sitka0-ocr-014.txt -- [10] sitka0-ocr-015.txt -- sitka0-ocr-016.txt -- [11] sitka0-ocr-017.txt -- 112] sitka0-ocr-018.txt -- [13] sitka0-ocr-019.txt -- [14] sitka0-ocr-020.txt -- [15] sitka0-ocr-021.txt -- [16] sitka0-ocr-022.txt -- [17] sitka0-ocr-023.txt -- [18] sitka0-ocr-024.txt -- [19] sitka0-ocr-025.txt -- 120] sitka0-ocr-026.txt -- [21] sitka0-ocr-027.txt -- [22] sitka0-ocr-028.txt -- [23] sitka0-ocr-029.txt -- [24] sitka0-ocr-030.txt -- [Blank Page] sitka0-ocr-031.txt -- [25] sitka0-ocr-032.txt -- [26] sitka0-ocr-033.txt -- [27] sitka0-ocr-034.txt -- [28] sitka0-ocr-035.txt -- [29] sitka0-ocr-036.txt -- [30] sitka0-ocr-037.txt -- [31] sitka0-ocr-038.txt -- [32] sitka0-ocr-039.txt -- [33] sitka0-ocr-040.txt -- [34] sitka0-ocr-041.txt -- [35] sitka0-ocr-042.txt -- [36] sitka0-ocr-043.txt -- [Blank Page] sitka0-ocr-044.txt -- [37] sitka0-ocr-045.txt -- [38] sitka0-ocr-046.txt -- [39] sitka0-ocr-047.txt -- [40] sitka0-ocr-048.txt -- [41] sitka0-ocr-049.txt -- [42] sitka0-ocr-050.txt -- [43] sitka0-ocr-051.txt -- [44] sitka0-ocr-052.txt -- [45] sitka0-ocr-053.txt -- [46] sitka0-ocr-054.txt -- [Blank Page] sitka0-ocr-055.txt -- [47] sitka0-ocr-056.txt -- [48] sitka0-ocr-057.txt -- [49] sitka0-ocr-058.txt -- [50] sitka0-ocr-059.txt -- [51] sitka0-ocr-060.txt -- [52] sitka0-ocr-061.txt -- [53] sitka0-ocr-062.txt -- [54] sitka0-ocr-063.txt -- [Blank Page] sitka0-ocr-064.txt -- [55] sitka0-ocr-065.txt -- [56] sitka0-ocr-066.txt -- sitka0-ocr-067.txt -- [57] sitka0-ocr-068.txt -- sitka0-ocr-069.txt -- [59] sitka0-ocr-070.txt -- [60] sitka0-ocr-071.txt -- sitka0-ocr-072.txt -- [61] sitka0-ocr-073.txt -- [62] sitka0-ocr-074.txt -- sitka0-ocr-075.txt -- [63] sitka0-ocr-076.txt -- [64] sitka0-ocr-077.txt -- [65] sitka0-ocr-078.txt -- [66] sitka0-ocr-079.txt -- sitka0-ocr-080.txt -- [67] sitka0-ocr-081.txt -- [68] sitka0-ocr-082.txt -- [69] sitka0-ocr-083.txt -- [70] sitka0-ocr-084.txt -- [71] sitka0-ocr-085.txt -- [72] sitka0-ocr-086.txt -- [73] sitka0-ocr-087.txt -- [74] sitka0-ocr-088.txt -- [75] sitka0-ocr-089.txt -- [76] sitka0-ocr-090.txt -- sitka0-ocr-091.txt -- [77] sitka0-ocr-092.txt -- [78] sitka0-ocr-093.txt -- [79] sitka0-ocr-094.txt -- [80] sitka0-ocr-095.txt -- [81] sitka0-ocr-096.txt -- [82] sitka0-ocr-097.txt -- [83] sitka0-ocr-098.txt -- 184] sitka0-ocr-099.txt -- [85] sitka0-ocr-100.txt -- [86] sitka0-ocr-101.txt -- [87] sitka0-ocr-102.txt -- [88] sitka0-ocr-103.txt -- [89] sitka0-ocr-104.txt -- [90] sitka0-ocr-105.txt -- [91] sitka0-ocr-106.txt -- [92] sitka0-ocr-107.txt -- [Blank Page] sitka0-ocr-108.txt -- [93] sitka0-ocr-109.txt -- [94] sitka0-ocr-110.txt -- [Blank Page] sitka0-ocr-111.txt -- [95] sitka0-ocr-112.txt -- [96] sitka0-ocr-113.txt -- [97] sitka0-ocr-114.txt -- [98] sitka0-ocr-115.txt -- [99] sitka0-ocr-116.txt -- [100] sitka0-ocr-117.txt -- sitka0-ocr-118.txt -- [101] sitka0-ocr-119.txt -- [102] sitka0-ocr-120.txt -- [103] sitka0-ocr-121.txt -- [104] sitka0-ocr-122.txt -- [105] sitka0-ocr-123.txt -- [106] sitka0-ocr-124.txt -- [107] sitka0-ocr-125.txt -- [108] sitka0-ocr-126.txt -- *** i will do a detailed look at that list, and explain everything in it, but you might wanna take a gander first, to see what _you_ see. since it might be more fun for you to figure it out for yourself, rather than plow through my pedantic bullshit... *** now, we need to do a little repair on some pages, as follows: the left-bracket was misrecognized on 3 files, so fix that: sitka017.txt -- 112] sitka025.txt -- 120] sitka098.txt -- 184] the first 4 pages are front-matter, so add some "f" pagenumbers: sitka001.txt -- add [f001] sitka002.txt -- add [f002] sitka003.txt -- add [f003] sitka004.txt -- add [f004] the first 2 pagenumbers were deleted by early proofers, so add back: sitka005.txt -- add [1] sitka006.txt -- add [2] page 6 really is a blank page, so let's add a pagenumber to it: sitka010.txt -- add [6] the pagenumber on 1 file wasn't picked up by scanner, so we'll add it: sitka068.txt -- add [58] the pagenumber on the last page, a map, wasn't there, so we'll add it: sitka126.txt -- add [109] the rest are illustration pages (even though some claim to be "blank"), which we can tell because they exist outside of the page-sequencing, so we'll add the "a" filenaming convention to slide them into place... append "a" to these unnumbered pages, which had no pagenumber: sitka015.txt -- add [10a} sitka066.txt -- add [56a} sitka074.txt -- add [62a} sitka079.txt -- add [66a} sitka090.txt -- add [76a} sitka117.txt -- add [100a} sitka030.txt -- change [blank page] to [24a] sitka043.txt -- change [blank page] to [36a] sitka054.txt -- change [blank page] to [46a] sitka063.txt -- change [blank page] to [54a] sitka107.txt -- change [blank page] to [92a] sitka110.txt -- change [blank page] to [94a] as i said in a short response to juliet yesterday, many of these missing and misrecognized pagenumbers _could_ have been "filled in" automatically, because of pagenumber redundancy. but editing them wasn't too difficult for this particular book... (i did the editing using my new editor interface, which i will be revealing to all you excited fans out there next week. oh boy!) *** once all of the pagenumbers in the files have been corrected, output from the above doglobal.pl script would look like this: sitka0-ocr-001.txt -- [f001] sitka0-ocr-002.txt -- [f002] sitka0-ocr-003.txt -- [f003] sitka0-ocr-004.txt -- [f004] sitka0-ocr-005.txt -- [1] sitka0-ocr-006.txt -- [2] sitka0-ocr-007.txt -- [3] sitka0-ocr-008.txt -- [4] sitka0-ocr-009.txt -- [5] sitka0-ocr-010.txt -- [6] sitka0-ocr-011.txt -- [7] sitka0-ocr-012.txt -- [8] ... sitka0-ocr-024.txt -- [19] sitka0-ocr-025.txt -- [20] sitka0-ocr-026.txt -- [21] sitka0-ocr-027.txt -- [22] sitka0-ocr-028.txt -- [23] sitka0-ocr-029.txt -- [24] sitka0-ocr-030.txt -- [24a] sitka0-ocr-031.txt -- [25] ... sitka0-ocr-126.txt -- [109] *** then we can do a variant of that output, to do the renaming for us: rename sitka0-ocr-001.txt as sitkaf001.txt rename sitka0-ocr-002.txt as sitkaf002.txt rename sitka0-ocr-003.txt as sitkaf003.txt rename sitka0-ocr-004.txt as sitkaf004.txt rename sitka0-ocr-005.txt as sitkap001.txt rename sitka0-ocr-006.txt as sitkap002.txt rename sitka0-ocr-007.txt as sitkap003.txt rename sitka0-ocr-008.txt as sitkap004.txt rename sitka0-ocr-009.txt as sitkap005.txt rename sitka0-ocr-010.txt as sitkap006.txt rename sitka0-ocr-011.txt as sitkap007.txt rename sitka0-ocr-012.txt as sitkap008.txt ... rename sitka0-ocr-024.txt as sitkap019.txt rename sitka0-ocr-025.txt as sitkap020.txt rename sitka0-ocr-026.txt as sitkap021.txt rename sitka0-ocr-027.txt as sitkap022.txt rename sitka0-ocr-028.txt as sitkap023.txt rename sitka0-ocr-029.txt as sitkap024.txt rename sitka0-ocr-030.txt as sitkap024a.txt rename sitka0-ocr-031.txt as sitkap025.txt ... rename sitka0-ocr-126.txt as sitkap109.txt *** remember that we have to do the scan files as well. (we'll just do a global change from ".txt" to ".png".) rename sitka0-ocr-001.png as sitkaf001.png rename sitka0-ocr-002.png as sitkaf002.png rename sitka0-ocr-003.png as sitkaf003.png rename sitka0-ocr-004.png as sitkaf004.png rename sitka0-ocr-005.png as sitkap001.png rename sitka0-ocr-006.png as sitkap002.png rename sitka0-ocr-007.png as sitkap003.png rename sitka0-ocr-008.png as sitkap004.png rename sitka0-ocr-009.png as sitkap005.png rename sitka0-ocr-010.png as sitkap006.png rename sitka0-ocr-011.png as sitkap007.png rename sitka0-ocr-012.png as sitkap008.png ... rename sitka0-ocr-024.png as sitkap019.png rename sitka0-ocr-025.png as sitkap020.png rename sitka0-ocr-026.png as sitkap021.png rename sitka0-ocr-027.png as sitkap022.png rename sitka0-ocr-028.png as sitkap023.png rename sitka0-ocr-029.png as sitkap024.png rename sitka0-ocr-030.png as sitkap024a.png rename sitka0-ocr-031.png as sitkap025.png ... rename sitka0-ocr-126.png as sitkap109.png *** this example makes it pretty clear that -- if you only leave the pagenumbers in the o.c.r., just leave 'em! -- it's pretty easy to use them to name your files wisely... pagenumbers in the runhead are easy to grab as well. they're either at the right side of the runhead (if odd) or at the left side of the runhead (on the even pages). (the runhead is usually the first line in the file, right?, but sometimes the pagenumber drops to the second. still it's usually the first _number_ you find in the file, so it's easy enough to code your script to look for that.) again, you have to check them!, to make sure they were recognized correctly, so you can fix 'em if they weren't. but once you've got them all in place, you are golden... and the beauty is that now your files are named wisely! you'll always know page 23 is in the file named "p023", and page 46 is in "p046", and page 123 is in "p123"... moreover, when you want to go to page 46, you will actually _end_up_ on page 46, not some other page that is kinda close, depending on what the "offset" is! *** and here's another nice thing. you'll notice that we had some unnumbered pages that were named with an appended "a"? well, we need to keep the recto and the verso straight, if we want to make good e-books, so we can't just add an "a" without a backside "b" too. but hey, that's no problem at all! after each "a" page, we just slide in a "b" name underneath it, and presto!, our recto/verso is right again. and we didn't have to _readjust_ all filenames that followed each "insertion", because those files were wisely-named to begin with. *** there's one more thing to talk about: coding apps... (if you don't do coding, you can leave now if you want; but it probably won't hurt you to read the rest of this. you made it _this_ far, so you must be a glutton for it.) first let's get the necessary admission out of the way... it's very easy to do your coding when you name your files in a stupid 001.txt-999.txt way, because you can simply code the number as a shortcut for the filename. you use an integer for your pagenumber, and it's easy. your _files_ go from 1 to 999, and so do your _names_. it's easy to keep track of things; you just go up or down. because of this ease, i can understand why you _might_ want to keep using those stupid filenames. but don't... still, at first, it may not be immediately obvious to you how to depart from this method. but it really is simple. instead of thinking of each filename as a _number_ (i.e., an integer), think of it as a "name" (i.e., a string). yes, the filename has a number _in_ it, and the number is the _important_part_ (to your end-user), but do not _think_ of it in this way, at least not for the time being. think of the filename as a string, nothing but a string... however, you will _load_ those strings into an _array_... you'll have as many items in the array as you have files, and the value of each item will be the _name_ of the file. then you think of the _index_ for that array as an integer -- because that's what it is! -- and you use _that_ in the exact same way you used your pagenumber integer before. so see, you didn't have to give up the easy convenience of a number to keep on-track like you thought you'd have to. your index array goes up and down, just like it did before. in other words, you can still think of your _files_ as going from 1 to 999, and increment your array index as before. but whenever you want to know the _filename_ of a page, you look-up the value of the array at that index-number. so let's look at how this would work for our "sitka" book. the string value of item array #1 would be "sitkaf001". the string value of item array #2 would be "sitkaf002". the string value of item array #5 would be "sitkap001", because it's page 1, and that's where the foreword starts. the string value of item array #11 would be "sitkap007", because that's page 7, and that's where chapter 1 starts... and the string value of item array #126 would be "sitkap109", and that's the map that's on the last (recto) page in the book. (of course, there will be a blank verso that'll be "sitkap110", since a book cannot have an odd number of pages, can it?) so the last question is "how do i populate the filename array?" there are various ways you can do it, but two good ones are: 1. read the book's subdirectory to glean the graphic filenames. 2. create a "map" file intended to provide the graphic filenames. you can also combine these 2 methods as "belt and suspenders"; you create a map file, but your viewer-app confirms the map by reading the subdirectory to ensure all the graphic files are there. it's not nearly as difficult to create a "map" file as you might think. for instance, look closely at the sitka file we're working on: > http://z-m-l.com/go/jimad/sitka0-ocr.txt just pull out the separator lines, and you've got your map file. of course, the current version of that file is using the current stupid filenames, but you can generate a new concatenated file after you've renamed your .txt files, and your map will be fine. you can also just view your subdirectory structure in a browser, and copy out the filenames, and save them in a file, and bingo!, there's your map file. myself, with z.m.l., i use the separator-line method, as you can see if you look at any paginated z.m.l. file. the lines which have double-braces enclosing a graphic filename constitute the map. *** all in all, if you start naming your files intelligently, you'll find that the benefits far outweigh the costs of doing any rename... still, i've tried here, in this post, to show you how to do a rename in the easiest possible way. just remember not to do like d.p. -- _and_keep_the_darn_filename_information_in_your_o.c.r._files_... -bowerbird

6 8

a sitka smoothreading glitch
by Bowerbird＠aol.com 21 Mar '10

21 Mar '10

rfrank's roundless site -- fadedpage.com -- now has their "sitka" book available for _smooth-reading_, at: > http://www.fadedpage.com/s/sitka/sitka.htm i'll have a lot of nice things to say about rfrank's work, because his e-books really do look quite clean and nice, but -- since this book is in-process and all -- for now, i'll just report a little glitch, on (waitforit) pagenumbers. seems a stray page-indicator found its way onto page 96, so that page 96 is now incorrectly short, with some of its text now being shown on what's called page 97, and with every page after page 96 having its pagenumber off by 1, so the last words ("may be made") are incorrectly reported as page 109, when in actuality they occurred on page 108. that kind of thing can happen when the pagenumbers are something that you recompute at the end of the process, rather than something integral to your entire workflow... -bowerbird

1 0

Re: save those pagenumbers!
by Bowerbird＠aol.com 20 Mar '10

20 Mar '10

i have a reply in the works, but i won't post it until monday... so keep your minds open until then, and have a nice weekend! -bowerbird

1 0

Re: "that doesn't work"
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

jim said: > I’ve put up a new copy of the tool pgdiff > that contains an option “-smarted” which > outputs the text in a form similar to what > I think you want BB for your “smart editor” tool. i'm sure some of your users will enjoy that option, jim... as you might expect, i'll likely stick with my own tools... but yes, this new option might allow me to perfect the tool that i've built in support of your pgdiff tool, i hope. > Your suggestions work in simple cases but I think > you will find that they fail relatively spectacularly > on difficult cases, such as when performing versioning > across different editions. well, if i'm gonna fail, please let me fail "spectacularly". i compare different editions using a different technique; essentially i do a _paragraph-level_ comparison for that. it's easy enough to unwrap texts to the paragraph level. indeed, i do paragraph-level analyses in my comparisons all the time. that's how i catch the paragraphing glitches. (it's also necessary to work at the paragraph level when you're fixing spacey-quotes, as i have mentioned before.) > I also updated the example output file “BBoutput.txt” > to show the new output. great. i'll go get it this afternoon... > Again, the problem is basically the domain > you are interested in working on and the domain > I am interested in working on is very different. actually, they're not. but that's another question for another day. here today's issue is finding and fixing errors by comparing two versions which are similar... > You want a tool that catches small changes > within a line of text, and I want a tool that catches > large changes within a file. two rejoinders. first, my tools are capable of finding "large differences" if they are what exist. but, like i just said, that arena is not of much particular interest here on the p.g. listserve. second, i have -- without knowing it at first -- worked on doing comparisons between what turned out to be different editions of a book. and most of the changes were not "large" ones, but rather "small" ones, notably punctuation variations reflecting different house "styles". i discussed this particular comparison at _great_ length over on the d.p. forums, under a thread with a title like "a revolutionary method of proofing", if you're interested. > It is easy to hypothesize what the “answer” is > if you are not the one doing the work. i agree. that's why i suggested we work on actual data. i find it best if i don't bias my research by selecting the data that i work on, so i work on other people's stuff, which is why i choose that book from rfrank. however, if you want to share some data on a book of your own, one you're working on, i would be happy to look at it... > But if you are the one doing the work you rapidly find > “oops that idea doesn’t work after all!” you know, i hear a lot of people saying "that doesn't work". but usually, they're being bamboozled by some _small_ issue that can be overcome quite easily if they just try... a good example of that was yesterday, when juliet said "your renaming solution won't work because pagenumbers are often misrecognized." well, yeah, that happens, but that particular "obstacle" can be hurdled with little effort. so i invite you to bring any "doesn't work" problems to me... i like the challenge of seeing if i can make it work regardless. -bowerbird

1 0

Re: jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

jim said: > In practice what one normally sees is some weird mixture of > the two possible situations, and it isn't clear to me > which display technique is best, so so far I have > chosen the easiest approach to implement in view of the frank admission, let me make some suggestions. i believe these would make your tool's output more workable for the end-user who has to resolve the diffs, no matter _what_ method they use, including the reg-ex editor you use yourself. *** let me pull out the last 2 of the 55 anomalies i posted for you... *** here's the first: > drew their stores of { krasnia | > Jcrasnia ruiba | > ruiba } (the red some people might prefer the version as it was in your file: > stores of { krasnia | Jcrasnia ruiba | ruiba } (the red rather than showing this as a single diff, i'd present it as two... the first would be: > { krasnia | Jcrasnia }. the second would be > { ruiba | ruiba } *** here's the second example: > the trough of the watering place of { the | > the" "Jamestown," | > Jamestown,"} came to the beach. This place may be or, more in keeping with how it's displayed in your output: > place of { the | the" "Jamestown," | Jamestown,"} came to again, i would present this as two diffs... the first would be: > { the | the" } the second would be: > { "Jamestown," | Jamestown,"} *** in both of these examples, i think combining the 2 diffs into one bracket-bound item confuses the item unnecessarily, and confuses the end-user in the process, making the resolution much more difficult than it needs to be... in many of these "multiple diff" brackets, i could have my tool pull apart the various diffs, and display them appropriately... so, you know, if you think the output you are showing now is done the way you _want_ to have it done, that's your decision. but i think it will be more clear if you did it slightly differently. *** another confusion i had with your output was that there were several bracketed items that contained some separator-lines... since all of those separator-lines were standaridized by you before you ran your pgdiff, it seems to me that none of them should've been included in any of the brackets. should they? if you start bringing non-diff material into the edit process, you're asking for problems, it would seem to me, so i would rework that code to try to avoid such problems if i were you. anyway, just a few suggestions, hopefully helpful ones... :+) -bowerbird

2 1

any arguments against "free-range" proofing?
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

the d.p. proofing system locks each page to a single proofer. (there's one and only one p1 proofer, p2 proofer, and so on.) so does rfrank's roundless system; once a page has been assigned to a proofer, it's semi-difficult to even look at it. and if someone else has reproofed it _after_ that person, then the old version is stored somewhere i can't figure out, so tracking the diffs simply cannot be done by an outsider. (the d.p. system at least allows you to do that tracking, and even has a routine that will show you round-to-round diffs.) it is by analyzing these round-to-round diffs very closely that you can get a sense for how a page progresses from the initial o.c.r. to its final -- hopefully perfect -- stage... *** the question i have today is whether there is a good reason why a page needs to be assigned-and-locked to one person. is there any reason why you shouldn't allow any proofer to go and proof any page in a book? yes, it would mean that some pages might be proofed several times, but so what? that's not necessarily a _bad_ thing, is it? i'm writing code now to build my own proofing system, and i'm curious about this particular aspect. i think it would be important to inform a proofer how many previous people have proofed each specific page, so as to let that proofer choose whether to do an additional proof, but if they _want_ to do it, is there any reason to disallow it? *** partly this ties into _incentives_... most people like _finding_and_fixing_ errors, so there'll be a good incentive for people to work in the "first" proofing... but even in that first proofing, there are a lot of pages that are _already_ perfect, so there are no errors to find or fix... and in the second and third proofings, the number of errors that are left will be small, even collected over a whole book. so i feel it's very important to reward people for _certifying_ a page -- i.e., confirming that the page is indeed error-free. if i was to put this in terms of a "point" system, it'd be this: > 5 points for fixing all of the remaining errors on a page. > 4 points for doing the first "certification" of a clean page. > 3 point for doing the second "certification" of a page. > 2 point for doing the third "certification" of a page. > 1 point for fixing _some_ (but not all) errors on a page. if you certify a page clean, and someone later finds an error, the points turn _negative_. so make sure of your certification! if you gather enough points, you win _a_million_dollars_! ;+) *** there are a few things you need to stipulate for such a system: 1. there is one -- and only one -- "correct" way to do a page. 2. which means there are no ambiguous guidelines in place. 3. and whitespace is significant. 4. which means there are _no_ "insignificant" diffs. 5. all diffs are reviewed, and can be challenged for correctness. 6. so when a page comes out of proofing, that page is _done_. 7. which means "postprocessing" is a largely automatic thing. *** you can discuss any aspect of this post, but what i'm seeking are any arguments people can think of _against_ free-range proofing. -bowerbird

6 7

Re: jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

and jim, it doesn't look like 5 people want to see my comparison tool, so you'll have to settle for a screenshot (with some fancy stuff deleted): > http://z-m-l.com/go/jimad/comparison-screenshot.png -bowerbird

1 0

Re: jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

for the sake of comparison, here's how i display the sitka diffs: > http://z-m-l.com/go/jimad/sitka-175diffs.html -bowerbird

1 0

Re: i'm a busy little bee
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

juliet said: > Only as easy as pie if the OCR got the page number correct. > In my experience, page numbers are very frequently misread. i'm getting ahead of myself, but yes, you must check them first. in the sample book which i'll be talking about, 4 pagenumbers were misrecognized in 108 pages, and 1 was entirely missing. and actually, the misrecognitions were on the left-bracket that preceded the pagenumber, rather than the pagenumber per se. but yes, it is true this book had atypically accurate recognition, and that pagenumbers are not infrequently misrecognized... however, it's also true there is a _huge_ amount of redundancy in pagenumbers -- they march on in a predictable sequence -- so routines can be written (and i have written a few of them) to "fill in" the missing numbers in an astonishingly accurate way... even the unnumbered pages stick out in a fairly distinctive way, since the numbering-sequence politely "steps around" them... but why don't we hold off further dialog until i do my full post? -bowerbird

1 0