- gutvol-d - lists.pglaf.org

Re: save those pagenumbers!
by Bowerbird＠aol.com 20 Mar '10

20 Mar '10

i have a reply in the works, but i won't post it until monday... so keep your minds open until then, and have a nice weekend! -bowerbird

1 0

Re: "that doesn't work"
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

jim said: > I’ve put up a new copy of the tool pgdiff > that contains an option “-smarted” which > outputs the text in a form similar to what > I think you want BB for your “smart editor” tool. i'm sure some of your users will enjoy that option, jim... as you might expect, i'll likely stick with my own tools... but yes, this new option might allow me to perfect the tool that i've built in support of your pgdiff tool, i hope. > Your suggestions work in simple cases but I think > you will find that they fail relatively spectacularly > on difficult cases, such as when performing versioning > across different editions. well, if i'm gonna fail, please let me fail "spectacularly". i compare different editions using a different technique; essentially i do a _paragraph-level_ comparison for that. it's easy enough to unwrap texts to the paragraph level. indeed, i do paragraph-level analyses in my comparisons all the time. that's how i catch the paragraphing glitches. (it's also necessary to work at the paragraph level when you're fixing spacey-quotes, as i have mentioned before.) > I also updated the example output file “BBoutput.txt” > to show the new output. great. i'll go get it this afternoon... > Again, the problem is basically the domain > you are interested in working on and the domain > I am interested in working on is very different. actually, they're not. but that's another question for another day. here today's issue is finding and fixing errors by comparing two versions which are similar... > You want a tool that catches small changes > within a line of text, and I want a tool that catches > large changes within a file. two rejoinders. first, my tools are capable of finding "large differences" if they are what exist. but, like i just said, that arena is not of much particular interest here on the p.g. listserve. second, i have -- without knowing it at first -- worked on doing comparisons between what turned out to be different editions of a book. and most of the changes were not "large" ones, but rather "small" ones, notably punctuation variations reflecting different house "styles". i discussed this particular comparison at _great_ length over on the d.p. forums, under a thread with a title like "a revolutionary method of proofing", if you're interested. > It is easy to hypothesize what the “answer” is > if you are not the one doing the work. i agree. that's why i suggested we work on actual data. i find it best if i don't bias my research by selecting the data that i work on, so i work on other people's stuff, which is why i choose that book from rfrank. however, if you want to share some data on a book of your own, one you're working on, i would be happy to look at it... > But if you are the one doing the work you rapidly find > “oops that idea doesn’t work after all!” you know, i hear a lot of people saying "that doesn't work". but usually, they're being bamboozled by some _small_ issue that can be overcome quite easily if they just try... a good example of that was yesterday, when juliet said "your renaming solution won't work because pagenumbers are often misrecognized." well, yeah, that happens, but that particular "obstacle" can be hurdled with little effort. so i invite you to bring any "doesn't work" problems to me... i like the challenge of seeing if i can make it work regardless. -bowerbird

1 0

Re: jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

jim said: > In practice what one normally sees is some weird mixture of > the two possible situations, and it isn't clear to me > which display technique is best, so so far I have > chosen the easiest approach to implement in view of the frank admission, let me make some suggestions. i believe these would make your tool's output more workable for the end-user who has to resolve the diffs, no matter _what_ method they use, including the reg-ex editor you use yourself. *** let me pull out the last 2 of the 55 anomalies i posted for you... *** here's the first: > drew their stores of { krasnia | > Jcrasnia ruiba | > ruiba } (the red some people might prefer the version as it was in your file: > stores of { krasnia | Jcrasnia ruiba | ruiba } (the red rather than showing this as a single diff, i'd present it as two... the first would be: > { krasnia | Jcrasnia }. the second would be > { ruiba | ruiba } *** here's the second example: > the trough of the watering place of { the | > the" "Jamestown," | > Jamestown,"} came to the beach. This place may be or, more in keeping with how it's displayed in your output: > place of { the | the" "Jamestown," | Jamestown,"} came to again, i would present this as two diffs... the first would be: > { the | the" } the second would be: > { "Jamestown," | Jamestown,"} *** in both of these examples, i think combining the 2 diffs into one bracket-bound item confuses the item unnecessarily, and confuses the end-user in the process, making the resolution much more difficult than it needs to be... in many of these "multiple diff" brackets, i could have my tool pull apart the various diffs, and display them appropriately... so, you know, if you think the output you are showing now is done the way you _want_ to have it done, that's your decision. but i think it will be more clear if you did it slightly differently. *** another confusion i had with your output was that there were several bracketed items that contained some separator-lines... since all of those separator-lines were standaridized by you before you ran your pgdiff, it seems to me that none of them should've been included in any of the brackets. should they? if you start bringing non-diff material into the edit process, you're asking for problems, it would seem to me, so i would rework that code to try to avoid such problems if i were you. anyway, just a few suggestions, hopefully helpful ones... :+) -bowerbird

2 1

any arguments against "free-range" proofing?
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

the d.p. proofing system locks each page to a single proofer. (there's one and only one p1 proofer, p2 proofer, and so on.) so does rfrank's roundless system; once a page has been assigned to a proofer, it's semi-difficult to even look at it. and if someone else has reproofed it _after_ that person, then the old version is stored somewhere i can't figure out, so tracking the diffs simply cannot be done by an outsider. (the d.p. system at least allows you to do that tracking, and even has a routine that will show you round-to-round diffs.) it is by analyzing these round-to-round diffs very closely that you can get a sense for how a page progresses from the initial o.c.r. to its final -- hopefully perfect -- stage... *** the question i have today is whether there is a good reason why a page needs to be assigned-and-locked to one person. is there any reason why you shouldn't allow any proofer to go and proof any page in a book? yes, it would mean that some pages might be proofed several times, but so what? that's not necessarily a _bad_ thing, is it? i'm writing code now to build my own proofing system, and i'm curious about this particular aspect. i think it would be important to inform a proofer how many previous people have proofed each specific page, so as to let that proofer choose whether to do an additional proof, but if they _want_ to do it, is there any reason to disallow it? *** partly this ties into _incentives_... most people like _finding_and_fixing_ errors, so there'll be a good incentive for people to work in the "first" proofing... but even in that first proofing, there are a lot of pages that are _already_ perfect, so there are no errors to find or fix... and in the second and third proofings, the number of errors that are left will be small, even collected over a whole book. so i feel it's very important to reward people for _certifying_ a page -- i.e., confirming that the page is indeed error-free. if i was to put this in terms of a "point" system, it'd be this: > 5 points for fixing all of the remaining errors on a page. > 4 points for doing the first "certification" of a clean page. > 3 point for doing the second "certification" of a page. > 2 point for doing the third "certification" of a page. > 1 point for fixing _some_ (but not all) errors on a page. if you certify a page clean, and someone later finds an error, the points turn _negative_. so make sure of your certification! if you gather enough points, you win _a_million_dollars_! ;+) *** there are a few things you need to stipulate for such a system: 1. there is one -- and only one -- "correct" way to do a page. 2. which means there are no ambiguous guidelines in place. 3. and whitespace is significant. 4. which means there are _no_ "insignificant" diffs. 5. all diffs are reviewed, and can be challenged for correctness. 6. so when a page comes out of proofing, that page is _done_. 7. which means "postprocessing" is a largely automatic thing. *** you can discuss any aspect of this post, but what i'm seeking are any arguments people can think of _against_ free-range proofing. -bowerbird

6 7

Re: jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

and jim, it doesn't look like 5 people want to see my comparison tool, so you'll have to settle for a screenshot (with some fancy stuff deleted): > http://z-m-l.com/go/jimad/comparison-screenshot.png -bowerbird

1 0

Re: jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

for the sake of comparison, here's how i display the sitka diffs: > http://z-m-l.com/go/jimad/sitka-175diffs.html -bowerbird

1 0

Re: i'm a busy little bee
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

juliet said: > Only as easy as pie if the OCR got the page number correct. > In my experience, page numbers are very frequently misread. i'm getting ahead of myself, but yes, you must check them first. in the sample book which i'll be talking about, 4 pagenumbers were misrecognized in 108 pages, and 1 was entirely missing. and actually, the misrecognitions were on the left-bracket that preceded the pagenumber, rather than the pagenumber per se. but yes, it is true this book had atypically accurate recognition, and that pagenumbers are not infrequently misrecognized... however, it's also true there is a _huge_ amount of redundancy in pagenumbers -- they march on in a predictable sequence -- so routines can be written (and i have written a few of them) to "fill in" the missing numbers in an astonishingly accurate way... even the unnumbered pages stick out in a fairly distinctive way, since the numbering-sequence politely "steps around" them... but why don't we hold off further dialog until i do my full post? -bowerbird

1 0

i'm a busy little bee
by Bowerbird＠aol.com 19 Mar '10

19 Mar '10

i'm a busy little bee these days... let's see... *** i'm waiting on a response from jim so i can continue to analyze his tool, and work on my support-application... *** i grabbed the "frankenstein" content from lee passey's site, and have mounted my own version of the book over here: > http://z-m-l.com/go/fpass/dofpass.pl the perl script lets you step through the pages of the book. the script shows lee's .html files as they were when grabbed. i don't wanna mess with the complications of an .html editor, so i won't be bothering to offer an edit capability on the text, at least not quite yet... i mention this because editing is one of the thorny aspects of using .html as your saved-text format. lee is using the "kupu" html-editor. too much trouble for me, and likely far too many cross-browser inconsistencies as well. but those are lee's problems to solve, not mine. good luck, lee. *** here's a little script that tells you if a word you enter is present in the dictionary that i use. i'm not sure why you'd want to know such information, but the script was developed in support of my spiffy spellchecking feature, so it's there if you _do_ have a need: > http://z-m-l.com/go/dict17577.pl *** i'm finishing up a long post on intelligent filenaming. yes, again. but there's a good angle, important enough to warrant exposition. the angle, to steal my own thunder, is that you can easily use the pagenumber information that's contained in the o.c.r. files to rename your .txt and .png files in a more-intelligent manner. if the pagenumber in file "011.txt" says that it's pagenumber 7, you'll rename "011.txt" and "011.png" to "007.txt" and "007.png". easy as pie! *** and finally, i'm still coding my online proofing system. it's grand. i love my tools... anyway, i'll probably unveil the thing next week. yeah, yeah, i know, you can hardly wait, you're so excited... ;+) -bowerbird

2 1

Re: @!@!@!@!@!@!@! Re: [SPAM] re: New Tool "pgdiff"
by Bowerbird＠aol.com 18 Mar '10

18 Mar '10

greg said: > The pglaf mailer includes a few spam filters, one > of which adds the [SPAM] string to the subject header. why? and for what purpose? which particular "filter" is it that is doing this? can it be deactivated? if not, can you tell us how to avoid setting it off? because this "filter" is doing nothing but emitting false alarms. and it's not stopping any spam, is it? -bowerbird

1 0

jim, i have some questions about pgdiff output
by Bowerbird＠aol.com 18 Mar '10

18 Mar '10

jim, here are 55 cases where your tool seems to give us more than just the 2 choices that i would expect to see... in some cases, such as the second one listed (hot springs), it's because one of the proofer's notes contained a "|" in it. you'll want to screen the input for your significant characters, i.e., any "{" or "}" or "|", and eliminate them to avoid confusion. i have some other questions as well, but let's see if the fix for this issue provides answers for those questions as well... -bowerbird > { Seattle, | > Seattle, Washington | > Washington##} [Illustration]# > Hot { Springs;[**probable printer's error. Should be ,|next: no, ; is fine] | > Springs; } each with its individual charm;# > connect the Sitka of the past, the { Novo | > Novo Ark-*angelsk | > Arkangelsk#} of the great Russian American Company# > and their fate will never be known to a { certainty.##[Footnote | > cer-taintv.##A: | > * } January { 20th.[**,?] | > 20th. 1820, | > 1820. } a letter written by the Directory at St.# > { ====sitka0-015.txt===###[Illustration: Copyright by E. W. Merrill, | > [10]####Sitka. | > ====sitka0-015.txt===###Mount | > O#Edgecumbe.] | > H####} ====sitka0-016.txt===# > He named the mountain { San | > San Jacinthus, | > Jacinthus,}# > toward the sea, Cape { del | > del Engano. | > Engano. } No one who# > 2,000 skins of the { Morski | > Morski bobrov, | > 'bo'brov, } as they# > or { Kolosh | > Kolosh Ryeku. | > Ryeku.##} On the morning of September 28th the# > { ====sitka0-030.txt===##[Illustration: Sitka in | > [24]####1805--From | > ====sitka0-030.txt===##Lisianski's | > [Blank Voyage.] | > Page]####} ====sitka0-031.txt===# > rose the town of New Archangel { (Novo | > (Novo#Arkangelsk,) | > Arkangelsk,) } and on the kekoor was built a# > valued at 450,000 { rubles.[B]##[Footnote | > rubles.f##A: | > * } The livestock taken to Sitka in 1804 consisted of "Four# > p. { 218.)]##[Footnote | > 218.)##B: | > t } Lisianski made the surveys and named the islands of the# > { yourts, | > yourts, } in which live the { kayours | > kayours } and the# > { [Footnote A: | > * } The Russian sazhen is { 7 | > I feet.] | > feet,# > { [Footnote A: | > * } These books and letters were brought by Resanof { in | > In } the# > theft in the years when there was no custodian of such { property.]###====sitka0-043.txt===###[Illustration: The Bakery and Shops of the Russians--Later the | > property.##Sitka | > [36]####Trading | > ====sitka0-043.txt===##Co.'s | > [Blank Building.] | > Page]####} ====sitka0-044.txt===# > the { dushnoi dereva | > dushnoi or | > dereva scented | > or.,@scented } wood of the# > Place of Islands { (Chasti | > (Chasti Ostrova) | > Ostrova) } is reputed# > { [Footnote | > 103.##A: | > * } Golofnin, Voyage of the Sloop "Kamchatka," in Mat. { Pt. | > Ft. } 4, p.# > { Wrangel's | > Wraiigel's } daughter--Mary." There { is | > Is } also { to | > t-> } be found: "Died,# > the church by a partition called the { Ikonastas, | > Ikonastas,#} which is ornamented with twelve { ikons, | > ikons, } or# > { repousse | > repousse } work in the true { Russian | > Eussian } style of# > { ====sitka0-066.txt=== | > [56]####[Illustration: | > ====sitka0-066.txt===###} The { Madonna.] | > Madonna.# > { ====sitka0-071.txt===###[Illustration: | > [60]####/* | > ====sitka0-071.txt===###} The Baranof Castle.# > The { U. | > IT. S[**.] | > S } Agricultural Department { building occupies | > building-occupies } the site at the# > { [Footnote A: | > * Narative[**Narrative?] | > Narative } of a Voyage Round the { World, | > World. } 1836-1842, by Captain# > Sir Edward Belcher, Vol. { 1,[**I?] | > I, } pages { 95 | > 05 } et { sen.# > { ober off | > ober offitzer | > User } who sought her hand in marriage.# > dead in one of the small drawing { rooms."[A]##[Footnote | > rooms."*##A: | > * } Frederick { Schwatka, | > Sohwatka, } the explorer, seems to have been one of# > 24th, 1896, and the time is fixed as being in the administration { of]* | > of##====sitka0-074.txt=== | > [62]####[Illustration: | > ====sitka0-074.txt===###} The Grave of the Princess { Maksoutoff.] | > Maisoutoff.# > martin from the Yukon, others { en | > en route | > route } to# > reason for their living on this distant { shore.[A]##[Footnote | > shore.*##A: | > * } Between 1821 and 1862 there were shipped by the Russian# > (Washington, Government Printing { Office).]###====sitka0-079.txt===##[Illustration: Sitka in 1860, Near the Close | > Office).##of | > [66]####the | > ====sitka0-079.txt===###Russian | > CD#Administration.] | > CO####} ====sitka0-080.txt===# > for calico and beads, blankets and { ammunition.[A] | > ammunition.*#} This market was closed by a { portcullised | > portcul-lised# > quids; fish priced according to { size[** | > size ;?] | > ? } all according to price list established# > B: | > t } Golobokoe Lake was sounded to a depth { of | > cf } 190 fathoms# > Ivan { Vasilivich | > Vasiiivich Furuhelm, | > Funihelm, } June 22, 1859, to Dec. 2, 1863.# > { ====sitka0-090.txt===###[Illustration: Sitka in 1869--During the Time of the Military | > [76]####Occupation.] | > ====sitka0-090.txt===#####} ====sitka0-091.txt===# > in the land that had so long been their { home.[C] | > home.t#} Among those who remained are the { Kashavaroffs, | > Kashavar-offs,# > { [Footnote A: | > * } The Russian soldiery were dressed { in | > In } a dark uniform, trimmed# > it down on the bayonets of the Russian { soldiery.]##[Footnote | > soldiery.##C: | > t } On December 14, { 1807, | > 1807. } the Russian ship "Czaritza," sailed for# > Russia, via London, with { 168 | > 368 } passengers. January { 1, | > I, } 1868, the# > Ex. Doc. H. R. 41st Cong. 2nd { Ses., | > Ses.. } p. 1030; Seattle { Intelligencer, | > intelligencer,# > 123; citizens by treaty, 229. { Total, | > Total,.444. 444. | > ? } Beardslee's Report, 47th# > Erussard, Ed. Doyle, George E. Pilz, Nicholas Haley, John { McKenna, | > MCKenna,#Reub | > Keub } Albertson, John Olds and { others.# > One of the traders of the town, { Caplin, said: | > Caplin,-said:#} "De Captain may go { to ---- | > to---wid wid | > his his | > tarn# > { ====sitka0-107.txt===##[**The CP failed to rotate this page correctly.][**Seems to be fixed now :-)]#[Illustration: Sitka--East on Lincoln Street--the Governor's Walk | > [92]####of | > ====sitka0-107.txt===##the | > [Blank Russians.] | > Page]####} ====sitka0-108.txt===# > { ====sitka0-110.txt===##[Illustration: Interior of Cathedral | > [94]####of | > ====sitka0-110.txt===##St. | > [Blank Michael] | > Page]####} ====sitka0-111.txt===# > { [Footnote A: | > ? } The first church in Alaska was built at { Kodiak | > Kodlak } (Paulovski) in# > towers above the bay to the height of { 3,216 | > 3,21.6#} feet. Along the river, known as the { Kolosh | > Kolosh# > is prominent the Devil's Club { (panax | > (panax horridus), | > horrid-us},# > { ====sitka0-117.txt=== | > [100]####[Illustration: | > ====sitka0-117.txt===###} Russian { Blockhouse.] | > Blockhouse.# > drew their stores of { krasnia | > Jcrasnia ruiba | > ruiba } (the red# > the trough of the watering place of { the | > the" "Jamestown," | > Jamestown,"#} came to the beach. This place may be#

1 0