roger said:
>   Even if I agreed with all the technical points,
>   there is what feels like a showstopper to me
>   that's equally if not more important.
>   Running a cooperative workflow of volunteers
>   involves motivating people. A leader of
>   such an effort can't just tell them what to do,
>   or sometimes even how to do it. They need to
>   gain their good will, evoke their interests,
>   stimulate their creativity, and communicate
>   with them in an open manner.

this is, of course, very astute.

and it reminds me of something that juliet said
in her recent post, which i will return to shortly...

but for now, i want to concentrate on this, instead:

>   I coded it myself and ran it on a non-commercial,
>   personal server. Neither the quality of the code nor
>   the bandwidth to the hardware were scalable, and it
>   wasn't my intention to run a full production site anyway.

i do believe you are mistaken about "the quality of the code",
roger, in terms of whether it could _scale_ to full production.

or rather, you're correct, but you fail to see the real problem.

you followed "the conventional wisdom" which tells coders
that they should _not_ use the file-system as their database,
but should instead use a "proper" database, typically mysql...

but for your purposes, that was a mistake, roger.

trying to funnel all of your users, and all their transactions,
through a single "gate" -- a.k.a., your mysql database -- is
_what_ complicated your code and caused the scaling issues.

what you should have done instead is to write a program that
can be dropped into a folder _with_ the files for a single book,
which then _administers_ the transactions involving those files.

since most of the files are small, changes are not a big deal...
besides, the fact is that most of the files will never be changed.
even ones that _are_ changed will only undergo a few iterations.
so there is nothing that's going to overwhelm the file-system...

thus with -- maybe -- _ten_ people on a book simultaneously
(at the _most!_, and more typically probably just one to three),
and few files being changed, there _are_ no "scaling" issues...
load is distributed across separate scripts in different folders.

that was your main mistake, roger, especially vis-a-vis scaling.

(your other big mistake, which increased your complexity load
unnecessarily, was your failure to use intelligent file-naming.)

your system can easily be changed to accommodate a site that
simultaneously digitized hundreds of books, even a thousand...

(of course, it'd be smarter to set the number to a few hundred,
and not let a new book come in until a current one is finished,
so as to avoid the d.p. situation where thousands sit undone.)

but you carried away the wrong lesson if you think your code
"won't scale", roger...

-bowerbird