you followed "the conventional wisdom" which tells coders
that they should _not_ use the file-system as their database,
but should instead use a "proper" database, typically mysql…

This wisdom is absolutely a must! the emphasis is on "[PROPER]". read below.

but for your purposes, that was a mistake, roger.

trying to funnel all of your users, and all their transactions,
through a single "gate" -- a.k.a., your mysql database -- is
_what_ complicated your code and caused the scaling issues.

what you should have done instead is to write a program that
can be dropped into a folder _with_ the files for a single book,
which then _administers_ the transactions involving those files.

Basically, you point out the main problem of the database. It is not layered and

modular. 

What most do not understand that in order to have a "proper" database for any

non-trivial task, you must add layers of abstraction in order to administer all

transactions and function needed. Without them a system will eventually bog down

and will not scale well.

As proof of concept I refer to content management systems, which do exactly

what is needed. 

regards

Keith

since most of the files are small, changes are not a big deal...
besides, the fact is that most of the files will never be changed.
even ones that _are_ changed will only undergo a few iterations.
so there is nothing that's going to overwhelm the file-system...

thus with -- maybe -- _ten_ people on a book simultaneously
(at the _most!_, and more typically probably just one to three),
and few files being changed, there _are_ no "scaling" issues...
load is distributed across separate scripts in different folders.

that was your main mistake, roger, especially vis-a-vis scaling.

[snip, snip]