Hiya All, Did I miss an outage notice? The PG server appears to be having hassles: I keep seeing "Could not connect to database server." when I try to access etexts. Thanks, P -- Help digitise public domain books: Distributed Proofreaders: http://www.pgdp.net "Preserving history one page at a time." Set free dead-tree books: http://bookcrossing.com/referral/servalan
Pauline wrote:
I keep seeing "Could not connect to database server." when I try to access etexts.
The PG site is just too popular. The database cannot serve more than ~30 requests at one time. Try accessing the site in the "off" hours. Rush hours are 9 AM - 18 PM EST. -- Marcello Perathoner webmaster@gutenberg.org
The PG site is just too popular. The database cannot serve more than ~30 requests at one time. I don't think there's any such thing as PG being too popular. :) But it does sound as if the DB is too anemic for the current (and future)
On Sunday 06 March 2005 01:57 pm, Marcello Perathoner wrote: popularity.
On Sun, 6 Mar 2005 15:28:42 -0500, D Garcia <donovan@abs.net> wrote: | On Sunday 06 March 2005 01:57 pm, Marcello Perathoner wrote: | > The PG site is just too popular. The database cannot serve more than ~30 | > requests at one time. | I don't think there's any such thing as PG being too popular. :) Agreed. Is there no way of transferring requests which cannot be handled to a mirror site? Cheaper than a bigger server? -- Dave F
Usually, it's cheaper and easier to use a bigger database server then try to redirect the requests to another site. The only time you'd want to redirect to another site would be in the event the primary site was down. Disclaimer: I'm a sysadmin at a hosting company. -brandon Dave Fawthrop wrote:
On Sun, 6 Mar 2005 15:28:42 -0500, D Garcia <donovan@abs.net> wrote:
| On Sunday 06 March 2005 01:57 pm, Marcello Perathoner wrote: | > The PG site is just too popular. The database cannot serve more than ~30 | > requests at one time. | I don't think there's any such thing as PG being too popular. :)
Agreed. Is there no way of transferring requests which cannot be handled to a mirror site? Cheaper than a bigger server?
Marcello Perathoner wrote:
Pauline wrote:
I keep seeing "Could not connect to database server." when I try to access etexts.
The PG site is just too popular. The database cannot serve more than ~30 requests at one time.
Try accessing the site in the "off" hours. Rush hours are 9 AM - 18 PM EST.
I've been recommending PG texts & now have a bunch of replies saying PG is unusable due to this problem. :( If the server cannot be configured to cope with increased load, please at least consider changing the error message to something more useful for the user. e.g. "Project Gutenberg is too busy at the moment to handle your request, please try again later. Current slack times are 18.00->8.00 EST." Thanks, P -- Help digitise public domain books: Distributed Proofreaders: http://www.pgdp.net "Preserving history one page at a time." Set free dead-tree books: http://bookcrossing.com/referral/servalan
Hi. A slight workaround for this is to refresh the page. In Internet Explorer, Control + F5 does the trick. I got that same error and it refreshed fine.
On Sun, Mar 06, 2005 at 07:57:41PM +0100, Marcello Perathoner wrote:
Pauline wrote:
I keep seeing "Could not connect to database server." when I try to access etexts.
The PG site is just too popular. The database cannot serve more than ~30 requests at one time.
Try accessing the site in the "off" hours. Rush hours are 9 AM - 18 PM EST.
Marcello, can you tell me what it would take to grow our capacity to handle hits? I know you're also looking at Web site mirrors (I can supply some sites for this, BTW). But if you could come up with some recommendations for what it would take for iBiblio to dramatically grow our capacity, I can try to put something together for them. 30 simultaneous requests to PostgreSQL does not seem like a whole lot, so I'm assuming that contention for resources with other hosted sites is the main problem. It would be nice to do better. I know that iBiblio claims network bandwidth is not an issue, but possibly we need to look at the whole system. Thanks for any ideas you (or others) can provide. -- Greg
Marcello, Could connection pooling fix this? Maybe combined with more concurrent connections to the database server? I'm not sure how big the database box is though. -brandon Greg Newby wrote:
On Sun, Mar 06, 2005 at 07:57:41PM +0100, Marcello Perathoner wrote:
Pauline wrote:
I keep seeing "Could not connect to database server." when I try to access etexts.
The PG site is just too popular. The database cannot serve more than ~30 requests at one time.
Try accessing the site in the "off" hours. Rush hours are 9 AM - 18 PM EST.
Marcello, can you tell me what it would take to grow our capacity to handle hits? I know you're also looking at Web site mirrors (I can supply some sites for this, BTW). But if you could come up with some recommendations for what it would take for iBiblio to dramatically grow our capacity, I can try to put something together for them.
30 simultaneous requests to PostgreSQL does not seem like a whole lot, so I'm assuming that contention for resources with other hosted sites is the main problem. It would be nice to do better.
I know that iBiblio claims network bandwidth is not an issue, but possibly we need to look at the whole system.
Thanks for any ideas you (or others) can provide. -- Greg _______________________________________________ gutvol-d mailing list gutvol-d@lists.pglaf.org http://lists.pglaf.org/listinfo.cgi/gutvol-d
Brandon Galbraith wrote:
Could connection pooling fix this? Maybe combined with more concurrent connections to the database server? I'm not sure how big the database box is though.
I'm not sure why the limit is so low. Maybe the folks at ibiblio have a good reason for it. We have to see what increasing the number of concurrent connections does to query response time. The database box is a dual-processor IBM whatever. I can look up the specs if you want. But this box and his brother are serving all sites hosted at ibiblio. Many of those sites are build with CMS and thus very database intensive. -- Marcello Perathoner webmaster@gutenberg.org
Marcello, Maybe it's time to talk about doing a master/slave replication configuration of postgres to handle the database load. If you want, contact me off list and I'd be willing to help any way I can. -brandon Marcello Perathoner wrote:
Brandon Galbraith wrote:
Could connection pooling fix this? Maybe combined with more concurrent connections to the database server? I'm not sure how big the database box is though.
I'm not sure why the limit is so low. Maybe the folks at ibiblio have a good reason for it. We have to see what increasing the number of concurrent connections does to query response time.
The database box is a dual-processor IBM whatever. I can look up the specs if you want. But this box and his brother are serving all sites hosted at ibiblio. Many of those sites are build with CMS and thus very database intensive.
Brandon Galbraith wrote:
Maybe it's time to talk about doing a master/slave replication configuration of postgres to handle the database load. If you want, contact me off list and I'd be willing to help any way I can.
First ibiblio will have to host another database server and dedicate that server to PG. A dedicated database server would probably solve our problem. But we'll have to talk the ibiblio people into doing that for us. It means money, more maintenance hassles and maybe problems from other sites hosted at ibiblio, who want a faster server too. OTOH a dedicated squid for PG would help too and be much cheaper. Replication to an external server will not help much, as the latency will be too big. Our current database server is: IBM Netfinity 6000R Quad Xeon PIII 700 MHz 4.5 GB RAM 108 GB storage iblinux But this one is shared with other sites hosted at ibiblio. See also: http://www.ibiblio.org/systems/hardware-details.html -- Marcello Perathoner webmaster@gutenberg.org
On Mon, Mar 07, 2005 at 08:55:53PM +0100, Marcello Perathoner wrote:
Brandon Galbraith wrote:
Maybe it's time to talk about doing a master/slave replication configuration of postgres to handle the database load. If you want, contact me off list and I'd be willing to help any way I can.
First ibiblio will have to host another database server and dedicate that server to PG. A dedicated database server would probably solve our problem. But we'll have to talk the ibiblio people into doing that for us. It means money, more maintenance hassles and maybe problems from other sites hosted at ibiblio, who want a faster server too.
OTOH a dedicated squid for PG would help too and be much cheaper.
Replication to an external server will not help much, as the latency will be too big.
Our current database server is:
IBM Netfinity 6000R Quad Xeon PIII 700 MHz 4.5 GB RAM 108 GB storage iblinux
But this one is shared with other sites hosted at ibiblio.
Thanks, and also for the info about squids. I will pitch ibiblio on the idea of a new system - we'll see what the response is. A current top-end quad Xeon system from someplace like asaservers.com is ~$20,000, which is a little steep for PG to pay for. -- Greg
See also:
http://www.ibiblio.org/systems/hardware-details.html
-- Marcello Perathoner webmaster@gutenberg.org
_______________________________________________ gutvol-d mailing list gutvol-d@lists.pglaf.org http://lists.pglaf.org/listinfo.cgi/gutvol-d
Migrating to MySQL might help -- and it's easier to replicate/mirror on the fly. -- James Greg Newby wrote:
On Sun, Mar 06, 2005 at 07:57:41PM +0100, Marcello Perathoner wrote:
Pauline wrote:
I keep seeing "Could not connect to database server." when I try to access etexts.
The PG site is just too popular. The database cannot serve more than ~30 requests at one time.
Try accessing the site in the "off" hours. Rush hours are 9 AM - 18 PM EST.
Marcello, can you tell me what it would take to grow our capacity to handle hits? I know you're also looking at Web site mirrors (I can supply some sites for this, BTW). But if you could come up with some recommendations for what it would take for iBiblio to dramatically grow our capacity, I can try to put something together for them.
30 simultaneous requests to PostgreSQL does not seem like a whole lot, so I'm assuming that contention for resources with other hosted sites is the main problem. It would be nice to do better.
I know that iBiblio claims network bandwidth is not an issue, but possibly we need to look at the whole system.
Thanks for any ideas you (or others) can provide. -- Greg _______________________________________________ gutvol-d mailing list gutvol-d@lists.pglaf.org http://lists.pglaf.org/listinfo.cgi/gutvol-d
There's also clustering available for Postgresql, which might be easier than migrating to MySQL. Either way, it would probably take more human resource time than throwing hardware at it (for example, a dual Opteron system with 4 GB RAM 4 250 GB SATA drive in 10 RAID for about $3300, which might be overkill). James Linden writes:
Migrating to MySQL might help -- and it's easier to replicate/mirror on the fly.
-- James
Greg Newby wrote:
On Sun, Mar 06, 2005 at 07:57:41PM +0100, Marcello Perathoner wrote:
Pauline wrote:
I keep seeing "Could not connect to database server." when I try to access etexts.
The PG site is just too popular. The database cannot serve more than ~30 requests at one time.
Try accessing the site in the "off" hours. Rush hours are 9 AM - 18 PM EST.
Marcello, can you tell me what it would take to grow our capacity to handle hits? I know you're also looking at Web site mirrors (I can supply some sites for this, BTW). But if you could come up with some recommendations for what it would take for iBiblio to dramatically grow our capacity, I can try to put something together for them.
30 simultaneous requests to PostgreSQL does not seem like a whole lot, so I'm assuming that contention for resources with other hosted sites is the main problem. It would be nice to do better.
I know that iBiblio claims network bandwidth is not an issue, but possibly we need to look at the whole system.
Thanks for any ideas you (or others) can provide.
Bruce Albrecht wrote:
There's also clustering available for Postgresql, which might be easier than migrating to MySQL. Either way, it would probably take more human resource time than throwing hardware at it (for example, a dual Opteron system with 4 GB RAM 4 250 GB SATA drive in 10 RAID for about $3300, which might be overkill).
what is the number of connections for postmaster (-N). You may just need to up this value.
James Linden wrote:
Migrating to MySQL might help -- and it's easier to replicate/mirror on the fly.
Yuck! MySQL is just a glorified file system with an SQL interface. They barely got transactions working. They still can't do referential integrity, views and triggers. And if you happen to need transactions, they only work with the InnoDB backend, which is slower than Postgres. Postgres replicates very well. Just where should we replicate to? -- Marcello Perathoner webmaster@gutenberg.org
Greg Newby wrote:
Marcello, can you tell me what it would take to grow our capacity to handle hits? I know you're also looking at Web site mirrors (I can supply some sites for this, BTW). But if you could come up with some recommendations for what it would take for iBiblio to dramatically grow our capacity, I can try to put something together for them.
We have doubled our page hits over the last year. We are now serving nearly 200.000 pages a day. Just recently we became a top 5000 internet site. See Alexa stats starting at: http://www.alexa.com/data/details/traffic_details?range=3m&size=large&compare_sites=gutenberg.net,promo.net&y=t&url=gutenberg.org To handle the ever increasing load we could implement one of the following solutions: 1) An array of on-site squids at ibiblio. But ibiblio isn't adding squids for the vhosted sites. At least that's what I was told. 2) Make ibiblio throw more hardware at us (all hosted sites). This may not be possible with the limited budget. They recently got a faster file server. 3) One or more dedicated squids for PG co-located at ibiblio. (Make ibiblio pay for the bandwidth.) Somebody had to donate us a server. Needs fast disks, lots of ram, average cpu, linux, ssh. 4) Big time solution. A hierarchy of squids distributed around the world. We would have a squid hierarchy like this: www.gutenberg.org (apache) + us1.cache.gutenberg.org (squid) + us2.cache.gutenberg.org (squid) + au.cache.gutenberg.org (squid) + eu.cache.gutenberg.org (squid) + de.cache.gutenberg.org (squid) + en.cache.gutenberg.org (squid) + fr.cache.gutenberg.org (squid) To do that we need squids 2.5 with the rproxy patch. I'm still exploring that solution, but if anybody has any experience please chime in. We need service providers donate us (or co-locate our) servers and donate the bandwidth. Also we need to explore the legal implications of offering PG services outside the US. The PG web site w/o file downloads averages a 5 GB of traffic / day. (The file downloads are 100 GB / day, but we ain't going to thrash the squids with the files.)
30 simultaneous requests to PostgreSQL does not seem like a whole lot, so I'm assuming that contention for resources with other hosted sites is the main problem. It would be nice to do better.
I just asked ibiblio to double that. I'm not sure why the limit is so low. -- Marcello Perathoner webmaster@gutenberg.org
participants (10)
-
Brandon Galbraith -
Bruce Albrecht -
D Garcia -
Dave Fawthrop -
Greg Newby -
James Linden -
Kevin Handy -
Marcello Perathoner -
Pauline -
Tony Baechler