70 votes

Resource Shortage on the Daily Paul - (Potential Nuclear Option)

Friends!

Our little community has made the Daily Paul the center of liberty activities on the web. We're close to being in the top 1,000 of US websites, which is no small accomplishment!

On the night of the Iowa Caucuses, the site got so overwhelmed that it crashed, and when it came back up, I had to put it into read only mode. Which was no fun for anyone, least of all me.

I saw the threads telling me that if I need more money to upgrade the servers, that I should do a chipin. I want to thank you all so much, and I know that a chipin for such a purpose would be filled quickly. But if only it were so simple!

Hardware is not the problem. We've got two big, beefy servers that are running at about 20% of capacity. The issue is how to best utilize the hardware resources we have, and that is a complicated process. There are hidden, mysterious bottlenecks - ghosts in the machine, if you will. Identifying them is only half the battle; then comes fixing them.

This time around, depending on how things go, I have another option - a force logout of everyone on the site.

IN THE MEAN TIME, as I have mentioned before, please log out of the site if you're not commenting. If you surfing the site logged in, it puts a much bigger tax on the server than if you're not. It seems that 750 people logged in is a number that we can handle. Beyond that, things start to slow down for everyone. Anonymous visitors (i.e. people who aren't logged in) are served a cached HTML file, which is much, much faster and much, much easier on the server

As a stopgap measure, if things start to bog down, I have an option to log everyone out simultaneously. The Daily Paul's nuclear option. This will get rid of a lot of sessions of people who are logged in and then walked away. Of course you're free to log right back in. But if you were just lurking while signed in, you might want to consider reading the site anonymously.

Also, if you're compulsively hitting Ctrl+R (refresh) every 2 seconds while logged in ... let's just say you're not part of the solution.

Thank you everyone for your cooperation and understanding. We have dedicated team working on this, and we all owe them a debt of gratitude for keeping the site running smoothly. Special thanks to BigMike and the other mods on content, Marc on network/hardware, Jon on JavaScripts, and Jaspreet on development and admin.

And of course you, the Daily Paul Community for reading, participating, sharing and making the Daily Paul what it is.

Thank you.

Michael Nystrom



Trending on the Web

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

You're the driver

we're along for the ride (in it to win it )
and a big thanks are in order for that magical Chauffeur
Nystrom----clarity is key

During times of very heavy

During times of very heavy activity, could the chat function be sent over to one of the better designed free online streaming chat sites? I understand some of them can be embedded.

Fallout

As for Drupal performance, I am assuming you have a php cacher running full duty, so the webserver does not need to rebuild the php files each reload?

I would hope you have drupal cache turned on, and are not rebuilding your theme each reload or anything.

Do you have CSS aggregation module installed and configured?

How big is the db? You could memcache the whole thing, which speeds up performance with lots of DB IO.

Think of using a reverse proxy such as varnish? How about lighttpd for serving static files? It looks like most content here is dynamic, so not sure how much that would help.

Just some ideas. :)

Enable anonymous commenting

Enable anonymous commenting

That would allow saboteurs an

That would allow saboteurs an irresistible opportunity to wreak havoc.

Problem

Michael, my husband works for Akamai and would be able to direct you towards a cheap and easy fix. let me know if you want to contact him. Thanks so much for spreading the message of Dr. Paul! Huge fan of the site and visit it multiple times a day.

LStrick

The best bad news.

I think this is the best bad news sorta. lol, the bad news is that the website just cant handle that many people logged in at a time and we have to log out, but the good side of it is that were being OVERWHELMED with people and that, its a great thing. :D

EDIT:

logging out now.. :P

Got it. Will stay logged out unless posting.

I recall that heavy traffic night - it was too bogged down to even log me out. Tried it twice but the process never completed. Will log out now and stay that way.

Thanks Michael

login-logout

maybe a logout button would help.. I am not a techie, but that seems reasonable, then we could log out... except when posting..
thanks for all your hard work
mel

Live free or DIE!

10-4 will do

I couldn't get on during the debate and I figured you were getting slammed.

THIS IS AWESOME!!

over and out
FOR NOW!!!

How many servers does

How many servers does dailypaul run on?

logging in to say...

your post made me chuckle Michael...you put things so sweetly! I'll log out now, lol. It's good to know about this so thanks for your post. oh, and Congrats for the popularity of the site. Why am I not surprised? Freedom is popular!

Cloud Solution

Why not move the servers to a cloud platform that lets you increase the resource capacity by demand? Trying to keep up with the demand with your own hardware seems like a never ending battle. Imagine if Paul gets the nomination. We'll have 40,000,000 people making their way to Daily Paul.

Drupal and the Cloud...

don't mix. And there is no time to completely redesign DP...

.
~wobbles but doesn't fall down~

Rarely post. mostly surf.

And I would be happy to surf more logged out. However. I'm usually viewing from an Android device. I've upgraded here because I enjoy the site and want to be supportive. It also allows me to view without the ads.

The ability to view without the ads is a huge deal for me.
Any suggestions?

I hope

this thread highlights the tremendous work and thought that goes on behind the scenes to keep the DP running smoothly.

Watching a sailboat from afar looks like a graceful smooth ride, but on closer inspection you see people on deck frantically cranking the wenches, checking the sails, and navigating the waves and wind gusts of uncertainty.

Some folks think you can just buy a domain name from godaddy, flip a switch, and all the lights come on.

Reading through some of the comments below should dispel any of those notions.

Another misconception is that Michael makes a lot of money from this website. Out of the THOUSANDS that visit here, only a microscopic handful have a monthly subscription. (around 50 people last I heard) Much of the ad space is donated by Michael, to causes he wants to help.

I also hope that those who TRULY appreciate what this site has done not only for them personally, but to advance our cause, will make a commitment of 10-15 bucks a month to make sure Michael has the resources he needs just to operate.
It's a small sacrifice for something that has been so dear to many of us, in so many ways.

Please consider tithing a small amount on a monthly basis. I'm not exactly rolling in cash either, but still donate 10 bucks a month. Hopefully you can as well.
http://www.dailypaul.com/donate

Thanks to everyone who volunteers their time, expertise and resources to bring something that many (including myself) sometimes take for granted.

Thanks especially to Michael, who has been very supportive of me and others behind the scenes through some pretty difficult times. He took the time to write emails and check on me when I was at a very uncertain place in my life. I'm truly grateful for his friendship. Just like Dr. Paul, he is the genuine article.

If it weren't for the Daily Paul, I would have never met some of the people I have, and never made a connection with the Paul family here in Texas. This has already proven to be fruitful.

Cranking the wenches?

That sounds painful or at least disrespectful.

I'm no sailor, but I think they do other things with the wenches and save the cranking for the winches.

Ahh....8)

Thanks for the correction. I am a former sailor, and should have known how to spell that.

Those wenches CAN get pretty cranky especially if you don't keep them wrapped in the sheets (ropes).

I still haven't been able to do an appleseed shoot. I'm looking forward to hopefully going in the spring. Providing we are not in the middle of WWIII.

Hope all is well.

Debbie's picture

Thank you, will do.

Thank you, will do.

Debbie

I am sure it has already been thought of

But how about an auto log out for inactivity? Banks use it on thier websites. If a person fails to post in 5 minutes, they are automatically logged out?

Sure, but maybe 15 minutes is

Sure, but maybe 15 minutes is better?

Usually commenting happens after reading and article or watching a video.

Might deter activity to log out too quickly/often.

Never occurred to me!

I can log in whenever I want to comment, but log out the rest of the time - just never thought that would help you.
Please remind folks about this more often.
Thanks for all the great years of DailyPaul and many many more to come.
By the way, paypal and me don't mix, can I send you a check??

ytc's picture

I found that using paypal as a member is a major pain, but

using it as a guest is far less of a headache. (As a member they keep on demanding us to give our bank acc #s to that unstable system, which double crossed Wikileaks!)

Note to the FBI

What Michael is saying about the "nuclear option" is metaphorical, not literal. Any "force" he applies is within his rights as the site owner.

What do you think? http://consequeries.com/

ytc's picture

Roger, Michael N! We'll log out during the debate.

We, too, wish to send our "special thanks to BigMike and the other mods on content, Marc on network/hardware, Jon on JavaScripts, and Jaspreet on development and admin".

This site truly induces mental, emotional and spiritual GROWTH among Liberty lovers. . . within ourselves and as the movement.

Heartfelt kudos to Michael N for harnessing our power. . . and, of course, to Dr. Ron Paul for inspiring us into action and excellence.

Could we get an IRC going for

Could we get an IRC going for refuge, while the site is under load? Besides that, IRC beats the chat function on the DP.

We can..

do that on our own... :)

.
~wobbles but doesn't fall down~

Maybe have an IRC chat link

Maybe have an IRC chat link on the DP somewhere?. Here's an existing one:
irc://irc.servercentral.net/RonPaul

Otherwise for login/logout just set a limit. No? I'm sure there's a way to get Drupal to do that.

Sorry, I just looked and I was logged on. Opps!

Now I will always be logged off unless I must comment.
I hope this will help Daily Paul.
If I understand the problem. If not please someone explain it better for us lovers of Daily Paul.

After reading everything/comments

Now I'm really wondering why you aren't with SquareSpace. What site admin wants to ask his users to log out of the site? I'm quite confident this would never happen with square space. and to be spending more than just your time - a team of people's time...that seems really counter productive. especially when you can have unlimited bandwidth, unlimited users, unlimited pages, unlimited editors & more for just $40/mo on the monthly plan and not ever again have to worry about the site crashing.

and no offense intended but i think the forums are better too.

love what you do...just trying to help make it easier for you and more useful to us.