Our server is stretched, what spec to go for, how do they work?

[jaydash]

I've been talking to EVA recently about setting up our forum but we really need a new server. Basically, our site is on a shared server which is something like 900 P3, 1Gb RAM, SCSI. It is with a Top UK host and as far as I'm aware, there is literally just a few sites on it.

However, last night we had 3000 people hit the site in an hour (1.7GB served all day) and it all-but locked up. Today, we've had 20,000 unique visitors and served 3.5Gb of data. Again, things are not fast with around 2500 to 3000 people per hour visiting the site. It's running at a crawl. I don't know what ratio of people are using the forum at this time.

So, what can we do. I was thinking of a P3 1Ghz, 1Gb ram, Scsi 160 dedicated server but frankly, that's roughly what we're on now and I'm sure the other sites on the server are MUCH smaller and less busy, so will it make a difference? Is that enough power/ram etc. or do we need more, how do you know how much you need? Are all connections the same, what do we need to ask for with the host?

That's why I was looking for a resource on how servers work. What you need ram for, what you need fast hard disk for etc.

If anyone can give advice on what spec or where to ask these questions, I'd be greatful. I thought that there might be a forum dedicated to servers, but I haven;t found it so far.

Thanks,

JD

[eva2000]

is this new traffic level the future anticipated level of traffic for your forum/site ? if it is you're definitely needing a dedicated server for yourself

20,000 visitors/day = 600,000 visitors per month
3,000 visitors/hr = 2.16 million visitors per month

are this hits to predominantly dynamic (php/mysql) or static web pages (normal html pages) ?

[jaydash]

The website itself consists of static pages and today seems reasonably okay speed-wise. However, the (dynamic) forum is desperately slow and I am finding myself double-clicking to get threads up and get things moving etc.

A can't work it out exactly, but a quick look at the stats for today so far say:

73k pages served, possibly around 12k to the forum? 7500 *site* visitors so far. I'm not sure if this helps.

Yes that is anticipated level for the near future. Well, hopefully, the 20,000 unique visitors per day that is.

Cheers

[eva2000]

is the forum /site only one going on the planned dedicated server ?

i know for sure you'll be needing at least 1 x dual cpu server with at least 1.5 - 2gb of ram and 2 scsi drives (1 for mysql)

[jaydash]

Yes it's the only site. What about 2 x tualatin 1.1Ghz?. You say about about ram, but what uses that, the site or the forum? This is the stuff I'm trying to understand. I think we are going to have to stick with the 1 x 10k Seagate SCSI for now, will that do?

Regards,

[eva2000]

yeah 1 x 10k scsi drive will do

the ram is for mysql mainly so yes for your forums

dual p3 tualatin 1.13 with 512k L2 cache would handle nicely i'd think

[jaydash]

Thanks Eva. They had already built the aforementioned spec in anticipation of me buying it I think, but the board is a 2 x tualitin board. So, I am assuming, looking at retail prices, that taking out the P3 1Ghz and adding a 2nd 1.1 tualatin, will up the price by about £150.

Not sure how much ram is, it only uses PC133 I think. So, another 500Mb? The thing is, they weredoing a deal on the server so were using 4 x 256, but now I guess I need 2 x 512 and 2 x 256. I understand the 412's are more.

Thanks for your advice.

[jaydash]

Site is limping along now...Host threw a wobbler and has possibly limited the (previously unlimited) bandwidth.

2000 unique visitors +5000 returning per day, 160,000 page impressions per day. Have *had* to switch the forum off.

New dedicated server ordered: Dual P3 1.1Ghz Tualatin 512k, 1.5Gb ECC PC133 Ram, 18Gb Seagate 10k Cheetah SCSI 160.

Amazing deal from rackservers.com (if anyone enquires in the UK, please mention me [Jason] as I really screwed them for a deal LOL).

Ram came from an advertising deal with Crucial... oh, this post is turning into an advert! Just letting others know who I used really, in case it helps.

Anyway, what software do you recommend? i.e. what version of Red hat, mysql etc.

Regards,

JD

[eva2000]

hmm i not sure how well Red Hat 7.3 holds up so either RH 7.2 or 7.3 with latest 2.4.x kernel

mysql-max 3.23.39a compiled with berkley and innobase database support

PHP 4.1.2 with at least gd and zlib support

PHP accelerator 1.3.1 (make sure to remove zend optimiser if you have it)

[rylin]

What you should really try, if you have someone who's really into linux/other unices, is to create a ramdrive approximately a hundred meg or so bigger than your current database files and make mysql read/write onto that
(you'll probably need that linux guru friend to set everything up properly ;)

of course, this means that when the machine reboots / crashes, data on the damdrive is lost, so backing up the db to the harddrive every five minutes or so could be a good idea

you will most likely gain ****loads of performance from this, as it will mean _all_ the mysql data is in memory (eg lightning fast, compared to harddrive), and you won't put unneccesary load on the cpus from reading from ram (cpu will still be used to actually process the query though)

in general, what you need to do if you go for this approach is to:

* change the default ramdisk size
* create a new ramdisk & mount it
* create a script that copies over the database files to ramdisk
* create a script that copies over the database from ramdisk to harddrive
* add the first script to the list of scripts run at boot time (along with hooking up the ramdisk)
* add the second script to crontab, and make it run every so often
* pray and hope things don't break

we run a fairly big forum, i'd say we get around 15 - 30k pvs per day on the forums alone (haven't checked for quite a few months though, so i could be wrong ;)), and right now we have load average: 1.03, 1.00, 1.05

the machine being used is a dual p3 933, using two 18gb ibm scsi drives (10k rpm)

the linux dist on that is rather heavily tweaked & compiled for our set of hardware though, so on a standard binary dist i'd guess the load would be around 2.5 - 3.0 during the same ammount of traffic

anyway, i haven't run that particular database in ram yet, but i have tested it with other projects, and it works pretty well.

just my $0.02 ;)

[alexi]

what is your max simoultaneous users online? If it is over 8 or 9 100 I doubt one server is going to do it!

[jaydash]

Hi alexi,

Do you mean to the site in general or just the board?

[alexi]

well it's going to depend what else is running on your site and how much load it creates. I know that when we were on one server, dedicated to just the boards, we were ok untill about 5 or 600 users, past 800 it would really slow down, and after a 1000? forget about it. Now we have 2 servers for just the boards and life is good again

[jaydash]

Well, I guess I don't know the figures. All I know is site was getting 3000 visitors an hour of late. Of that, some would have been using the forum, but I imagine it's only 100 or so at the same time using the forum? I'm not sure, the site is mainly about the news, not the forum. Although the forum *is* important.

I guess we'll just have to see how things go and I will get proper monitoring sorted with the new server. God help us if we need another one 'cos the piggy bank is empty now!

[alexi]

well if all you are doing is serving up static pages on the rest of the site.. that's not a lot of load! You will have to see how the combination works out. With a dedicated server you will be able to use the top command etc and it will become pretty clear very quickly what is going on.