Gzip and Mod_gzip clarifications needed please?

[Kathy]

Hi guys,

this is something I've probably asked in part before but i'm concerned enough I'd like to revisit the info and make sure i'm doing all I can for the server and bandwidth issues.

Prior to the end of February, I was running my VB on my server using the gzip option in the vb cpanel. My bandwidth each month was less than 100 gig a month. (flashback a year earlier...we were way up over 130 gig and then ChrisS installed/fixed our server to use gzip in the vb cpanel and saved us lots of bandwidth. We dropped down to 80 - 100 gig per month at that point)

At the end of February a new hard drive was installed on our same server, replacing the old. Our files were reconfigured to use PLESK (we didn't have it on the last hard drive) and so all the file config had to be done. Mod_gzip was installed and wah-la...we are gzipped without enabling the gzip in the admin cp of vbulletin.

In fact for a while, I had it enabled in the vbulletin cpanel and the pages acted weird, sometimes looping and not loading and finishing at all. Once I understood that mod_gzip worked without enabling in vb, I turned it off in the vb. (For those two days I enabled it in the VB cp, the bandwith was normal=3 gig a day)

However....my bandwidth for the same traffic amounts pre hard drive...is 50% more than before we installed.

Instead of using less than 100 gig a month, we are using 5+ gig a day so that at the end of our first month we will be using 150 gig.

This may not sound like a big deal to anyone, but this is a huge increase in costs because we needed a larger hard drive.

Even my "out go" from my server has doubled from less than .5 gig a day to over 1.

Any ideas? Does mod_gzip use more bandwidth? What good is using mod_gzip if there is no help with saving bandwidth?

Which is why this is so absurd to me. It feels like I'm not saving bandwidth one bit. We are using more bandwidth than before, pre-Chris Scheiber days when we didn't use gzip at all...and used over 130 gig a month without gzip.

So what's the deal? How can I test the gzip savings? I've tested the site on the url where gzipped is tested...and we are gzipped....can't this be tweaked or tested?

thanks for your help and input. :D

[Kathy]

Okie...and here is another question...and this may explain it:

mod_gzip is enabled in apache....

and zlib is enabled in php....

are these both compression tools on the server and should one of them be disabled??

Could the two be working against each other causing the higher server loads and bandwidth usage?

here is my php page... http://www.hystersisters.com/phpinfo.php

I've read that only one compression tool should be used:

Choices:

gzip using the vbulletin cp
zlib in php
or
mod_gzip.

True? If so...I need to disable my zlib?

[HostCom]

Hm, I was wondering the same thing.

I have to use Gzip in the CP :(


mod_gzip isn't installed because my host says I need to pay 15.00 for them to install it? It's stupid anyways. Doesn't take long, plus it isn't that hard.

[ChanServ]

Hm, I was wondering the same thing.

I have to use Gzip in the CP :(


mod_gzip isn't installed because my host says I need to pay 15.00 for them to install it? It's stupid anyways. Doesn't take long, plus it isn't that hard.

$15.00 to recompile o.o meh, I do it for free on my servers.

I don't have that great of bw troubles so i dont really need g_zip.

[HostCom]

Yes, from www.hostdime.com


I've payed them alot of money already and they still want me to pay for a dumb recompile :(

[eva2000]

Hi guys,

this is something I've probably asked in part before but i'm concerned enough I'd like to revisit the info and make sure i'm doing all I can for the server and bandwidth issues.

Prior to the end of February, I was running my VB on my server using the gzip option in the vb cpanel. My bandwidth each month was less than 100 gig a month. (flashback a year earlier...we were way up over 130 gig and then ChrisS installed/fixed our server to use gzip in the vb cpanel and saved us lots of bandwidth. We dropped down to 80 - 100 gig per month at that point)

At the end of February a new hard drive was installed on our same server, replacing the old. Our files were reconfigured to use PLESK (we didn't have it on the last hard drive) and so all the file config had to be done. Mod_gzip was installed and wah-la...we are gzipped without enabling the gzip in the admin cp of vbulletin.

In fact for a while, I had it enabled in the vbulletin cpanel and the pages acted weird, sometimes looping and not loading and finishing at all. Once I understood that mod_gzip worked without enabling in vb, I turned it off in the vb. (For those two days I enabled it in the VB cp, the bandwith was normal=3 gig a day)

However....my bandwidth for the same traffic amounts pre hard drive...is 50% more than before we installed.

Instead of using less than 100 gig a month, we are using 5+ gig a day so that at the end of our first month we will be using 150 gig.

This may not sound like a big deal to anyone, but this is a huge increase in costs because we needed a larger hard drive.

Even my "out go" from my server has doubled from less than .5 gig a day to over 1.

Any ideas? Does mod_gzip use more bandwidth? What good is using mod_gzip if there is no help with saving bandwidth?

Which is why this is so absurd to me. It feels like I'm not saving bandwidth one bit. We are using more bandwidth than before, pre-Chris Scheiber days when we didn't use gzip at all...and used over 130 gig a month without gzip.

So what's the deal? How can I test the gzip savings? I've tested the site on the url where gzipped is tested...and we are gzipped....can't this be tweaked or tested?

thanks for your help and input. :D
didn't you get my email ? when i looked into your server you logs showed ALOT of googlebots hitting your server!

[eva2000]

Okie...and here is another question...and this may explain it:

mod_gzip is enabled in apache....

and zlib is enabled in php....

are these both compression tools on the server and should one of them be disabled??

Could the two be working against each other causing the higher server loads and bandwidth usage?

here is my php page... http://www.hystersisters.com/phpinfo.php

I've read that only one compression tool should be used:

Choices:

gzip using the vbulletin cp
zlib in php
or
mod_gzip.

True? If so...I need to disable my zlib?
zlib is fine..

it should be

either

mod_gzip
or
vB gzip
or
PHP's output_handler which isn't enabled in your phpinfo.php report

[Kathy]

Hi George,

Yes, i got your email....but this doesn't account for "outgo" doubling on the day I enabled mod_zip and the rest of the hits.

I've checked and my "hits" were in the same ball park as before. I've always had google bot hitting my site.

My hits are similar, visitors the same. All the stats are similar with the exception of bandwidth...more being used to process the same hits/files/visitors/traffic.

I don't get it. :(

http://traffic.ethos.net/cat2924-49-5-5-month2.html

Here are some bandwidth stats...this is for the month of Feb.

All normal stats for my server (see January also) until the 24th when they downloaded my files...and installed the new hard drive. Then I enabled gzip on the vb cp. It was quirky so I turned it off on the 26th -end of month. Notice the jump in bandwidth?

I then turned on the vb cp again for first of March:

http://traffic.ethos.net/cat2924-49-5-5-days.html

And for two days as the vb cp had "enabled gzip" we had normal bandwidth but complaints of pages looping, reloading over and over.

I turned off the vb cp again on March 3....and the bandwidth stats jumped up beyond normal again.

Compare this month with last month....and you'll see that the jump is enormous considering the non-difference in hits/files.

I lost all my webalizer info from the last months when they installed the new hard drive...and they didn't start it logging until the 3rd of March. But I keep watch over this stuff because of the cost of bandwidth and I'm sure that we aren't doing more files/hits/pages than last month. We just have more bandwidth going through the machine.

https://www.hystersisters.com/webstat/usage_200303.html

there is something "wrong" to spend 50% more....doncha think when the settings are the same, traffic is the same, file/pages/visitors are the same or less :(

Thanks for your help.

[Kathy]

Would it be beneficial to diable mod_gzip and use the vb gzip instead and see if there is a difference?

[eva2000]

Would it be beneficial to diable mod_gzip and use the vb gzip instead and see if there is a difference?
yes i'd try that if you haven't already..

also i'd download your servers log file and run it through fast stats log analyser www.mach5.com to get a more complete report of which files on your server are being accessed

[Kathy]

Update !

I disabled mod_gzip in apache and just enabled my vb gzip set at compression level 1.

I'm back to normal levels of bandwidth usage and I don't see much difference in the page load times.

What is interesting is that while using mod_gzip, my bandwidth went up 50% over normal bandwidth usage per day. (from less than 3 gig a day to 5+ gig a day) :eek:

Anyone know why this is? It seems my server loads were getting way up there (2.5+-5) on days when we were running 150+ online at the same time on the vb.

I'm watching the server loads...and not only are we lower, but bandwidth is at a savings too. Why?

I would think that by compressing the entire server, all my pages, it would save on bandwidth instead of causing it to go up.

[MarkB]

How do you disable mod_gzip? :confused:

[Kathy]

Hmmm...well, I emailed my sysadmin and asked him to do it. :-)

Its in the apache config file I think!

[eva2000]

that is strange... what were your mod_gzip config settings in httpd.conf ?

[Kathy]

that is strange... what were your mod_gzip config settings in httpd.conf ?

# mod_gzip stuff
LoadModule gzip_module modules/mod_gzip.so
AddModule mod_gzip.c

mod_gzip_on No
mod_gzip_minimum_file_size 0
mod_gzip_maximum_file_size 1000000
mod_gzip_maximum_inmem_size 100000
mod_gzip_item_include file .htm$
mod_gzip_item_include file .html$
mod_gzip_item_include file .shtml$
mod_gzip_item_include file .shtm$
mod_gzip_item_include file .jsp$
mod_gzip_item_include file .php$
mod_gzip_item_include file .php3$
mod_gzip_item_include file .php4$
mod_gzip_item_include file .pl$
mod_gzip_item_include file .cgi$
mod_gzip_item_include mime text/.*
mod_gzip_item_include mime ^application/x-httpd-php
mod_gzip_item_include mime httpd/unix-directory
mod_gzip_item_include handler ^perl-script$
mod_gzip_item_include handler ^server-status$
mod_gzip_item_include handler ^server-info$
mod_gzip_item_include file .css$
mod_gzip_item_exclude file .js$
mod_gzip_item_exclude mime ^image/.*
mod_gzip_dechunk yes
mod_gzip_temp_dir /tmp
mod_gzip_keep_workfiles No
mod_gzip_can_negotiate yes
mod_gzip_item_exclude mime ^application/x-shockwave-flash
mod_gzip_item_exclude file .swf$
mod_gzip_command_version mod_gzip_version



Settings are the same now...we just turned it OFF instead of ON

[eva2000]

which version of mod_gzip 1.3.19 or 1.3.26 ?

[eva2000]

# mod_gzip stuff
LoadModule gzip_module modules/mod_gzip.so
AddModule mod_gzip.c

mod_gzip_on No
mod_gzip_minimum_file_size 0
mod_gzip_maximum_file_size 1000000
mod_gzip_maximum_inmem_size 100000
mod_gzip_item_include file .htm$
mod_gzip_item_include file .html$
mod_gzip_item_include file .shtml$
mod_gzip_item_include file .shtm$
mod_gzip_item_include file .jsp$
mod_gzip_item_include file .php$
mod_gzip_item_include file .php3$
mod_gzip_item_include file .php4$
mod_gzip_item_include file .pl$
mod_gzip_item_include file .cgi$
mod_gzip_item_include mime text/.*
mod_gzip_item_include mime ^application/x-httpd-php
mod_gzip_item_include mime httpd/unix-directory
mod_gzip_item_include handler ^perl-script$
mod_gzip_item_include handler ^server-status$
mod_gzip_item_include handler ^server-info$
mod_gzip_item_include file .css$
mod_gzip_item_exclude file .js$
mod_gzip_item_exclude mime ^image/.*
mod_gzip_dechunk yes
mod_gzip_temp_dir /tmp
mod_gzip_keep_workfiles No
mod_gzip_can_negotiate yes
mod_gzip_item_exclude mime ^application/x-shockwave-flash
mod_gzip_item_exclude file .swf$
mod_gzip_command_version mod_gzip_version



Settings are the same now...we just turned it OFF instead of ON
for each include item you're missing a forward slash

mod_gzip_item_include file .htm$

should be

mod_gzip_item_include file \.htm$

not sure if that makes a difference

[Kathy]

for each include item you're missing a forward slash

mod_gzip_item_include file .htm$

should be

mod_gzip_item_include file \.htm$

not sure if that makes a difference

My sysadmin guy said:
it shouldn't this was taken directly from mod_gzip for the PSA httpd.conf


PSA=Plesk system admin

Version: mod_gzip/1.3.26.1a

[Kathy]

Okie, update.

While saving bandwidth on gzip within vb, turning off mod_gzip, I was happy.

Then I discovered that by turning on the gzip within the vb, now messed up the photopost software that is integrated with my vb users.

(Weird, since before on the original hard drive, I had vb gzip enabled and it worked perfectly with the photopost software)

Photopost has its own "glib compression" setting "ON" or "OFF" which was always turned to OFF.

so, I've since disabled the vb gzip....because it renders photopost unusable.

I'm on a hunt. There must be some setting that is incorrect somewhere that is messing up the compression settings on my server.

I don't have mod_gzip enabled. I don't have vb gzip enabled. i don't have photopost glib compression enabled.

But now I'm back to tons of bandwidth going out the door.

Who can I get to help me with mod_gzip settings on my server to see if I can get that to work properly...to compress to SAVE on bandwidth, not use more?

[eva2000]

me thinks you need to get off that server... it's been problem after problem on that server IIRC :(

hmmm just noticed at https://www.hystersisters.com/webstat/usage_200303.html you have an abnormally high figure for files served per day for the amount of visitors you get ?

i have a site on my server with close to your traffic in terms of visitors and pages per day and no where near that amount of files served/day and hits/day

also looks like the amount of hits and files/day is proportionate to the bandwidth used i.e.

march 12th has same pages/day as march 10th and less visitors/day than march 10th but has 33% less bandwidth used and 1/2 the number of hits and files/day

attached is example of one of my sites bandwidth close your visitor and pages/day figures

[Kathy]

me thinks you need to get off that server... it's been problem after problem on that server IIRC :(

hmmm just noticed at https://www.hystersisters.com/webstat/usage_200303.html you have an abnormally high figure for files served per day for the amount of visitors you get ?

i have a site on my server with close to your traffic in terms of visitors and pages per day and no where near that amount of files served/day and hits/day

also looks like the amount of hits and files/day is proportionate to the bandwidth used i.e.

march 12th has same pages/day as march 10th and less visitors/day than march 10th but has 33% less bandwidth used and 1/2 the number of hits and files/day

attached is example of one of my sites bandwidth close your visitor and pages/day figures

George? Does this site you are showing me have a busy vb?

I have since figured out how to keep the vb gzipped...even though the rest of the server isn't.

Starting on March 3, we installed and enabled mod_gzip which caused (for whatever reason) all my stats to go hair-wire.

Here is my old webalizer stats from before the hard drive swap out:

February: http://www.hystersisters.com/webalizer/usage_200302.html
(days left off the end of the month because we installed new hard drive on Feb 24)

January: http://www.hystersisters.com/webalizer/usage_200301.html

Notice the difference in stats from then to now. Wow, huh?

The good thing, though...is that yesterday we managed to get mod_gzip disabled and enabled the vb's gzip.

Now we finally figured out the photopost quirk that was messing up the photopost gallery so we are now up and running.

I'd love to ditch the server...but I bought it two years ago. I had thought that replacing the hard drive would be a good thing...to extend the life of it a few more years. :(

I need this to work...and just need to figure out what settings may be wrong...or if I'm perhaps logging stuff that doesn't need logging, or a setting is "quirky" in order to get things back in line.

I do think the current day's bandwidth (and half of yesterday's) will show better stats than the month of March was showing.

I really think that mod_gzip was behaving weird...perhaps doubling files, or something...because now my stats for half of yesterday and today are back down in the 100 gig a month range.



:confused: Thanks for your help George...and if you see any settings or think of anything, please let me know? Paypal is a good thing for tweakings. ;)

[eva2000]

hmmmmmmm does your host bill you by bandwidth based solely on your webalizer stats ? you have mrtg graphs ?

as i suspect that maybe your webalizer stats could of not been correct before, and didn't count all of your hits/files per day and now count them correctly

or have you changed the site design etc since changing hdds ? i.e. add more banners/images to your forum ?

Feb 2003
hits/day: 458439
files/day: 281886
pages/day: 55159
visitors/day: 4695

March 2003
Hits/Day: 752399
Files/Day: 704048
Pages/Day: 10737
Visits/Day: 2970

[Kathy]

hmmmmmmm does your host bill you by bandwidth based solely on your webalizer stats ? you have mrtg graphs ?

as i suspect that maybe your webalizer stats could of not been correct before, and didn't count all of your hits/files per day and now count them correctly

or have you changed the site design etc since changing hdds ? i.e. add more banners/images to your forum ?

Feb 2003
hits/day: 458439
files/day: 281886
pages/day: 55159
visitors/day: 4695

March 2003
Hits/Day: 752399
Files/Day: 704048
Pages/Day: 10737
Visits/Day: 2970

The difference in those stats I THINK is not the webalizer stats but the mod_gzip gone bizerk. Because...now that I have mod_gzip turned off..and vb gzip working...the stats should be getting back to normal...atleasts for bandwidth.

the data center charges me by bandwidth...but not based on webalizer.

I have this url that the datacenter follows for my server:

http://traffic.ethos.net/cat2924-49-5-5.html

Which shows today hourly bandwidth in and out...

and the previous months....

so that you can compare that up until Feb 24, I was under 3 gig a day (once I gzipped the vb)

After that date, with mod_gzip enabled off and on (and a few days with both! :eek:) but as of yesterday, no mod_gzip but the vb gzip turned on.

Shouldn't mod_gzip work so that I'm using less bandwidth??

What is wrong in that I was using more...with lots more files? I suspect that the mod_gzip was going nuts and compressing, decompressing files several times each hit. :lol: