My company run a CMS (typo3) on LAMP on RH linux.
Basically all users have same typo3 page as home page and thousands of users starts their browser every morning causing server very slow or even to hang (swap full because of too many apache processes)..
99% of the traffic is this home page which take 2s in a non busy time to be delivered.
I improve this time to 1.5s using php cache. Others mysql/apache tunning have so far not given me any significant improvement (I have been trhu web pages on apache/mysql tunning and test differents things).
As far I have seen mysql is ok and httpd processes are taking all cpu/mem.
My question is :
1 - As this home page (php page, 30K) in 99% of the traffic how can i improve the process not getting 1000 times the same request to my apache server.
I have no way to modify this home page policy, I can just work on the infrastructure side. I was thinking on some proxy/cache server but have no experience on that. Any efficient and open source solution ?
2 - Apart from php cache any hint to help me improve apache performance ?
I did not find anything else number of thread and related stuff in apache.
It is bad that you can't modify optimize the home page, 30k php page sounds like a lot, combined with your users' concurrency (how much is it ? average, peaks ? ). Apache can be tuned in many ways, depending on the usage. Ironically, this is what I'm working on currently, so I will have to ask about the version of apache you're using. Depending on the answer I should be able to provide more info. Also, let us know if you are using SSL somewhere to serve content, and, if it's not secret, what are the hardware parameters of the server. For better PHP handling, I can suggest FastCGI - I have seen very good results with it in the past.
1a. APC cache was the quickest way to get the most improvement in speed on my server. Memcached is also an option, but only has benefits above APC when you can distribute the caching load to backend servers or if you have multiple front-end servers.
1b. Squid. See Squid (software) - Wikipedia, the free encyclopedia. You put your LAMP server on another port, and proxy it using SQUID. Squid will cache as much as can be cached and fetch from your LAMP server. If most of your content is dynamic, this may not help much.
2a. You absolutely must minimize swap in such an environment. If you start going to swap, your system loses.
In sysctl.conf, change:
vm.swappiness = 1
and "tune" your swap to use a swap file that's about 80M. If you need more swap then that, your system will crash. Better to let Apache think it cannot get more memory and return 503 (server busy) errors than to wait forever.
2b. You can play around with the following:
MinSpareServers # keep this at least 5 or 10.
MaxSpareServers # no problem to set this 1/4th max.
ServerLimit # set to what your server can handle, memory-wise
MaxClients # Set to what your server can handle, memory-wise
KeepAlive # keep on
KeepAliveTimeout # keep to 30 seconds or less
HostnameLookups # set to OFF
2c. More tuning at the TCP level. See sysctl parameters such as:
net.core.somaxxconn
net.core.netdev_max_backlog
net.ipv4.tcp_keepalive_intvl
net.ipv4.tcp_tw_reuse
net.core.optmem_max
net.core.rmem_max
net.core.wmem_max
net.ipv4.tcp_rmem
net.ipv4.tcp_wmem
net.ipv4.tcp_congestion_control
Also, LOWER the amount of shared memory, to ensure you have more memory for HTTP. (Note: This may not work if you are running MySql or postgres on the same server!)
kernel.shmmax
kernel.shmall
2d. If you have the memory, put all your HTTP/PHP files onto a ramdisk. Otherwise, install everything on a FAST internal (or SAN) disk. When I say FAST, I mean hardware-RAID1 15k RPM SCSI Fiberchannel. Store all log files on a SEPARATE disk, which can be any junk drive.
My apache version is : Apache/2.0.52 with 4x2.8 mhz xeon with 3gb RAM and 2gb swap.
On dev I got VM machine with 2x1.86Mhz xeon and 2.5gb RAM and 2Gb swap (I do all tests there).
99% of the traffic is when users starts their browser around 9h.
A lot of connections (may be 1000 users).
I do my tests on dev with curl and ab but did not experience any swap pb (i did not start more than 100 thread each time).
I use eaccelerator for cache because of my boss choice, I tried mmcache and got slightly better results.
1a : I will try APC.
1b : as typo3 is php script and home page is index.php I wonder how squid will help. I thought that squid is not easy to setup, is it worth to try ??
2a : I agree with you better having 503 than swapping.
I understand vm.swappiness, but it's not clear on what you say about tunning swap, is it trying different values on swappiness ? It looks tricky :-)
2b : Some parameters are not clear for me especially ServerLimit/maxclient. I am a little bit reluctant to change that as I am not sure of what I am doing. I'l take deeper look in documentation.
keepalive is on and I will try to play with the value.
But most of my connections are 1 time connection, user start browser ask home page and that's all.
For 1 connection like that I was gessing that I do not need a high keepalive as I got only 1 http request, I am right ?
Is there any relation between tcp keepalive and apache keepalive ?
Finally as I tried to emphasize, I got 99% request on the same page which is dynamic page BUT do not change 99% of the time.
Can I cache that kind of things in any proxy or cache to avoid going thru all the layers (browser httpd mysql httpd ...).
And as otheus said I rather like to have 503 but quick response than having server crashing.
Maybe the swap tuning is my best help in this case ???
1b : as typo3 is php script and home page is index.php I wonder how squid will help. I thought that squid is not easy to setup, is it worth to try ??
I'm not really sure. If the ENTIRE home page is from PHP, then it won't. However, if there's a lot of media, static JSS, static CSS on the page, it might help. Then again, if it's just one page, and there's a fixed number of media files, it won't -- HTTP and the OS should cache most of that.
Quote:
2a : I agree with you better having 503 than swapping.
I understand vm.swappiness, but it's not clear on what you say about tunning swap, is it trying different values on swappiness ? It looks tricky :-)
After you move /var/log to its own disk, find the disk least used in your system. Say it's already mounted as /vol3.
That's it. Now go into /etc/fstab and change the /swap mountpoint. Finally, make sure rc.sysinit does the right thing. Do:
If mount comes before swapon, you're fine.
Quote:
2b : Some parameters are not clear for me especially ServerLimit/maxclient. I am a little bit reluctant to change that as I am not sure of what I am doing. I'l take deeper look in documentation.
Nothing to fear here.
This is straight from Apache documentation. In your case, you want to set the maximum servers to something like 1024 (!) and the MinSpareServers to something like 128 or maybe even 512.
Quote:
keepalive is on and I will try to play with the value.
But most of my connections are 1 time connection, user start browser ask home page and that's all. For 1 connection like that I was gessing that I do not need a high keepalive as I got only 1 http request, I am right ?
You got it! You might not need at all, but I am guessing your home page also consists of graphics, JS source files, CSS files, etc., all coming from your server. So its better to keep it on, but to timeout after only a few seconds.
Quote:
Is there any relation between tcp keepalive and apache keepalive ?
Only if you have lots of connections all over the internet. If everyone is coming from a LAN or something, you don't need to touch it.
Quote:
Finally as I tried to emphasize, I got 99% request on the same page which is dynamic page BUT do not change 99% of the time.
Can I cache that kind of things in any proxy or cache to avoid going thru all the layers (browser httpd mysql httpd ...).
If you know the parameters that change it, then sure! Just pre-generate the page and serve that page. If the condition is determined on IP or COOKIE or some header info, you can use Mod_rewrite to give redirect the user to the dynamic one.
Quote:
My disk are internal HD that I can't change.
What's MySQL doing in the picture? Is the DB read-only? If so, can you fit it on a RAM-disk?
Run this command before and after each set of changes you make, assuming you make one change per day. Post the early and improved results here.
Location: Asia Pacific, Cyberspace, in the Dark Dystopia
Posts: 19,118
Thanks Given: 2,351
Thanked 3,359 Times in 1,878 Posts
Quote:
Originally Posted by otheus
This is straight from Apache documentation. In your case, you want to set the maximum servers to something like 1024 (!) and the MinSpareServers to something like 128 or maybe even 512.
These numbers are too high and will cause performance problems.
FYI, our site is quite busy (do you have more traffic than this site?) and our numbers are much lower!
How do I install a LAMP server on a new installation of Debian 9 using the lalest versions of
AMP?
Here is what I have.
Corrections please.
MYSQL
apt-get install mysql-server mysql-client
You can verify the MySQL server status using command:
systemctl status mysql
-------
PHP7... (0 Replies)
I wish to setup LAMP environment. Amongst, I have successfully installed Linux 6.1. I am looking fwd to install - Apace Web Server, My Sql Database & PHP environment.
Say if I look for MySql, could see downloadable available in rpm format. But this must be copied over to Linux machine.
As... (2 Replies)
Hello All,
I need a good lamp(linux, apache, mysql, php) tutorial for web development. I searched in net but not finding suitable one. suggest me something. (3 Replies)
Hello everyone,
I would like to setup a lamp server from a minimal distro and to compile PHP, MySQL and Apache myself.
I have chosen CentOS minimal for the OS and I am trying to build the stack by hand... But well, it appears I need some help!
First: I am looking for good and recent... (3 Replies)
Hello All,
I have a LAMP server (CentOS 5.5) which has a lot of php scripts and the php pages runs fine. My question is if I have to allow the OS udpates regularly on the server, will there be any problems in running the existing php scripts? Please advise if my apache/mysql/php functionality... (3 Replies)
How does everyone else handle this?
My Setup:
Server A:
CentOS 5.x
10.0.0.1
Apache
MySQL Master
Server B:
CentOS 5.x
10.0.0.2
Apache
MySQL Slave
My Domains:
dom1.com A record: 10.0.0.1
A Record: 10.0.0.2
dom2.com A record: 10.0.0.1
A Record: 10.0.0.2 (3 Replies)
Hi there, I've got a debian 3.1r4 half working (no x yet :P)
I need some pointers for installing a Apache - MySQL - PHP configuration on my box, and set them up to work together. I know to use vim a bit :)
I would like something that is similar to tanguay.at, or even simpler, since I'm a real... (1 Reply)