I'm posting for help here, because I'll be honest I've reached the end of my tether, hopefully someone can give me some assistance and help me maintain a level of sanity...
I maintain a number of webservers on RHEL 5 64Bit (Red Hat Enterprise Linux Server release 5.3 (Tikanga)), the smallest with 8GB of RAM, the largest with 12GB. As I serve these behind a load balancer I thought it was best to not pack them with RAM.
In short my problem is that I consume MaxClients within 2 minutes. At present my MaxClients setting within prefork.c is at 600. HTTPD has taken this setting and will run to the level.
My solution is as follows:
LoadBalancer: The LB will take connections on port 80 and 443 for our websites using VIP's (Virtual IP) and then pass to an internal web server based on connection count in the load balancer. SSL termination is completed on the load balancer.
Web Servers:
Our front facing application is HTTPD (httpd-2.2.16 - RPM) which takes all connections and then proxies traffic onto Tomcat 6.0.18 (tar.gz from Website) using AJP13 mounts (same server). Our website is built using Java hence the need for Tomcat. I'm using JRE 1.6.17 with a symlink to the java binary in /usr/bin/java so any process can call Java.
Catalina has the JRE_HOME option defined.
I have attempted to install APR using the following commands, as my predecessors' tell me that unless we're running APR what I'm seeing with happen.
Our Database is MSSQL, using the JTDS.JDBC connection pool.
I have installed APR using the following (instructions passed down from previous, and working on other servers)
Code:
wget http://archive.apache.org/dist/apr/apr-1.3.3.tar.gz
wget http://archive.apache.org/dist/apr/apr-util-1.3.4.tar.gz
tar -zxvf apr-1.3.3.tar.gz
cd apr-1.3.3
./configure --prefix=/opt
make
make install
tar -zxvf apr-util-1.3.4.tar.gz
cd apr-util-1.3.4
./configure --prefix=/opt --with-apr=/opt/bin
make
make install
However, if I run rpm -qa | grep apr I get the following results.
Note the -Jason, signifies the files came from the Jason Litka Repo found at http://www.jasonlitka.com/yum-repository this is because one of our sites required mod_subsititute which is not installed as part of 2.2.3 (stock on RHEL5)
The command apr-1-config --version, tells me im running 1.2.7.
I've also ensured that libtcnative is installed using
Code:
cd /opt/apache-tomcat-6.0.18/bin/
tar -zxvf tomcat-native.tar.gz
cd /opt/apache-tomcat-6.0.18/bin/tomcat-native-1.1.14-src/jni/native
./configure --with-apr=/opt/bin --with-java-home=/opt/jdk1.6.0_17
make
make install
cd /usr/lib
ln -s /usr/local/apr/lib/libtcnative-1.so libtcnative-1.so
ln -s /usr/local/apr/lib/libtcnative-1.so.0 libtcnative-1.so.0
Inside $CATALINA_HOME/logs/catalina.out I can see that APR is loaded
Code:
INFO: Loaded APR based Apache Tomcat Native library 1.1.14.
Nov 29, 2010 9:46:14 PM org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
However, I also see the following, above this statement.
Code:
Nov 29, 2010 9:46:14 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'debug' to '0' did not find a matching property.
Nov 29, 2010 9:46:14 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'minSpareThreads' to '25' did not find a matching property.
Nov 29, 2010 9:46:14 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxSpareThreads' to '75' did not find a matching property.
Nov 29, 2010 9:46:14 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'debug' to '0' did not find a matching property.
Nov 29, 2010 9:46:14 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'minSpareThreads' to '25' did not find a matching property.
Nov 29, 2010 9:46:14 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'maxSpareThreads' to '75' did not find a matching property.
All files within $CATALINA_HOME are owned by the user tomcat6. FYI: $CATALINA_HOME is /opt/apache-tomcat-6.0.18
My tomcat setenv.sh file is as follows (changes depending on amount of ram):
I cannot see errors as to why we're hitting MaxClients within any *-error_log file within httpd log directory. Catalina.out does show exception errors, such as NullPointerException, which my development team tell me is normal (our code has bugs).
I have a script which calls pstree -G | grep httpd and returns the value and inserts into a log file, every 2 minutes. When this script goes above a threshold, which I believe will not recover from, it issues an httpd stop/start and tomcat6 stop/start
At time of writing this message, the stats are as follows
Web Server: 129 running processes
Load Balancer: 290 active connections (260 HTTP / 30 SSL)
Can anyone assist as to why we're reaching MaxClients. Is APR not setup properly? I'm literally at my wits end. Other servers within the solution are working fine, including connectivity to the database. Which I have confirmed is working from the servers having this issue. Switches, cables and network cards all changed. The server recently having this issue, is brand new.
Thank you for your time, and I welcome any questions to help solve this.
I need to install Apache TOmcat 4.01 on SCO Openserver 5.07 rather than using the default 4.1.29 version which comes with the original release of 5.07
anyone got any ideas hw to go about this, really struggling having downloaded the source and binariys and getting nowhere fast (0 Replies)
Hi,
I have a machine with Unixware 7.1.3 and Apache 2 and Tomcat 4.0.3 it's works, but in the archive log from mod_jk.log
ajp13_process_callback - write failed
: Error
ajp13_process_callback - write failed
: Error
ajp13_process_callback - write failed
: Error... (1 Reply)
Hi,
I am facing some problem with Apache to Tomcat forwarding. I have Tomcat (6.0.18) behind Apache (2.2). I want to use ‘https' for receiving request from the client. This request, when forwarded by Apache to Tomcat, will use ‘http'. My application's name is ‘abc', which is deployed in... (1 Reply)
I want to setup apache tomcat on my linux distribution .
I Have downloaded apache tomcat from below mentioned link.
Administration Web Application:
Apache Tomcat - Apache Tomcat 5 Downloads
The setup was in following format.
apache-tomcat-5.5.29-admin.tar.gz
Below are the command i... (7 Replies)
Hi
I have tried setting up of Apache http server - httpd-2.4.25.
During configuration, I understand it needs APR to be setup. Hence I have downloaded APR & APR-Util.
Performed,
tar xvfC apr-1.5.2.tar /root/httpd-2.4.25/srclib/apr
What is happening is there is another directory... (1 Reply)
Hi,
I have been assigned a task to build a server with these requirements :
> has multiple tomcats running under the same apache.
> tomcat & jdk has to be binded with apache.
> latest openssl with all necesarry "mod" to bind with apache & tomcat.
I am fairly new to... (1 Reply)
11-29-2010