Fri Jul 17 10:22:04 CDT 2009
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_MON_STARTING
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_ON_STARTED
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_monitor_start (scalable svc)> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/sc/ssm_wrapper>:tag=<apache-rg.apache-rs.7>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:23 tommy_sun SC[SUNW.apache:4.1,apache-rg,apache-rs,SSM_MONITOR_START]: [ID 487022 daemon.notice] The networking components for scalable resource apache-rs have been configured successfully for method MONITOR_START.
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <apache_monitor_start (scalable svc)> completed successfully for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, time used: 0% of timeout <300 seconds>
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_ONLINE
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_ONLINE
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_JUST_STARTED
Jul 17 10:21:23 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_ONLINE_UNMON
Jul 17 10:21:24 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_ONLINE
Jul 17 10:21:24 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_ONLINE
Jul 17 10:21:25 tommy_sun Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="apache-rg,apache-rs,0.svc", cmd="/bin/sh -c /usr/apache2/bin/apachectl start", Failed to stay up.
Jul 17 10:21:25 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_probe]: [ID 670283 daemon.notice] Issuing a resource restart request because the application exited.
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 494478 daemon.notice] resource apache-rs in resource group apache-rg has requested restart of the resource on tommy_sun.
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_ON_PENDING_R_RESTART
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_FAULTED
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Service daemon not running.>
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_MON_STOPPING
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_monitor_stop (scalable svc)> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/sc/ssm_wrapper>:tag=<apache-rg.apache-rs.8>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_FAULTED
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Service daemon not running.>
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_ON_PENDING_R_RESTART
Jul 17 10:21:25 tommy_sun SC[SUNW.apache:4.1,apache-rg,apache-rs,SSM_MONITOR_STOP]: [ID 487022 daemon.notice] The networking components for scalable resource apache-rs have been configured successfully for method MONITOR_STOP.
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <apache_monitor_stop (scalable svc)> completed successfully for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, time used: 0% of timeout <300 seconds>
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_ONLINE_UNMON
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STOPPING
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_stop> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_UNKNOWN
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Stopping>
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscapc/bin/apache_svc_stop>:tag=<apache-rg.apache-rs.1>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:25 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 584207 daemon.notice] Stopping Apache Web Server.
Jul 17 10:21:25 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 401400 daemon.notice] Successfully stopped the application
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_OFFLINE
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <apache_svc_stop> completed successfully for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, time used: 0% of timeout <300 seconds>
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OFFLINE
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STARTING_DEPEND
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OK_TO_START
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_ONLINE
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STARTING
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_start (scalable svc)> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:25 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/sc/ssm_wrapper>:tag=<apache-rg.apache-rs.0>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_ONLINE_UNMON
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_UNKNOWN
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Stopping>
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STOPPING
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_OFFLINE
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OFFLINE
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STARTING_DEPEND
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OK_TO_START
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_PENDING_ONLINE
Jul 17 10:21:26 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STARTING
Jul 17 10:21:27 tommy_sun SC[SUNW.apache:4.1,apache-rg,apache-rs,SSM_START]: [ID 487022 daemon.notice] The networking components for scalable resource apache-rs have been configured successfully for method START.
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 743923 daemon.notice] Starting server with command /usr/apache2/bin/apachectl start.
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 187883 daemon.error] Apache service with startup script </usr/apache2/bin/apachectl> does not configure PIDFILE.
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 569559 daemon.notice] Start of /usr/apache2/bin/apachectl start completed successfully.
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_ONLINE
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Completed successfully.>
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_ONLINE
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Completed successfully.>
Jul 17 10:21:27 tommy_sun Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="apache-rg,apache-rs,0.svc", cmd="/bin/sh -c /usr/apache2/bin/apachectl start", Failed to stay up.
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 423137 daemon.error] A resource restart attempt on resource apache-rs in resource group apache-rg has been blocked because the number of restarts within the past Retry_interval (620 seconds) would exceed Retry_count (2)
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 819721 daemon.error] Failed to start Apache Web Server.
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 938318 daemon.error] Method <apache_svc_start (scalable svc)> failed on resource <apache-rs> in resource group <apache-rg> [exit code <1>, time used: 0% of timeout <300 seconds>]
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_START_FAILED
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_OFF_START_FAILED
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_FAULTED
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STOPPING
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_UNKNOWN
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Stopping>
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_stop> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:27 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscapc/bin/apache_svc_stop>:tag=<apache-rg.apache-rs.1>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 584207 daemon.notice] Stopping Apache Web Server.
Jul 17 10:21:27 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 401400 daemon.notice] Successfully stopped the application
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_OFFLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <apache_svc_stop> completed successfully for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, time used: 0% of timeout <300 seconds>
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OFFLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_OFFLINE_START_FAILED
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_OFFLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_ONLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STARTING_DEPEND
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OK_TO_START
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STARTING
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_start (scalable svc)> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/sc/ssm_wrapper>:tag=<apache-rg.apache-rs.0>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_START_FAILED
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_FAULTED
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_PENDING_OFF_START_FAILED
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_UNKNOWN
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Stopping>
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STOPPING
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_OFFLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OFFLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_OFFLINE_START_FAILED
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_OFFLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_PENDING_ONLINE
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STARTING_DEPEND
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OK_TO_START
Jul 17 10:21:28 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STARTING
Jul 17 10:21:29 tommy_sun SC[SUNW.apache:4.1,apache-rg,apache-rs,SSM_START]: [ID 487022 daemon.notice] The networking components for scalable resource apache-rs have been configured successfully for method START.
Jul 17 10:21:29 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 743923 daemon.notice] Starting server with command /usr/apache2/bin/apachectl start.
Jul 17 10:21:29 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 187883 daemon.error] Apache service with startup script </usr/apache2/bin/apachectl> does not configure PIDFILE.
Jul 17 10:21:29 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 569559 daemon.notice] Start of /usr/apache2/bin/apachectl start completed successfully.
Jul 17 10:21:29 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_ONLINE
Jul 17 10:21:29 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Completed successfully.>
Jul 17 10:21:29 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_ONLINE
Jul 17 10:21:29 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Completed successfully.>
Jul 17 10:21:29 tommy_sun Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="apache-rg,apache-rs,0.svc", cmd="/bin/sh -c /usr/apache2/bin/apachectl start", Failed to stay up.
Jul 17 10:21:30 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 423137 daemon.error] A resource restart attempt on resource apache-rs in resource group apache-rg has been blocked because the number of restarts within the past Retry_interval (620 seconds) would exceed Retry_count (2)
Jul 17 10:21:30 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 819721 daemon.error] Failed to start Apache Web Server.
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 938318 daemon.error] Method <apache_svc_start (scalable svc)> failed on resource <apache-rs> in resource group <apache-rg> [exit code <1>, time used: 0% of timeout <300 seconds>]
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_START_FAILED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_OFF_START_FAILED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_FAULTED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STOPPING
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_UNKNOWN
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Stopping>
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_stop> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscapc/bin/apache_svc_stop>:tag=<apache-rg.apache-rs.1>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:30 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 584207 daemon.notice] Stopping Apache Web Server.
Jul 17 10:21:30 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 401400 daemon.notice] Successfully stopped the application
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_OFFLINE
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <apache_svc_stop> completed successfully for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, time used: 0% of timeout <300 seconds>
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OFFLINE
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_OFFLINE_START_FAILED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_OFFLINE
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 447451 daemon.notice] Not attempting to start resource group <apache-rg> on node <tommy_sun> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_START_FAILED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_FAULTED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_PENDING_OFF_START_FAILED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_UNKNOWN
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Stopping>
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STOPPING
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_OFFLINE
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OFFLINE
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_OFFLINE_START_FAILED
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_OFFLINE
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 447451 daemon.notice] Not attempting to start resource group <apache-rg> on node <tommy_sun1> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 447451 daemon.notice] Not attempting to start resource group <apache-rg> on node <tommy_sun> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jul 17 10:21:30 tommy_sun Cluster.RGM.global.rgmd: [ID 674214 daemon.notice] rebalance: no primary node is currently found for resource group <apache-rg>.
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_PENDING_ONLINE
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_ONLINE
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STARTING_DEPEND
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OK_TO_START
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STARTING
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_start (scalable svc)> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</usr/cluster/lib/sc/ssm_wrapper>:tag=<apache-rg.apache-rs.0>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STARTING_DEPEND
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OK_TO_START
Jul 17 10:21:51 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STARTING
Jul 17 10:21:52 tommy_sun SC[SUNW.apache:4.1,apache-rg,apache-rs,SSM_START]: [ID 487022 daemon.notice] The networking components for scalable resource apache-rs have been configured successfully for method START.
Jul 17 10:21:52 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 743923 daemon.notice] Starting server with command /usr/apache2/bin/apachectl start.
Jul 17 10:21:52 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 187883 daemon.error] Apache service with startup script </usr/apache2/bin/apachectl> does not configure PIDFILE.
Jul 17 10:21:52 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 569559 daemon.notice] Start of /usr/apache2/bin/apachectl start completed successfully.
Jul 17 10:21:52 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_ONLINE
Jul 17 10:21:52 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Completed successfully.>
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_ONLINE
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Completed successfully.>
Jul 17 10:21:53 tommy_sun Cluster.PMF.pmfd: [ID 887656 daemon.notice] Process: tag="apache-rg,apache-rs,0.svc", cmd="/bin/sh -c /usr/apache2/bin/apachectl start", Failed to stay up.
Jul 17 10:21:53 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 423137 daemon.error] A resource restart attempt on resource apache-rs in resource group apache-rg has been blocked because the number of restarts within the past Retry_interval (620 seconds) would exceed Retry_count (2)
Jul 17 10:21:53 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_start]: [ID 819721 daemon.error] Failed to start Apache Web Server.
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 938318 daemon.error] Method <apache_svc_start (scalable svc)> failed on resource <apache-rs> in resource group <apache-rg> [exit code <1>, time used: 0% of timeout <300 seconds>]
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_START_FAILED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_PENDING_OFF_START_FAILED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_FAULTED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_STOPPING
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_UNKNOWN
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Stopping>
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 224900 daemon.notice] launching method <apache_svc_stop> for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, timeout <300> seconds
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 703477 daemon.notice] 38 fe_rpc_command: cmd_type(enum):<1>:cmd=</opt/SUNWscapc/bin/apache_svc_stop>:tag=<apache-rg.apache-rs.1>: Calling security_clnt_connect(..., host=<tommy_sun>, sec_type {0:WEAK, 1:STRONG, 2
ES} =<1>, ...)
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_START_FAILED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_FAULTED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_PENDING_OFF_START_FAILED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_UNKNOWN
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Stopping>
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_STOPPING
Jul 17 10:21:53 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 584207 daemon.notice] Stopping Apache Web Server.
Jul 17 10:21:53 tommy_sun SC[,SUNW.apache:4.1,apache-rg,apache-rs,apache_svc_stop]: [ID 401400 daemon.notice] Successfully stopped the application
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun change to R_FM_OFFLINE
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 515159 daemon.notice] method <apache_svc_stop> completed successfully for resource <apache-rs>, resource group <apache-rg>, node <tommy_sun>, time used: 0% of timeout <300 seconds>
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun change to R_OFFLINE
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_OFFLINE_START_FAILED
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun change to RG_OFFLINE
Jul 17 10:21:53 tommy_sun Cluster.RGM.global.rgmd: [ID 447451 daemon.notice] Not attempting to start resource group <apache-rg> on node <tommy_sun> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 784560 daemon.notice] resource apache-rs status on node tommy_sun1 change to R_FM_OFFLINE
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 922363 daemon.notice] resource apache-rs status msg on node tommy_sun1 change to <Successfully stopped Apache Web Server.>
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 443746 daemon.notice] resource apache-rs state on node tommy_sun1 change to R_OFFLINE
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_OFFLINE_START_FAILED
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 529407 daemon.notice] resource group apache-rg state on node tommy_sun1 change to RG_OFFLINE
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 447451 daemon.notice] Not attempting to start resource group <apache-rg> on node <tommy_sun1> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 447451 daemon.notice] Not attempting to start resource group <apache-rg> on node <tommy_sun> because this resource group has already failed to start on this node 2 or more times in the past 3600 seconds
Jul 17 10:21:54 tommy_sun Cluster.RGM.global.rgmd: [ID 674214 daemon.notice] rebalance: no primary node is currently found for resource group <apache-rg>.