Login or Register to Ask a Question and Join Our Community


Solaris

scstat: unexpected error , sc 3.2

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems Solaris scstat: unexpected error , sc 3.2
# 1  
Old 07-24-2009
scstat: unexpected error , sc 3.2
Hello,

Looks like cluster nodes are completely messed up. I had configured mysql resource as failover group using smf.proxy.failover and it looked okay so I 'init 6' one node, other node somehow hung and when both were reset, both have problems .

-bash-3.00# scstat -g

Quote:
scstat: unexpected error.

-bash-3.00# svcs -xv
Quote:
svc:/system/cluster/cl-svc-enable:default (Enabling the cluster userland services)
State: maintenance since Fri Jul 24 08:42:34 2009
Reason: Completes a dependency cycle.
See: Sun Message ID: SMF-8000-HP
Impact: 25 dependent services are not running:
svc:/system/cluster/ql_upgrade:default
svc:/system/cluster/mountgfs:default
svc:/system/cluster/clusterdata:default
svc:/system/cluster/ql_rgm:default
svc:/system/zones:default
svc:/system/cluster/pnm:default
svc:/system/cluster/rgm-starter:default
svc:/application/management/common-agent-container-1:default
svc:/system/cluster/cl-svc-cluster-milestone:default
svc:/system/cluster/sc_svtag:default
svc:/system/cluster/sckeysync:default
svc:/system/cluster/scsymon-srv:default
svc:/system/cluster/scslmclean:default
svc:/system/cluster/sc_pnm_proxy_server:default
svc:/system/cluster/rpc-fed:default
svc:/system/cluster/sc_zones:default
svc:/system/cluster/scprivipd:default
svc:/system/cluster/cznetd:default
svc:/system/cluster/rpc-pmf:default
svc:/system/cluster/sc_zc_member:default
svc:/system/cluster/scdpm:default
svc:/system/cluster/cl-event:default
svc:/system/cluster/cl-eventlog:default
svc:/system/cluster/cl-ccra:default
svc:/system/cluster/scqdm:default

svc:/network/cswmysql5:default (?)
State: uninitialized since Fri Jul 24 08:42:34 2009
Reason: Restarter svc:/system/cluster/sc_restarter:default has not initialized service state.
See: Sun Message ID: SMF-8000-4D
Impact: 6 dependent services are not running:
svc:/milestone/multi-user:default
svc:/application/management/common-agent-container-1:default
svc:/milestone/multi-user-server:default
svc:/system/basicreg:default
svc:/system/zones:default
svc:/system/iscsitgt:default
-bash-3.00# ps -ef
Quote:
UID PID PPID C STIME TTY TIME CMD
root 0 0 0 08:42:21 ? 0:19 sched
root 1 0 0 08:42:22 ? 0:00 /sbin/init
root 2 0 0 08:42:22 ? 0:00 pageout
root 3 0 0 08:42:22 ? 0:00 fsflush
root 4 0 0 08:42:22 ? 0:00 cluster
root 119 1 0 08:43:02 ? 0:00 /lib/svc/method/iscsid
root 8 1 0 08:42:26 ? 0:15 /lib/svc/bin/svc.startd
root 10 1 0 08:42:26 ? 0:28 /lib/svc/bin/svc.configd
root 227 1 0 08:43:17 ? 0:00 /usr/lib/inet/in.mpathd
root 67 1 0 08:42:43 ? 0:00 /sbin/dhcpagent
daemon 197 1 0 08:43:14 ? 0:00 /usr/lib/crypto/kcfd
upendra 627 626 0 08:44:24 ? 0:00 /usr/lib/ssh/sshd
root 162 1 0 08:43:11 ? 0:00 /usr/cluster/lib/sc/sc_delegated_restarter
root 210 1 0 08:43:15 ? 0:00 /usr/lib/picl/picld
root 360 359 0 08:43:56 ? 0:00 /usr/cluster/lib/sc/clexecd
root 328 1 0 08:43:54 ? 0:00 /usr/cluster/lib/sc/failfastd
root 441 1 0 08:44:00 ? 0:00 /usr/sbin/cron
root 292 1 0 08:43:43 ? 0:00 /usr/cluster/lib/sc/qd_userd
root 200 1 0 08:43:14 ? 0:00 /usr/lib/sysevent/syseventd
root 300 1 0 08:43:51 ? 0:00 /usr/cluster/lib/scadmin/ql/ql_server
root 319 1 0 08:43:53 ? 0:00 /usr/sbin/nscd
root 359 1 0 08:43:56 ? 0:00 /usr/cluster/lib/sc/clexecd
root 232 1 0 08:43:21 ? 0:00 devfsadmd
root 341 1 0 08:43:55 ? 0:00 /usr/cluster/lib/sc/rtreg_proxy_serverd
root 342 1 0 08:43:55 ? 0:00 /usr/cluster/lib/sc/ifconfig_proxy_serverd
root 331 1 0 08:43:54 ? 0:00 /usr/cluster/lib/sc/cl_execd
root 332 331 0 08:43:54 ? 0:00 /usr/cluster/lib/sc/cl_execd
root 357 1 0 08:43:55 ? 0:00 /usr/cluster/lib/sc/pmmd
noaccess 678 1 0 08:44:31 ? 0:18 /usr/java/bin/java -server -Xmx128m -XX:+UseParallelGC -XX:ParallelGCThreads=4
root 518 8 0 08:44:04 console 0:00 /usr/lib/saf/ttymon -g -d /dev/console -l console -m ldterm,ttcompat -h -p upen
root 470 466 0 08:44:01 ? 0:00 /usr/lib/saf/ttymon
daemon 416 1 0 08:43:58 ? 0:00 /usr/sbin/rpcbind
root 626 522 0 08:44:24 ? 0:00 /usr/lib/ssh/sshd
daemon 421 1 0 08:43:59 ? 0:00 /usr/lib/nfs/nfsmapid
root 471 1 0 08:44:01 ? 0:02 /usr/lib/fm/fmd/fmd
root 680 634 0 08:44:44 pts/1 0:00 -bash
upendra 634 627 0 08:44:29 pts/1 0:00 -bash
root 522 1 0 08:44:04 ? 0:00 /usr/lib/ssh/sshd
root 502 1 0 08:44:03 ? 0:00 /usr/sbin/syslogd
root 466 8 0 08:44:01 ? 0:00 /usr/lib/saf/sac -t 300
root 467 1 0 08:44:01 ? 0:02 /usr/lib/inet/inetd start
root 493 1 0 08:44:02 ? 0:00 /usr/lib/utmpd
root 686 680 0 08:48:40 pts/1 0:00 ps -ef
root 533 1 0 08:44:08 ? 0:00 /usr/lib/inet/xntpd


dmesg and cluster status are here :
pastebin - Stuff - post number 1505764

Forgot to mention the OS is solaris x86_64 - with latest patches
Quote:
5.10 Generic_141415-05 i86pc i386 i86pc
This is pretty much scene on both the nodes.

Last thing, i don't want to do is reinstall or do everything from scratch, so could someone please help?

Thank you in advance.

---------- Post updated at 09:24 AM ---------- Previous update was at 09:14 AM ----------

if i type scstat instead of scstat -g

-bash-3.00# scstat
Quote:

------------------------------------------------------------------

-- Cluster Nodes --

Node name Status
--------- ------
Cluster node: tommy_sun1 Online
Cluster node: tommy_sun Online

------------------------------------------------------------------

-- Cluster Transport Paths --

Endpoint Endpoint Status
-------- -------- ------
Transport path: tommy_sun1:e1000g2 tommy_sun:e1000g2 Path online

------------------------------------------------------------------

-- Quorum Summary from latest node reconfiguration --

Quorum votes possible: 3
Quorum votes needed: 2
Quorum votes present: 2


-- Quorum Votes by Node (current status) --

Node Name Present Possible Status
--------- ------- -------- ------
Node votes: tommy_sun1 1 1 Online
Node votes: tommy_sun 1 1 Online


-- Quorum Votes by Device (current status) --

Device Name Present Possible Status
----------- ------- -------- ------
Device votes: /dev/did/rdsk/d5s2 1 1 Online

------------------------------------------------------------------

-- Device Group Servers --

Device Group Primary Secondary
------------ ------- ---------


-- Device Group Status --

Device Group Status
------------ ------


-- Multi-owner Device Groups --

Device Group Online Status
------------ -------------

------------------------------------------------------------------
scstat: unexpected error.


---------- Post updated at 09:41 AM ---------- Previous update was at 09:24 AM ----------

svcadm clear cl-svc-enable
Quote:
Jul 24 09:28:25 tommy_sun svc.startd[8]: [ID 894785 daemon.error] Transitioning svc:/system/cluster/cl-svc-enable:default to maintenance because it completes a dependency cycle (see svcs -xv for details):
Jul 24 09:28:25 tommy_sun svc:/milestone/multi-user:default
Jul 24 09:28:25 tommy_sun svc:/network/cswmysql5
Jul 24 09:28:25 tommy_sun svc:/network/cswmysql5:default
Jul 24 09:28:25 tommy_sun svc:/system/cluster/sc_restarter:default
Jul 24 09:28:25 tommy_sun svc:/system/cluster/rgm-starter:default
Jul 24 09:28:25 tommy_sun svc:/system/cluster/pnm:default
Jul 24 09:28:25 tommy_sun svc:/system/cluster/cl-svc-enable:default
Jul 24 09:28:44 tommy_sun svc.startd[8]: [ID 894785 daemon.error] Transitioning svc:/system/cluster/cl-svc-enable:default to maintenance because it completes a dependency cycle (see svcs -xv for details):
Jul 24 09:28:44 tommy_sun svc:/milestone/multi-user:default
Jul 24 09:28:44 tommy_sun svc:/network/cswmysql5
Jul 24 09:28:44 tommy_sun svc:/network/cswmysql5:default
Jul 24 09:28:44 tommy_sun svc:/system/cluster/sc_restarter:default
Jul 24 09:28:44 tommy_sun svc:/system/cluster/rgm-starter:default
Jul 24 09:28:44 tommy_sun svc:/system/cluster/pnm:default
Jul 24 09:28:44 tommy_sun svc:/system/cluster/cl-svc-enable:default
# 2  
Old 07-26-2009
Hi,
it seems your first reboot after cconfig of your cluster.
did you try to get it up one by one. lets see its behavior.
get up only tommy_sun1.

Good luck
# 3  
Old 07-27-2009
Quote:
Originally Posted by samar
Hi,
it seems your first reboot after cconfig of your cluster.
did you try to get it up one by one. lets see its behavior.
get up only tommy_sun1.

Good luck
Hello Samar,

No - not first reboot after initial cluster install-configuration
Yes- first reboot after the mysql failover resources were configured

I can try only one node as you said, I will post update soon. Sorry for the delay.

Thanks

---------- Post updated at 09:02 AM ---------- Previous update was at 08:52 AM ----------

Only booted tommy_sun1, but still status is same as earlier. Now I am wondering if a patch for sun cluster messed up the things
svc:/system/cluster/cl-svc-enable:default (Enabling the cluster userland services)
State: maintenance since Mon Jul 27 08:47:59 2009
Reason: Completes a dependency cycle.
See: Sun Message ID: SMF-8000-HP
Impact: 25 dependent services are not running:
svc:/system/cluster/ql_upgrade:default
svc:/system/cluster/mountgfs:default
svc:/system/cluster/clusterdata:default
svc:/system/cluster/ql_rgm:default
svc:/system/zones:default
svc:/system/cluster/pnm:default
svc:/system/cluster/rgm-starter:default
svc:/application/management/common-agent-container-1:default
svc:/system/cluster/cl-svc-cluster-milestone:default
svc:/system/cluster/sc_svtag:default
svc:/system/cluster/sckeysync:default
svc:/system/cluster/scsymon-srv:default
svc:/system/cluster/scslmclean:default
svc:/system/cluster/sc_pnm_proxy_server:default
svc:/system/cluster/rpc-fed:default
svc:/system/cluster/sc_zones:default
svc:/system/cluster/scprivipd:default
svc:/system/cluster/cznetd:default
svc:/system/cluster/rpc-pmf:default
svc:/system/cluster/sc_zc_member:default
svc:/system/cluster/scdpm:default
svc:/system/cluster/cl-event:default
svc:/system/cluster/cl-eventlog:default
svc:/system/cluster/cl-ccra:default
svc:/system/cluster/scqdm:default

svc:/network/cswmysql5:default (?)
State: uninitialized since Mon Jul 27 08:47:59 2009
Reason: Restarter svc:/system/cluster/sc_restarter:default has not initialized service state.
See: Sun Message ID: SMF-8000-4D
Impact: 6 dependent services are not running:
svc:/milestone/multi-user:default
svc:/application/management/common-agent-container-1:default
svc:/milestone/multi-user-server:default
svc:/system/basicreg:default
svc:/system/zones:default
svc:/system/iscsitgt:default
# 4  
Old 07-28-2009
So you're getting something here ?
Clustering - scstat: unexpected error , sc 3.2
# 5  
Old 07-28-2009
Quote:
Originally Posted by incredible
So you're getting something here ?
Clustering - scstat: unexpected error , sc 3.2
not this time Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Getting this error syntax error near unexpected token `)'

Hi Everyone, my script was running Ok, but suddenly it started giving this error. ./update_env_bi.sh: line 54: syntax error near unexpected token `)' ./update_env_bi.sh: line 54: `sed -i "s/PORT=*1/PORT=$2/" repository.xml' The line 54 has this code. sed -i "s/PORT=*1/PORT=$2/"... (2 Replies)
Discussion started by: shajay12
2 Replies

2. Shell Programming and Scripting

IF section problem. syntax error: unexpected end of file error

Hello, I have another problem with my script. Please accept my apologies, but I am really nooby in sh scripts. I am writing it for first time. My script: returned=`tail -50 SapLogs.log | grep -i "Error"` echo $returned if ; then echo "There is no errors in the logs" fi And after... (10 Replies)
Discussion started by: jedzio
10 Replies

3. Shell Programming and Scripting

Getting unexpected error

Hi All, I am writing one script to automate one of the process and for that I am using below command at one line : uncompress `cat tmp` content in tmp file are : /home/vas/aqbatch/newbatch/archive/output/RIM_GUIDEPRICE_AQ_02108.DAT.Z... (5 Replies)
Discussion started by: NirajThakar
5 Replies

4. Shell Programming and Scripting

syntax error: `$' unexpected

Hi all, Am very new to Unix and am currently Involved in Migrating some Shell Scripts from AIX 4 to Solaris 10. While using teh for loop am getting the below error: $ echo $SHELL /usr/bin/ksh $ for file in $(ls *SEBE*) syntax error: `$' unexpected while the same works without issue on... (4 Replies)
Discussion started by: paragkhanore
4 Replies

5. Solaris

Unexpected error

When I typed #svcs -x sma I received this following error, svcs: svcs.c:335: Unexpected libscf error: invalid argument. Exiting. I have googled around but could not find a solution. IS this a bug? I am using Solaris 5.10 Generic_137137-09 sun4v sparc... (2 Replies)
Discussion started by: sundar63
2 Replies

6. Shell Programming and Scripting

unexpected syntax error

Hi, i am getting following syntax error ...kingly advice why is it coming ?? #!/bin/bash find . -name "common.log" if ; then echo "1" fi Himnashu@home /bin $ ./a.sh ./a.sh: line 7: syntax error near unexpected token `fi' ./a.sh: line 7: `fi' (9 Replies)
Discussion started by: himvat
9 Replies

7. Shell Programming and Scripting

sed error : Syntax error: redirection unexpected

My script is throwing the error 'Syntax error: redirection unexpected' My line of code.. cat nsstatustest.html | sed s/<tr><td align="left">/<tr><td align="left" bgcolor="#000000"><font color="white">/ > ztmp.Ps23zp2s.2-Fpps3-wmmm0dss3 HTML tags are getting in the way but they're needed to... (3 Replies)
Discussion started by: phpfreak
3 Replies

8. Shell Programming and Scripting

unexpected crontab error

Yesterday i installed a new cron on solaris box. After a few minutes after loading the cron, i got the following error. Your "crontab" on machine_name ! unexpected end of line This entry has been ignored. Do any of you have any idea about this error ? (12 Replies)
Discussion started by: dhanamurthy
12 Replies

9. UNIX for Dummies Questions & Answers

unexpected error

./sample.5: syntax error: `(' unexpected I get this error and Im trying to solve it but the line that it refering to is a notation so I don't understand why it is affecting the program. (2 Replies)
Discussion started by: helpme2008
2 Replies

10. UNIX for Dummies Questions & Answers

awk Shell Script error : "Syntax Error : `Split' unexpected

hi there i write one awk script file in shell programing the code is related to dd/mm/yy to month, day year format but i get an error please can anybody help me out in this problem ?????? i give my code here including error awk ` # date-month -- convert mm/dd/yy to month day,... (2 Replies)
Discussion started by: Herry
2 Replies
Login or Register to Ask a Question