Visit Our UNIX and Linux User Community

Operating Systems AIX Crash dump and Panic message : RSCT Dead Man Switch Timeout for HACMP; halting non-responsive node Post 303042416 by zxmaus on Tuesday 24th of December 2019 01:08:46 AM
Old 12-24-2019
You lost all heartbeats from node 1 to node 2 - thats the reason for the crash. This might happen when your system is simply too busy - but since you should have both heartbeat on disk and heartbeat via network, you should think that there is time enough to send at least one every couple of seconds, Your cluster heartbeat settings might be too tight - giving it more time for the heartbeat might help preventing this issue in the future.
Just out of curiosity - using GPFS and HACMP and RAC on the same systems appears to me to be a completely unnecessary setup, as you are running essentially 3 different cluster products on a system when RAC alone would suffice. Why ?
This User Gave Thanks to zxmaus For This Post:

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

help, what is the difference between core dump and panic dump?

help, what is the difference between core dump and panic dump? (1 Reply)
Discussion started by: aileen
1 Replies

2. HP-UX

crash dump

hi friends, i know that when there is a crash then that memory image is put into /var/adm/crash but if the system hangs up and if i have access to console of that machine then how can i take the crash dump manully. thanks (2 Replies)
Discussion started by: mxms755
2 Replies

3. Solaris

crash dump

Can anyone of you help me in enabling crash dump on Solaris 5.5.1 (1 Reply)
Discussion started by: csreenivas
1 Replies

4. AIX

Node Switch Reasons in HACMP

Hi Guys, I have two nodes clustered. Each node is AIX 5.2 & they are clustered with HACMP 5.2. The mode of the cluster is Active/Passive which mean one node is the Active node & have all resource groups on it & the 2nd node is standby. Last Monday I noted that all resource groupes have been... (2 Replies)
Discussion started by: aldowsary
2 Replies

5. Solaris

crash dump

hi , i have machine that is crashed how i can enable core dump file & how can i find it ? :confused: (4 Replies)
Discussion started by: lid-j-one
4 Replies

6. UNIX for Advanced & Expert Users

Linux heartbeat on redhat 4:node dead

Hi. I have started heartbeat on two redhat servers. Using eth0. Before I start heartbeat I can ping the two server to each other. Once I start heartbeat both the server become active as they both have warnings that the other node is dead. Also I am not able to ping each other. After stopping... (1 Reply)
Discussion started by: amrita garg
1 Replies

7. AIX

hacmp in a 7 node configuration ?

Hi Guys, I have to design a multinode hacmp cluster and am not sure if the design I am thinking of makes any sense. I have to make an environment that currently resides on 5 nodes more resilient but I have the constrain of only having 4 frames. In addition the business doesnt want to pay for... (7 Replies)
Discussion started by: zxmaus
7 Replies

8. AIX

HACMP switch over

Hi I had an active passive cluster. Node A went down and all resource groups moved to Node B. Now we brought up Node A. What is the procedure to bring everything back to Node A. Node A #lssrc -a | grep cl clcomdES clcomdES 323782 active clstrmgrES cluster... (9 Replies)
Discussion started by: samsungsamsung
9 Replies

9. HP-UX

Prevent crash dump when SG cluster node reboots

Hi Experts, I have configured HP-UX Service Guard cluster and it dumps crash every time i reboot a cluster node. Can anyone please help me to prevent these unnecessary crash dumps at the time of rebooting SG cluster node? Thanks in advance. Vaishey (2 Replies)
Discussion started by: Vaishey
2 Replies

10. OS X (Apple)

MacOS 10.15.2 Catalina display crash and system panic

MacPro (2013) 12-Core, 64GB RAM (today's crash): panic(cpu 2 caller 0xffffff7f8b333ad5): userspace watchdog timeout: no successful checkins from in 120 seconds service:, total successful checkins since load (318824 seconds ago): 31883, last successful... (3 Replies)
Discussion started by: Neo
3 Replies
o2cb(7) 							OCFS2 Manual Pages							   o2cb(7)

o2cb - Default cluster stack for the OCFS2 file system. DESCRIPTION
o2cb is the default cluster stack for the OCFS2 file system. It includes a node manager (o2nm) to keep track of the nodes in the cluster, a heartbeat agent (o2hb) to detect live nodes, a network agent (o2net) for intra-cluster node communication and a distributed lock manager (o2dlm) to keep track of lock resources. All these components are in-kernel. It also includes an in-memory file system, dlmfs, to allow userspace to access the in-kernel dlm. This cluster stack has two configuration files, namely, /etc/ocfs2/cluster.conf and /etc/sysconfig/o2cb. Whereas the former keeps track of the cluster layout, the latter keeps track of the cluster timeouts. Both files are only read when the cluster is brought online. Values in use by the online cluster can be perused in the /sys/kernel/config/cluster directory structure. CONFIGURATION
The cluster layout is specified in /etc/ocfs2/cluster.conf. While it is easier to populate and propagate this configuration file using ocfs2console(8), one can also do it by manually as long as care is taken to format the file correctly. While the console utility is intuitive to use, there are few points to keep in mind. 1. The node name needs to match the hostname. It does not need to include the domain name. For example, can be appserver. 2. The IP address need not be the one associated with that hostname. As in, any valid IP address on that node can be used. O2CB will not attempt to match the node name (hostname) with the specified IP address. For best performance, use of a private interconnect (lower latency) is recommended. The cluster.conf file is in a stanza format with two types of stanzas, namely, cluster and node. A typical cluster.conf will have one clus- ter stanza and multiple node stanzas. The cluster stanza has two parameters: node_count Total number of nodes in the cluster name Name of the cluster The node stanza has five parameters: ip_port IP port ip_address IP address number Unique node number from 0-254 name Hostname cluster Name of the cluster Users populating cluster.conf manually should follow the format strictly. As in, stanza header should start at the first column and end with a colon, stanza parameters should start after a tab, a blank line should demarcate each stanza and care taken to avoid stray white- spaces. The O2CB cluster timeouts are specified in /etc/sysconfig/o2cb and can be configured using the o2cb init script. These timeouts are used by the O2CB clusterstack to determine whether a node is dead or alive. While the use of default values is recom- mended, users can experiment with other values if the defaults are causing spurious fencing. The cluster timeouts are: Heartbeat Dead Threshold The Disk Heartbeat timeout is the number of two second iterations before a node is considered dead. The exact formula used to con- vert the timeout in seconds to the number of iterations is as follows: O2CB_HEARTBEAT_THRESHOLD = (((timeout in seconds) / 2) + 1) For e.g., to specify a 60 sec timeout, set it to 31. For 120 secs, set it to 61. The default for this timeout is 60 secs (O2CB_HEARTBEAT_THRESHOLD = 31). Network Idle Timeout The Network Idle timeout specifies the time in milliseconds before a network connection is considered dead. It defaults to 30000 ms. Network Keepalive Delay The Network Keepalive specifies the maximum delay in milliseconds before a keepalive packet is sent to another node to check whether it is alive or not. If the node is alive, it will respond. Its defaults to 2000 ms. Network Reconnect Delay The Network Reconnect specifies the minimum delay in milliseconds between connection attempts. It defaults to 2000 ms. EXAMPLES
A sample /etc/ocfs2/cluster.conf. cluster: node_count = 3 name = webcluster node: ip_port = 7777 ip_address = number = 7 name = node7 cluster = webcluster node: ip_port = 7777 ip_address = number = 6 name = node6 cluster = webcluster node: ip_port = 7777 ip_address = number = 10 name = node10 cluster = webcluster SEE ALSO
mkfs.ocfs2(8) fsck.ocfs2(8) tunefs.ocfs2(8) debugfs.ocfs2(8) ocfs2console(8) AUTHORS
Oracle Corporation COPYRIGHT
Copyright (C) 2004, 2010 Oracle. All rights reserved. Version 1.6.4 September 2010 o2cb(7)

Featured Tech Videos

All times are GMT -4. The time now is 01:57 AM.
Unix & Linux Forums Content Copyright 1993-2021. All Rights Reserved.
Privacy Policy