Server hung, is this a stack trace? Post: 302885751

Sponsored Content

Operating Systems Linux Server hung, is this a stack trace? Post 302885751 by badoshi on Tuesday 28th of January 2014 08:41:04 AM

01-28-2014

Registered User

Server hung, is this a stack trace?

Hi everyone,

Our Red Hat server hung yesterday, and I managed to log into the console and see the following message:

Code:

RIP: 0010:[<ffffffff8006b981>]  [<ffffffff8006b981>] mwait_idle_with_hints+0x66/
0x67
RSP: 0018:ffffffff80457f40  EFLAGS: 00000046
RAX: 0000000000000010 RBX: ffff810c20075910 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000010
RBP: 134d6ef57a8e3fe8 R08: 0000000000000033 R09: 0000000000000033
R10: ffff810001005710 R11: ffffffff8014c2c9 R12: 0000000000000000
R13: ffff810c20075800 R14: 0000000000000000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffffff80425000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00000000f7fb6980 CR3: 0000000c016ba000 CR4: 00000000000006e0
Process swapper (pid: 0, threadinfo ffffffff80456000, task ffffffff80310b60)
Stack:  ffffffff801a1337 0000000000000000 000000000000039c 0000000000000000
 000000000000039c 0000000000000000 ffffffff801a11e1 0000000000090000
 0000000000000000 0000000000000000 ffffffff800492fd 0000000000200800
Call Trace:
 [<ffffffff801a1337>] acpi_processor_idle_simple+0x156/0x316
 [<ffffffff801a11e1>] acpi_processor_idle_simple+0x0/0x316
 [<ffffffff800492fd>] cpu_idle+0x95/0xb8
 [<ffffffff80461807>] start_kernel+0x220/0x225
 [<ffffffff8046122f>] _sinittext+0x22f/0x236


Code: c3 41 57 41 56 49 89 f6 41 55 49 89 fd 41 54 4c 8d a7 e0 02

I had to reset the server, as I couldnt ping, or get a prompt, and the above is pretty much all we have to go on. I can't find anything meaningful in /var/log/messages or dmesg. Any ideas what the above output is, or what it indicates?

Many thanks.

badoshi

View Public Profile for badoshi

Find all posts by badoshi

10 More Discussions You Might Find Interesting

1. Programming

Core file without a readable stack trace

I am using gdb to examine a core file but the output contains only the method addresses in hex. Is there anyway to translate these addresses to a human-readable form? :confused:

2. Programming

printing a stack trace with backtrace

I am trying to print a stack trace programatically using backtrace and backtrace_symbols. The problem is that the stack being printed in a mangled format. Is there a way to get the output in more of a human readable form? I am using Red Hat and the program is written in c++.

3. UNIX for Advanced & Expert Users

help required - stack trace

Hi all, One of our programs written in Java, produced this logfile. This job runs 48 threads and only one thread failed with this error. The code is a blackbox(external product), so cant look at the source code. From what I can infer from the log, the job was trying to write the log messages into...

4. Filesystems, Disks and Memory

get stack trace from C program on Solaris 8

I'm on solaris 8. I need to check the stack trace inside my C program. I don't have printstack or walkstack. I tested getcontext and it works. But how do I get the symbols from "stack_t" ? Help please. Many thanks!

5. Solaris

Need to retrieve stack trace from core using mdb

Hello , I use Solaris 5.10 . I have huge core file , 48 GB , resulted from an application that was running and got crashed with SIGSEGV. On my system only mdb works. Please help me to retrieve the stack trace from this core file. I am novice to mdb and its nuaunces. Please help me with...

6. AIX

How to get the stack trace using kdb?

Hi All, I am trying to debug my core file using kdb. When I try to get the stack trace I am facing this error. core mapped from @ 700000000000000 to @ 70000000306fc04 Preserving 1680415 bytes of symbol table Dump does not start with valid magic number WARNING: Possibly truncated or...

7. Shell Programming and Scripting

Stack Trace

Hi All Thought it would be kind of fun to implement a stack trace for a shell script that calls functions within a sub shell. This is for bash under Linux and probably not portable - #! /bin/bash error_exit() { echo "=======================" echo $1 echo...

8. UNIX for Dummies Questions & Answers

View User Mode Call Stack of Hung Process

I have a multithreaded usermode program(actually a daemon) which is in hanged state. To debug it I tried attaching the process to gdb, but the gdb hangs. gstack also gets hanged. I peeped into the proc file system and saw the process to be in sleeping state. /proc/sysrq-trigger I guess...

9. Programming

View Stack Trace of different cores

I have a C program which is running as daemon and has some threads. The program is running on dual core cpu and it may happen that different threads may run on different cpu core. The problem is sometimes it crashes with some heap memory corruption probably between threads. GDB command(t a a...

10. Shell Programming and Scripting

Java stack trace parser in awk

I want the developers to get a mail with Java stack traces on a daily bases. When something is flaged as known issue and will get a fix but mean while this does not need to get sent each dayl. This is what I got so far. It's a bash script that runs some AWK in it. To get the files that needs to...

LEARN ABOUT CENTOS

trace-cmd-start

TRACE-CMD-START(1)														TRACE-CMD-START(1)

NAME

       trace-cmd-start - start the Ftrace Linux kernel tracer without recording

SYNOPSIS

       trace-cmd start [OPTIONS]

DESCRIPTION

       The trace-cmd(1) start enables all the Ftrace tracing the same way trace-cmd-record(1) does. The difference is that it does not run threads
       to create a trace.dat file. This is useful just to enable Ftrace and you are only interested in the trace after some event has occurred and
       the trace is stopped. Then the trace can be read straight from the Ftrace pseudo file system or can be extracted with trace-cmd-extract(1).

OPTIONS

       The options are the same as trace-cmd-record(1), except that it does not take options specific to recording (-s, -o, -F, -N, and -t).

SEE ALSO

       trace-cmd(1), trace-cmd-record(1), trace-cmd-report(1), trace-cmd-stop(1), trace-cmd-extract(1), trace-cmd-reset(1), trace-cmd-split(1),
       trace-cmd-list(1), trace-cmd-listen(1)

AUTHOR

       Written by Steven Rostedt, <rostedt@goodmis.org[1]>

RESOURCES

       git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/trace-cmd.git

COPYING

       Copyright (C) 2010 Red Hat, Inc. Free use of this software is granted under the terms of the GNU Public License (GPL).

NOTES

	1. rostedt@goodmis.org
	   mailto:rostedt@goodmis.org

								    06/11/2014							TRACE-CMD-START(1)