Emergency UNIX and Linux Support

Hi friends,
I am quite used to using glance in HPUX servers for analysis performance issues with a particular process as requested by app. folks.

The options which are very helpful to me are the "F" , "W" , "L" stats. How can i get the similar details on Linux servers without using glance? Unfortately we can't have glance due to cost cutting . I am currently having collectl installed there and i checked with its developer and he says we can't using collectl. Can you please suggest me a tool to get this useful data or please confirm that we simply can't have this data in Linux.

For quick reference, these are the options which i am refering to collectl on HPUX server:

+++++++++++++++++++++++++++++++++++++++++++++++++++
F - Process Open Files
W - Process Wait States
L - Process System Calls

Glance Commands Menu

h - Online Help q - exit (or e) A - Application List
g - Process List d - Disk Report P - PRM Group List
a - CPU By Processor i - IO By File System Y - Global System Calls
c - CPU Report u - IO By Disk F - Process Open Files
m - Memory Report U - IO By HBA Card M - Process Memory Regions
t - System Tables v - IO By Logical Volume R - Process Resources
w - Swap Space N - NFS Global Activity W - Process Wait States
B - Global Waits n - NFS By System L - Process System Calls
Z - Global Threads l - Network By Interface y - Renice Process
G - Process Threads T - Trans Tracker s - Select Process
I - Thread Resource H - Alarm History J - Thread Wait
+++++++++++++++++++++++++++++++++++++++++++++++++++



e.g output of these.

*****************************************************************************
-F output >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Open Files PID: 21212, ora_p060_PNB PPID: 1 euid: 150 Userracle
Open Open
FD File Name Type Mode Count Offset

0 /dev/null chr read 1 0
1 /dev/null chr write 1 0
2 /dev/null chr write 1 0
3 /dev/null chr read 489 0
4 /dev/null chr read 489 0
5 reg write 163 40898
6 /dev/null chr read 489 0
7 /usr/local/oracle/11.1.0/dbs/hc_PNBI36A.dat reg rd/wr 163 0
8 reg rd/wr 163 0
9 /dev/null chr read 330 0
10 /logs/.../PNBI36A/trace/PNBI36A_ora_29064.trm reg write 163 36057
11 /dev/null chr read 330 0
12 /dev/null chr read 3 0
13 /dev/null chr read 330 0
14 /usr/local/oracle/11.1.0/dbs/hc_PNBI36A.dat reg rd/wr 110 0
15 /usr/local/oracle/11.1.0/dbs/lkPNBI36A reg rd/wr 110 0
16 /dev/null chr read 3 0
17 /usr/local/oracle/11.1.0/rdbms/mesg/oraus.msb reg read 1 203776

-W output >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Resources PID: 21212, ora_p060_SBC PPID: 1 euid: 150 Userracle

CPU Usage (util): 0.0 Log Reads : 0 Wait Reason : OTHER
User/Nice/RT CPU: 0.0 Log Writes: 0 Total RSS/VSS : 15.1mb/ 19.2mb
System CPU : 0.0 Phy Reads : 0 Traps / Vfaults: 0/ 0
Interrupt CPU : 0.0 Phy Writes: 0 Faults Mem/Disk: 0/ 0
Cont Switch CPU : 0.0 FS Reads : 0 Deactivations : 0
Scheduler : NOAGE FS Writes : 0 Forks & Vforks : 0
Priority : 178 VM Reads : 0 Signals Recd : 0
Nice Value : 20 VM Writes : 0 Mesg Sent/Recd : 0/ 0
Dispatches : 1 Sys Reads : 0 Other Log Rd/Wt: 0/ 0
Forced CSwitch : 0 Sys Writes: 0 Other Phy Rd/Wt: 0/ 0
VoluntaryCSwitch: 1 Raw Reads : 0 Proc Start Time
Running CPU : 112 Raw Writes: 0 Mon Apr 7 01:31:53 2014
CPU Switches : 0 Bytes Xfer: 0kb :
Cmd : ora_p060_SBCI36A

-L output >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

Sys Calls PID: 21212, ora_p060_PNB PPID: 1 euid: 150 Userracle
Elapsed Elapsed
System Call Name ID Count Rate Time Cum Ct CumRate CumTim

*****************************************************************************

Thanks,
Kunwar ( My bits with this question )

All of that data is available in the /proc "filesystem" which is really a kernel maintained readonly dataset that has tons of information.

Since what you need is spread acoss several /proc files consider this primer on /proc
/proc

Note that if this is, for example, Oracle RedHat (or some other flavors of Linux) there are differences in what is or is not supported. Compared to the link above.

kunwar - to be clear, collectl CAN get a lot of data from /proc but as I told you earlier, not all of that information is available by process in /proc, at least not in /proc/pid where collectl gets its data.

Specifically I'm talking about your big 3: open files, system calls and process wait states. Collectl DOES report the number of system calls as reported in /proc/pid/io, but those are only for read/write I/O calls.

There is some process level info on waiting but don't think it's extensive. Furthermore collectl gets most data from stat and io, and a little from status. I've got eliminating reading status on my todo list since reading all this data for nearly 1000 processes can get pretty expensive and I don't want to read more data, I want to read less. My overarching philosophy for collectl is first and foremost 'do no harm' as far as performance goes.

-mark