Sponsored Content
Top Forums UNIX for Advanced & Expert Users Facing file processing timeouts Post 302144916 by videsh77 on Sunday 11th of November 2007 11:54:52 PM
Old 11-12-2007
Facing file processing timeouts

Hi

In our ETL application we have used simple scripts to move & split the files. We are randomly facing timeouts on these jobs. It indicates we are running out of resources.

How should I confirm the resources are inadequate?

I know we have commands like vmstat & iostat which displays performance stats. Problem with them they mention some disks in their outputs, but I am not sure how the paths/dirs those files are being operated maps to those disks?

Any suggestion to track this?
 

9 More Discussions You Might Find Interesting

1. Cybersecurity

UNIX Network timeouts

Hi, can anyone point me in the right direction regarding a UNIX setting that determines when a network connection will timeout? I am getting network timeouts and I would like to know if there is a setting in UNIX 11i OS that I can modify to increase the time limit. Thank you (4 Replies)
Discussion started by: Allano
4 Replies

2. UNIX for Dummies Questions & Answers

Device Timeouts on Unix Server

Hi I am brand new to the forum I have very limited understanding of unix, but am very computer literate. At work we have a unix server which has been the bane of our lives. We have stopped being able to access the network drives and the screen has lots of device timeouts. I have learned how to do... (1 Reply)
Discussion started by: scovell01
1 Replies

3. Shell Programming and Scripting

Facing problems when FTPING file from server to local (Very Urgent)

Hi, when i am trying to FTP one file (test.txt) from Unix server to my local machine then i am getting the following error, as shown below ftp> binary 200 Type set to I. ftp> put C:\FTP\test.txt C:FTPtest.txt: No such file or directory ftp> The red color text is the error. Can... (3 Replies)
Discussion started by: sunitachoudhury
3 Replies

4. Shell Programming and Scripting

How to make parallel processing rather than serial processing ??

Hello everybody, I have a little problem with one of my program. I made a plugin for collectd (a stats collector for my servers) but I have a problem to make it run in parallel. My program gathers stats from logs, so it needs to run in background waiting for any new lines added in the log... (0 Replies)
Discussion started by: Samb95
0 Replies

5. Shell Programming and Scripting

Timeouts in expect script

Hi all. I have an expect script which for walks across servers, checks freespace and provides sorted list of biggest files #!/usr/bin/expect -- set timeout 600 stty -echo log_user 0 spawn -noecho sudo -u introot /home/introot/bin/twssh expect root send_user "Entering \r" send "uname\r"... (15 Replies)
Discussion started by: urello
15 Replies

6. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

7. UNIX for Dummies Questions & Answers

Facing difficulties to untar a tra.gz file

Hi , I trying to cp a files to one location and trying to untar there,the tar.gz files get copied to the specified location but wont untar. Here is what i did if cp /path/of/the/file.tar.gz /path/to/the/file/file.tar.gz then tar -zxvf /path/to/the/file/file.tar.gz echo "FILE... (2 Replies)
Discussion started by: vikatakavi
2 Replies

8. UNIX for Dummies Questions & Answers

awk - Rename output file, after processing, same as input file

I have one input file ABC.txt and one output DEF.txt. After the ABC is processed and created output, I want to rename ABC.txt to ABC.orig and DEF to ABC.txt. Currently when I am doing this, it does not process the input file as it cannot read and write to the same file. How can I achieve this? ... (12 Replies)
Discussion started by: High-T
12 Replies

9. IP Networking

Problems and doubts with sockets and timeouts

Hi! I need some help to understand a little bit more the behaviour about socket and TCP connections... Here is my problem I have a client and a server that were written in python. The server program wait until a message arrive and then print the message but if the message not arrive in a second... (2 Replies)
Discussion started by: Kovalevski
2 Replies
vmstat(1M)						  System Administration Commands						vmstat(1M)

NAME
vmstat - report virtual memory statistics SYNOPSIS
vmstat [-cipqsS] [disks] [ interval [count]] DESCRIPTION
vmstat reports virtual memory statistics regarding kernel thread, virtual memory, disk, trap, and CPU activity. On MP (multi-processor) systems, vmstat averages the number of CPUs into the output. For per-processor statistics, see mpstat(1M). vmstat only supports statistics for certain devices. For more general system statistics, use sar(1), iostat(1M), or sar(1M). Without options, vmstat displays a one-line summary of the virtual memory activity since the system was booted. During execution of the kernel status command, the state of the system can change. If relevant, a state change message is included in the vmstat output, in one of the following forms: <<device added: sd0>> <<device removed: sd0>> <<processors added: 1, 3>> <<processors removed: 1, 3>> See System Administration Guide: Advanced Administration for device naming conventions for disks. OPTIONS
The following options are supported: -c Report cache flushing statistics. This option is obsolete, and no longer meaningful. This option might be removed in a future version of Solaris. -i Report the number of interrupts per device. count and interval does not apply to the -i option. -p Report paging activity in details. This option will display the following, respectively: epi Executable page-ins. epo Executable page-outs. epf Executable page-frees. api Anonymous page-ins. apo Anonymous page-outs. apf Anonymous page-frees. fpi File system page-ins. fpo File system page-outs. fpf File system page-frees. When executed in a zone and if the pools facility is active, all of the above only report actitivity on the processors in the pro- cessor set of the zone's pool. -q Suppress messages related to state changes. -s Display the total number of various system events since boot. count and interval does not apply to the -s option. -S Report on swapping rather than paging activity. This option will change two fields in vmstat's ``paging'' display: rather than the ``re'' and ``mf'' fields, vmstat will report ``si'' (swap-ins) and ``so'' (swap-outs). OPERANDS
The following operands are supported: count Specifies the number of times that the statistics are repeated. count does not apply to the -i and -s options. disks Specifies which disks are to be given priority in the output (only four disks fit on a line). Common disk names are id, sd, xd, or xy, followed by a number (for example, sd2, xd0, and so forth). interval Specifies the last number of seconds over which vmstat summarizes activity. This number of seconds repeats forever. inter- val does not apply to the -i and -s options. EXAMPLES
Example 1: Using vmstat The following command displays a summary of what the system is doing every five seconds. example% vmstat 5 kthr memory page disk faults cpu r b w swap free re mf pi p fr de sr s0 s1 s2 s3 in sy cs us sy id 0 0 0 11456 4120 1 41 19 1 3 0 2 0 4 0 0 48 112 130 4 14 82 0 0 1 10132 4280 0 4 44 0 0 0 0 0 23 0 0 211 230 144 3 35 62 0 0 1 10132 4616 0 0 20 0 0 0 0 0 19 0 0 150 172 146 3 33 64 0 0 1 10132 5292 0 0 9 0 0 0 0 0 21 0 0 165 105 130 1 21 78 1 1 1 10132 5496 0 0 5 0 0 0 0 0 23 0 0 183 92 134 1 20 79 1 0 1 10132 5564 0 0 25 0 0 0 0 0 18 0 0 131 231 116 4 34 62 1 0 1 10124 5412 0 0 37 0 0 0 0 0 22 0 0 166 179 118 1 33 67 1 0 1 10124 5236 0 0 24 0 0 0 0 0 14 0 0 109 243 113 4 56 39 ^C example% The fields of vmstat's display are kthr Report the number of kernel threads in each of the three following states: r the number of kernel threads in run queue b the number of blocked kernel threads that are waiting for resources I/O, paging, and so forth w the number of swapped out lightweight processes (LWPs) that are waiting for processing resources to finish. memory Report on usage of virtual and real memory. swap available swap space (Kbytes) free size of the free list (Kbytes) page Report information about page faults and paging activity. The information on each of the following activities is given in units per second. re page reclaims -- but see the -S option for how this field is modified. mf minor faults -- but see the -S option for how this field is modified. pi kilobytes paged in po kilobytes paged out fr kilobytes freed de anticipated short-term memory shortfall (Kbytes) sr pages scanned by clock algorithm When executed in a zone and if the pools facility is active, all of the above (except for "de") only report activity on the processors in the processor set of the zone's pool. disk Report the number of disk operations per second. There are slots for up to four disks, labeled with a single letter and number. The letter indicates the type of disk (s = SCSI, i = IPI, and so forth); the number is the logical unit number. faults Report the trap/interrupt rates (per second). in interrupts sy system calls cs CPU context switches When executed in a zone and if the pools facility is active, all of the above only report actitivity on the processors in the processor set of the zone's pool. cpu Give a breakdown of percentage usage of CPU time. On MP systems, this is an average across all processors. us user time sy system time id idle time When executed in a zone and if the pools facility is active, all of the above only report actitivity on the processors in the processor set of the zone's pool. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |Interface Stability |See below. | +-----------------------------+-----------------------------+ Invocation is evolving. Human readable output is unstable. SEE ALSO
sar(1), iostat(1M), mpstat(1M), sar(1M), attributes(5) System Administration Guide: Basic Administration System Administration Guide: Advanced Administration NOTES
The sum of CPU utilization might vary slightly from 100 because of rounding errors in the production of a percentage figure. The -c option (Report cache flushing statistics) is not supported in this release. SunOS 5.10 20 Dec 2004 vmstat(1M)
All times are GMT -4. The time now is 04:56 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy