Sponsored Content
Top Forums Shell Programming and Scripting Monitoring processes in parallel and process log file after process exits Post 303003294 by shunya on Tuesday 12th of September 2017 12:42:53 AM
Old 09-12-2017
Monitoring processes in parallel and process log file after process exits

I am writing a script to kick off a process to gather logs on multiple nodes in parallel using "&". These processes create individual log files. Which I would like to filter and convert in CSV format after they are complete. I am facing following issues:
1. Monitor all Processes parallelly. whichever process completes first I would like to convert o/p log file of corresponding process to csv format. Sometimes process might take more than 30 mins to complete. Problem with my code is it will go serially converting files and not any process which completes or releases the file first. Please help

I have following code
Code:
#!/bin/sh -x

infile="$1"
PWD=`pwd`
shift
exec 2>&1
for IPADD in `cat $infile|awk '{print $1}' |tr -d '\015'`
do
OPFILE=${IPADD}_syslog.txt
nohup ${PWD}/collect_log.sh $IPADD > ${OPFILE} &
DPID=$!
echo -e "$DPID $OPFILE" >> pid_pfile.txt
done

##### Loop to parse and rename the files after data collection is complete.
check_palive()
{
PALIVE=`ps cax | grep $DPID | grep -o '^[ ]*[0-9]*'`
if [ -z $PALIVE ];then
HNAME=`grep -i hostname |awk '{print $NF}'`
awk '/name/,/exit/'  $OPFILE |head -n -1 |awk '{print $1,$2,$3,$4,$5}'> ${HNAME}.txt
fi
}


PCOUNT=`pgrep collect_log.sh |wc -l`

while [ $PCOUNT -gt 0 ];do

      for DPID in `pgrep collect_log.sh |awk '{print $1}'`
       do

       PALIVE=`ps -p $DPID --no-headers | wc -l`
          if [ $PALIVE == 0 ];then
            wait $DPID
            OPFILE=`grep $DPID pid_pfile.txt|awk '{print $2}'`
            check_palive
           # sed "/$DPID/d" pid_pfile.txt > pid_pfile.txt
        else
          sleep 120
       fi
done

PCOUNT=`pgrep "junk" |wc -l`
done
rm -f pid_pfile.txt

---------- Post updated at 11:42 PM ---------- Previous update was at 11:38 PM ----------

$infile has list of IP addresses of nodes.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

killing parallel oracle process

Hi guys: I have a an oracle job which uses 10 parallel hints and would like to killit when it hangs. I want to kill all the processes that have been spawned. what I do right now is get the pid of the scheduler process which initiated theis job and the do a ps -ef| grep 'pid' and trace through... (1 Reply)
Discussion started by: oracle8
1 Replies

2. Shell Programming and Scripting

split process files in parallel and rejoin

Hi I need to split a huge file into multiple smaller files using split command. After that i need to process each file in the back ground with sql loader .Sql loader is a utlity to load CSV files into oracle . Check the status of each of these sqlloaders and then after sucessfull... (6 Replies)
Discussion started by: xiamin
6 Replies

3. Shell Programming and Scripting

process monitoring

hi all, i would like to write the shell script to monitoring the processing, but if i passing the parameter the number of process is incorrect how to slove it? many thx got the correct number of process as following script: ===========================================================... (3 Replies)
Discussion started by: eric_wong_ch
3 Replies

4. UNIX and Linux Applications

How can i see if a unix Process Aplication i.e oracle is running in parallel

There is a unix process process in oracle running and i see running by typing ps -fea|grep GE_CLIENTES. The question is How can i see if this process is running in paralel. I dont know with a Unix command or specifically its a comand from Oracle. I kow a Parallel process ia a process that... (1 Reply)
Discussion started by: alexcol
1 Replies

5. Shell Programming and Scripting

Automate Log Monitoring Process

I am a new member of this forum and am also new to unix shell scripting. I joined the forum to seek for help to achieve my task as this forum helps people. here's what i do manually on daily basis 1)Loginto different unix box 2)Ftp the log files (morethan 50 each dir) to windows 3)use text pad... (3 Replies)
Discussion started by: sharugan
3 Replies

6. Shell Programming and Scripting

Finding the age of a unix process, killing old processes, killing zombie processes

I had issues with processes locking up. This script checks for processes and kills them if they are older than a certain time. Its uses some functions you'll need to define or remove, like slog() which I use for logging, and is_running() which checks if this script is already running so you can... (0 Replies)
Discussion started by: sukerman
0 Replies

7. Shell Programming and Scripting

Parallel process in Perl

HI All, I have scenerio where I need to call sub modules through for loop for (i=0; i<8000 ;i++) { .. BLOCKA } BLOCKA { .. .. subroutine 1; subroutine 2; } I want this to be run in parallel process1 BLOCKA { (6 Replies)
Discussion started by: gvk25
6 Replies

8. Programming

Parallel process in java

Hello; Please Are both threads execute in parallel? Thank you (4 Replies)
Discussion started by: chercheur857
4 Replies

9. UNIX for Dummies Questions & Answers

Running parallel process

i am having 4 process,have to run parallel and not after one by one. sample1.sh sample2.sh sample3.sh sample4.sh Thanks in advance. i (11 Replies)
Discussion started by: sagar_1986
11 Replies

10. Shell Programming and Scripting

Make process parallel

Hi, I have a file which has a list of 200 tables e.g: table.txt I need to do a count for each table and store it in a file. So I did something like this: for TABLE in `cat table.txt` do T_CNT=$(sqlplus -s -l / as sysdba <<EOF set echo off heading off feadback off SELECT count(*) FROM... (1 Reply)
Discussion started by: wahi80
1 Replies
GREP(1) 						      General Commands Manual							   GREP(1)

NAME
grep - search a file for a pattern SYNOPSIS
grep [ option ... ] pattern [ file ... ] DESCRIPTION
Grep searches the input files (standard input default) for lines (with newlines excluded) that match the pattern, a regular expression as defined in regexp(6). Normally, each line matching the pattern is `selected', and each selected line is copied to the standard output. The options are -c Print only a count of matching lines. -h Do not print file name tags (headers) with output lines. -i Ignore alphabetic case distinctions. The implementation folds into lower case all letters in the pattern and input before interpre- tation. Matched lines are printed in their original form. -l (ell) Print the names of files with selected lines; don't print the lines. -L Print the names of files with no selected lines; the converse of -l. -n Mark each printed line with its line number counted in its file. -s Produce no output, but return status. -v Reverse: print lines that do not match the pattern. Output lines are tagged by file name when there is more than one input file. (To force this tagging, include /dev/null as a file name argument.) Care should be taken when using the shell metacharacters $*[^|()= and newline in pattern; it is safest to enclose the entire expression in single quotes '...'. SOURCE
/sys/src/cmd/grep.c SEE ALSO
ed(1), awk(1), sed(1), sam(1), regexp(6) DIAGNOSTICS
Exit status is null if any lines are selected, or non-null when no lines are selected or an error occurs. GREP(1)
All times are GMT -4. The time now is 10:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy