While loop hangs in function running in background
Hello Everyone,
I am writing a shell script to fetch log files from remote servers within a time range. It copies log files to local server, grep for date and then compares the time stamp of each log entry with the once specified.
Below is the code.
Code:
# Get log and Parsing function
GetAndParse() {
echo -e "\nWorking on server $host"
# Looping through the log paths given in the argument.
for logpath in $LOG_PATH
do
echo "Fetching log files from $logpath"
# Looping through the log file names provided in the arguement.
for logfile in $FILE_NAMES
do
# Finding the log file size on remote server
logfilesize=$(ssh -n $host du -skLc $logpath/$logfile 2> /dev/null|awk '{print $1}'|tail -1)
# Comapring the log file size with limit LOG_FILE_SIZE
if [ ! -z $logfilesize ] && [ $logfilesize -le $LOG_FILE_SIZE ] && [ $logfilesize -gt 0 ]
then
# If the log file size is under limit copt it to local WORK_DIR
scp $host:$logpath/$logfile $WORK_DIR/temp
LOCAL_TEMP_FILE="$WORK_DIR"/temp/"$host"_"$logfile"_"$RANDOM".temp
# Grep for dates in log entries
cat $WORK_DIR/temp/$logfile| grep -E "$START_TIME|$END_TIME" > $LOCAL_TEMP_FILE && rm $WORK_DIR/temp/$logfile
while read line
do
line_date1=$(echo $line|grep -o "[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]")
line_date=$(echo $line_date1|tr -s " "|cut -d " " -f1,2)
UNIX_DATE=$(date -d"$line_date" +%s)
if [ $UNIX_DATE -gt $START_UNIX_DATE -a $UNIX_DATE -lt $END_UNIX_DATE ]
then
echo "$host $line" >> $OUTPUT_UNSORTED_LOG
fi
done < $LOCAL_TEMP_FILE
else
echo -e "\e[0;33mNo file with name $logfile below $LOG_FILE_SIZE KB found under $logpath on $host\e[0m"
fi
done
done
}
I call the above function as shown below
Code:
for host in ${ARR_HOSTS[@]}
do
GetAndParse &
done
When i run this, the while loop which is reading lines one by one in log file is not terminating once it reaches EOF.
I have tried running it in foreground (removing '&' while calling function) which works perfect.
I want to run it in background because there are 20 servers i need to ssh, scp and read log files which are more than 10 MB each.
Please suggest me if there are any changes needed in this code.
I have a function with multiple while loops that fetch files from remote servers. I have paths in $paths variable and file names in $logs variable (Space separated).
Code:
Function () {
while read path from $paths
do
while read log from $logs
do
ssh -n $host du -skLc $logpath/$logfile # get the size of remote file
scp $host:$logpath/$logfile $localfile # Download remote file
while read line
do
grep $string $line > output file # Grep for string in each line
done < $localfile
done
done
}
When i call this function as below it works fine
Code:
for host in $hosts
do
Function
done
When i try to run it in background as below the inner most while loop which is reading log entries line by line does not exit after finishing the file.
Here is some back ground on the script. The script is to poll an arbitrary number of DB's. To do this I am creating a function that takes the file_path to the DB and the min poll interval as arguments. The function will be called for each DB and then ran in the background. The function I was... (6 Replies)
Hello, I was trying to make some processes to run at background and went to a problem.
First I tried just to loop in one line something like this:
for i in {1..10}; do echo 'hello world' &; done;
but it pops a syntax error, so I tried several ways to fix it but wasn't able to understand... (2 Replies)
This is the gdb backtrace.
^C
Program received signal SIGINT, Interrupt.
0xffffe424 in __kernel_vsyscall ()
(gdb) bt
#0 0xffffe424 in __kernel_vsyscall ()
#1 0xb7e56a70 in __nanosleep_nocancel () from /lib/libc.so.6
#2 0xb7e568bb in __sleep (seconds=0) at sleep.c:138
#3 0x080496d5 in... (6 Replies)
Hi,
I am trying to find files with specific name using find / -name core -print..
this command is hanging and never completes as it is searching for all the automount filesystems ..i cant eliminate using "! -fstype nfs" as this is not a nfs filesystem
pls let me know if anyone know how to... (2 Replies)
Guys,
can you help me in killing the process which is running in back ground under for loop
I am not able to find the PID using ps -afx|grep <word in command I entered> (1 Reply)
Dear all,
I'm writing a KornShell script that calls inside it a function in background mode
#!/bin/ksh
function myfunction
{ . . .}
myfunction |&
. . .
How can I capture the PID of the function myfunction that runs in background?
Thanks in advance
:) (2 Replies)
consider the given prg.
main() {
.....
function1(); /* to write into a file or log */
printf(" ");
.....
}
when the control reaches function1(), it should get executed in the
background.At the same time main's printf(" ") statement should also get executed.i.e... (5 Replies)
Hello all,
Can someone explain to me the advantage between using subshell over a function call in scripts? To me these are the same. Am I wrong to think this? (4 Replies)