Script in bash that works only some of the time


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script in bash that works only some of the time
# 1  
Old 11-22-2015
Script in bash that works only some of the time

I ran this script yesterday (in the background)
/usr/bin/nohup myfilelocation/myscriptname.sh &

the script worked perfectly.

i ran it today (also in the background) and just sat there. So i killed it and ran it normally and it worked perfectly.
Anyone suggest why it just sat there and didnt work? but worked when i ran it again right after it worked fine?

partial script below

Code:
#!/bin/bash
TZ=date+24
DATE1=`date '+%d%m%y'`
DATE=`date '+%Y%m%d'`

#this bit waits for the file to appear
mytest1=`ls -lart /pathtofile/ | grep -c filename.$DATE.TXT`
while [ $mytest1 -ne 1 ]
do
mytest1=`ls -lart /pathtofile/ | grep -c filename.$DATE.TXT`
while [ $mytest1 -ne 1 ]
sleep 160
done

#this bit waits for the file to finish generating
while [ $mytest15 -ne 1 ]
do
mytest15=`cat /pathtofile/ | grep -c filename.$DATE.TXT | grep TRL | wc -l`
sleep 180
done

goes on to change file name and move it
# 2  
Old 11-22-2015
Quote:
Originally Posted by twinion
Code:
mytest15=`cat /pathtofile/ | grep -c filename.$DATE.TXT | grep TRL | wc -l`

I don't see why you're performing cat on a directory. Is this a typo? If not, you will not get your intended results. Also, you can grep directly on the file instead of piping the cat output to grep. Read this "useless use of cat".
# 3  
Old 11-22-2015
Other then mentioned, use #!/bin/bash -x and examine the nohup.out during the hang.

If you are on linux system, inotify should provide much better interface for actions on file changes.

Please, if you can, post the entire code and/or explain what are you trying to do so perhaps a better approach could be written.

Hope that helps
Regards
Peasant.
This User Gave Thanks to Peasant For This Post:
# 4  
Old 11-22-2015
Quote:
Originally Posted by balajesuri
I don't see why you're performing cat on a directory. Is this a typo? If not, you will not get your intended results. Also, you can grep directly on the file instead of piping the cat output to grep. Read this "useless use of cat".
sorry that is a typo when written into the forum

mytest15=`cat /pathtofile/filename | grep -c filename.$DATE.TXT | grep TRL | wc -l`

basically i know the file has completed writing when it has the word TRL in the file.

---------- Post updated at 02:10 AM ---------- Previous update was at 02:04 AM ----------

Quote:
Originally Posted by Peasant
Other then mentioned, use #!/bin/bash -x and examine the nohup.out during the hang.

If you are on linux system, inotify should provide much better interface for actions on file changes.

Please, if you can, post the entire code and/or explain what are you trying to do so perhaps a better approach could be written.

Hope that helps
Regards
Peasant.
-x i will give that a try.

im positive it could be written a hell of alot better as im still very new.

what the script does is waits for a file to be written to a directory.
when that file arrives, it then waits for it to finish being built by looking for TRL inside the file.
Then it renames the file and moves it.

as i said ran ti yesterday worked fine.
ran it today in the background it didnt see that the file arrived.
so i killed it then ran it
ran it normally...it spotted the file and worked as expected.
i just dont understand why it was in the background it didnt notice that the file was there, and yesterday it worked fine
# 5  
Old 11-22-2015
And even if grep of a directory made sense, the commands at the end of the pipeline:
Code:
grep -c filename.$DATE.TXT | grep TRL | wc -l

will always return 0; never 1. The grep -c output will always be a numeric string (that cannot contain the string TRL) and wc -l reading from an empty pipe will always return 0.

Therefore, this loop in your code:
Code:
#this bit waits for the file to finish generating
while [ $mytest15 -ne 1 ]
do
mytest15=`cat /pathtofile/ | grep -c filename.$DATE.TXT | grep TRL | wc -l`
sleep 180
done

cannot ever complete. It is an infinite loop unless the while loop is failing with a syntax error because $mytest expands to an empty string the first time through the loop yielding a syntax error on:
Code:
[ -ne 1 ]

but, since you said it is working successfully when you run it in the foreground, there obviously were not any diagnostic messages printed. So, we have to assume that part of the code you chose not to show us predefined mytest to have the value 1 before the above loop was executed.

If you want to know whether or not a file contains the string TRL, you would not do that with:
Code:
mytest15=`cat /pathtofile/filename | grep -c filename.$DATE.TXT | grep TRL | wc -l`

either. You might want to try a simpler loop like:
Code:
#this bit waits for the file to finish generating
while ! grep -q TRL /pathtofile/filename
do	sleep 180
done

This User Gave Thanks to Don Cragun For This Post:
# 6  
Old 11-22-2015
Are you sure it's just sitting there? Are you sure it "worked perfectly"? There's at least a do ... done missing for the second while.

Different approach for waiting "for a file to appear":
Code:
while [ ! -f filename.$DATE.TXT ]; do sleep 120; done

And, if one single occurrence of the string "TRL" indicates the file is complete, try
Code:
while ! grep -q TRL filename.$DATE.TXT; do sleep 120; done

This User Gave Thanks to RudiC For This Post:
# 7  
Old 11-22-2015
Many thanks, some brilliant suggestion guys. I'm going to get some sleep then make some improvements and test it again tonight. I'm self taught on scripting but I believe I maybe addicted.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Command works interactively but not in bash script

The below command works in the terminal interactively but not as part of a bash script. I though maybe I needed to escape the "$dir" so it isn't interpreted literally, but that's not it. Thank you :). interactively in terminal dir=/path/to new=$(ls "$dir"/*.csv -tr | tail -n 1) && echo... (6 Replies)
Discussion started by: cmccabe
6 Replies

2. Shell Programming and Scripting

Calling bash script works when called manually but not via Cron?

Hi, I've got a Bash backup script I'm trying to run on a directory via a cron job nightly. If I ssh in and run the script manually it works flawlessly. If I set up the cron to run evertything is totally messed up I don't even know where to begin. Basically the path structure is ... (6 Replies)
Discussion started by: wyclef
6 Replies

3. Shell Programming and Scripting

Bash Script Looping all the time

Hello, I have a database file, named data.txt, and a shell script (convert.sh) to convert data.txt from columns to row. Output file name will be column_to_row.txt In this example data.txt has only four rows. Format of data.txt is: info name surname telefon_nr Data.txt info boris... (1 Reply)
Discussion started by: baris35
1 Replies

4. UNIX for Dummies Questions & Answers

Bash script dont works when executed as cronjob

Hello, i have cronjob: crontab -l * * * * * pkill -f domexpcheck;sh /root/dom/domexpcheck.sh it runs: /var/log/cron Mar 25 12:11:01 vps crond: (root) CMD (pkill -f domexpcheck;sh /root/dom/domexpcheck.sh) but somehow script dont run properly via cronjob. But when i execute cronjob... (7 Replies)
Discussion started by: postcd
7 Replies

5. Shell Programming and Scripting

Need bash script to ping the servers and rename the output file each time the script is ran

HI, I have a file serverlist in that all host names are placed. i have written a small script #./testping #! /bin/bash for i in `cat serverlist` do ping $i >> output.txt done so now it creates a file output.txt till here fine.. now each time i run this script the output file... (4 Replies)
Discussion started by: madhudeva
4 Replies

6. Shell Programming and Scripting

Ftp script hangs for first time,but works every second time

Hi I have an ftp script which works fine when i execute through a test scheduler(UC4), but when i run it through the prod scheduler(UC4), it hungs indefinetely, when we cancel the job and re-run it it works perfectly fine. here is the code,, any idea why this is happening ???? ... (1 Reply)
Discussion started by: selvankj
1 Replies

7. Shell Programming and Scripting

How to compare time in bash script?

Hi, Anyone know how to compare the time in bash script? I want to compare say 30 min. to 45 min. ( AIX ) Thanks. (1 Reply)
Discussion started by: sumit30
1 Replies

8. Shell Programming and Scripting

bash script to count the time of transaction

Halo, Bash Script can get the time of process the trasaction or not? For example, bash script use to procee the trasaction, like select and checking.. then generate the XML. after it, i need to get the time which to count the process. Anyone can help me? Thank you (1 Reply)
Discussion started by: ryanW
1 Replies

9. Shell Programming and Scripting

Script works with bash 3.0 but not 3.2.

Hello, So my knowledge of bash scripting is not that great and I have been trying to solve this problem on my own for awhile to no avail. Here's the error I get when running it with an OS that uses bash 3.2.x: testagain.sh: line 10: *-1: syntax error: operand expected (error token is... (2 Replies)
Discussion started by: forkandspoon
2 Replies

10. Shell Programming and Scripting

Works from bash prompt, but not from script!

I'm trying to use unison from bash on windows with cygwin. I don't know if this is a cygwin question, bash question or unison question. Since I always get reprimanded by the cygwin mailing list for assuming it is a cygwin problem, I'll assume it is a bash question. The following commands work... (7 Replies)
Discussion started by: siegfried
7 Replies
Login or Register to Ask a Question