Home Man
Search
Today's Posts
Register

BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Nohup apending data each time i run script

Tags
shell script, shell scripts, solved

Login to Reply

 
Thread Tools Search this Thread
# 8  
Old 08-06-2016
You are quite tight when it comes to answering questions intended to help, aren't you? Except for OS and shell, none of the questions asked in various posts have been answered. How do you expect anybody to give you a meaningful hint?

Last edited by RudiC; 08-06-2016 at 05:50 AM..
# 9  
Old 08-06-2016
Quote:
Originally Posted by mirwasim
my script is of 230 lines including email.
shall i post whole script here?
If you would like for us to look at your script and try to help you find out what is going wrong, yes! Or, you can choose not to post your script and we will have to assume that you don't want our help.
Quote:
and in side my script i am calling other .jmx script also and my script send result on the basis of out put from that script.

which is fine and run as expected which i run
Code:
sh -x script.sh

but when i schedule in cron it mess it up. it post old data also with time and data.
it is keeping old data some where is buffer.
Again, if you want us to look at your code and try to help you find out what is going wrong, we need to see your code. If you don't want to show us your code, we will have to assume that you don't want our help.
Quote:
i am using SUNOS

output in email, can i attach the same here?
i am using bash
We would prefer that you copy and paste the output here in code tags, but if the output is big, you can attach it. (Just be aware that fewer people will try to help if they have to download big files to do so.)
The Following User Says Thank You to Don Cragun For This Useful Post:
mirwasim (08-06-2016)
# 10  
Old 08-06-2016
Dear Don,

Thanks for your time and i am sorry if i am not able to provide adequate information sometime.
below is my script, input file which is getting generated within a script and out file which is messed up, taking date and time twice also is taking output.
i dont know where it is saving the old information
Main script which i am running in nohup.
each time i am running it is appending information.
Code:
DATE_1=`date | awk '{print $2,$3,$4}'`
SCRIPT_NAME_1="ACCOUNT INQUIRY FOR MOBILE"

##i remove all the files in the begning of the script before it run

cd /location
> acc_inq_mob.csv

cd path


sh /path/jmeter -n -t path/jmeter.jmx -l log.jtl ## this script i am calling here and all the log which it create i am doing further scripting on that

nawk -F, '/false/ && match ($0, /http:\/\/[^\/]*\/[^\/]*/) {print $3, substr ($0, RSTART, RLENGTH)}' log.jtl > ERROR_FILE.log

nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv   > path/acc_inq_mob.log

nawk '{gsub (/<[^>]*>/,_);sub (/\/[^\/]*$/, _, $1); print $1}' FS="[?;]" acc_inq_mob.log > acc_inq_output1.log
cut -d',' -f3 acc_inq_output1.log > acc_inq_output.log


DATE_END_1=`date | awk '{print $2,$3,$4}'`

echo "<html>" >> date.html
echo "<Body>" >> date.html
echo "<table border="1">" >> date.html
echo "<tr>" >> date.html
echo "<td colspan="0" bgcolor="#7FFFD4">" >> date.html
echo "START_DATE : " $DATE_1  >> date.html ## start time details
echo "</td>" >> date.html
echo "</tr>" >> date.html
echo "</table>" >> date.html
echo "</Body>" >> date.html
echo "</html>" >> date.html
echo "<br />"  >> date.html

echo "<html>" >> date_end.html
echo "<Body>" >> date_end.html
echo "<table border = "1">" >> date_end.html
echo "<tr>" >> date_end.html
echo "<td colspan="0" bgcolor="#7FFFD4">"  >> date_end.html
echo "END_DATE:      " $DATE_END_1  >> date_end.html ## END time details
echo "</td>" >> date_end.html
echo "</tr>" >> date_end.html
echo "</table>" >> date_end.html
echo "</Body>" >> date_end.html
echo "</html>" >> date_end.html
echo "<br />"  >> date_end.html
echo "<br />"  >> date_end.html

echo "<br />"  >> SUCCESS.html
echo "<html>" >> SUCCESS.html
echo "<Body>" >> SUCCESS.html
echo "<table border = "1">" >> SUCCESS.html
echo "<tr>" >> SUCCESS.html
echo "<td colspan="0" bgcolor="#008000">"  >> SUCCESS.html
echo "<b>" >> SUCCESS.html
echo "<font color = "#FFFFFF">" >> SUCCESS.html
echo "SUCCESS RESULT" >> SUCCESS.html ## PRINTS SUCCESS TAG
echo "</font>" >> SUCCESS.html
echo "</b>" >> SUCCESS.html
echo "</td>" >> SUCCESS.html
echo "</tr>" >> SUCCESS.html
echo "</table>" >> SUCCESS.html
echo "</Body>" >> SUCCESS.html
echo "</html>" >> SUCCESS.html
echo "<br />"  >> SUCCESS.html

echo "<br />"  >> ERROR.html
echo "<html>" >> ERROR.html
echo "<Body>" >> ERROR.html
echo "<table border = "1">" >> ERROR.html
echo "<tr>" >> ERROR.html
echo "<td colspan="0" bgcolor="#FF0000">" >> ERROR.html
echo "<b>" >> ERROR.html
echo "<font color = "#FFFFFF">" >> ERROR.html
echo "ERROR RESULT" >> ERROR.html ## PRINTS ERROR TAG
echo "</font>" >> ERROR.html
echo "</b>" >> ERROR.html
echo "</td>" >> ERROR.html
echo "</tr>" >> ERROR.html
echo "</table>" >> ERROR.html
echo "</Body>" >> ERROR.html 
echo "</html>" >> ERROR.html
echo "<br />"  >> ERROR.html


echo "<br />"  >> output.html
echo "<html>" >> output.html
echo "<Body>" >> output.html
nawk 'BEGIN{print "<table border="1">"} 
          {print "<tr>";
                  for(i=1;i<=NF;i++)print "<td>" $i"</td>";
                  print "</tr>"} END{print "</table>"}' acc_inq_output.log >> output.html ##URLs with ports which are euccessfully tested
echo "</Body>" >> output.html
echo "</html>" >> output.html
echo "<br />"  >> output.html


echo "<html>" >> ERROR_FILE.html
echo "<Body>" >> ERROR_FILE.html
nawk 'BEGIN{print "<table border="1">"}
       {print "<tr>";
       print "<TD>";
       for(i=1;i<NF;i++) printf("%s ",  $i);
       print "</TD>";
       print "<TD>" $NF "</TD>";
       print "</tr>"}
     END{print "</table>"}'  ERROR_FILE.log >> ERROR_FILE.html

echo "</Body>" >> ERROR_FILE.html
echo "</html>" >> ERROR_FILE.html

COUNT_ERROR_FILE=`wc -l ERROR_FILE.log  |sed 's/\|/ /'|awk '{print $1}'`

FROM="server"
SUBJECT="SANITY TEST RESULT FOR $SCRIPT_NAME_1"
ATTACH1="/path/ERROR_FILE.html"
ATTACH2="/path/date.html"
ATTACH3="/path/date_end.html"
ATTACH4="/path/SUCCESS.html"
ATTACH5="/path/ERROR.html"
ATTACH6="/path/output.html"
ATTACH_NAME1=${ATTACH1##*/}
MAILTO="email@gmail.com"
##CC="email@gmail.com"


if [  $COUNT_ERROR_FILE -eq 0 ]
then
(
 echo "From: $FROM"
 echo "To: $MAILTO"
 echo "CC: $CC"
 echo "Subject: $SUBJECT"
 echo "MIME-Version: 1.0"
 echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
 echo '---q1w2e3r4t5'
 echo "Content-Type: text/html"
 echo "Content-Disposition: inline"
 cat "$ATTACH2" 
 echo
 cat "$ATTACH3"
 echo
 cat "$ATTACH4"
 echo
  cat "$ATTACH6"
 echo
) | /usr/sbin/sendmail $MAILTO $CC
exit
else
(
 echo "From: $FROM"
 echo "Cc: $CC"
 echo "To: $MAILTO"
 echo "Subject: $SUBJECT"
 echo "MIME-Version: 1.0"
 echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
 echo
 echo '---q1w2e3r4t5'
 echo "Content-Type: text/html"
 echo "Content-Disposition: inline"
 echo
 cat "$ATTACH2" 
 echo
 cat "$ATTACH3"
 echo
 echo 
 cat "$ATTACH5"
 echo 
 cat "$ATTACH1"
 echo
 cat "$ATTACH4"
 echo
 cat "$ATTACH6"
 echo
 echo '---q1w2e3r4t5'
 echo 'Content-Type: application; name="'${ATTACH_NAME1}'"'
 echo "Content-Transfer-Encoding: uuencode"
 echo 'Content-Disposition: attachment; filename="'${ATTACH_NAME1}'"'
 echo
 uuencode $ATTACH1 $ATTACH1
 echo
) | /usr/sbin/sendmail $MAILTO $CC
fi

output it is giving statrt time . it have created more that 2 also

Code:
<html>
<Body>
<table border=1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
START_DATE :  Aug 6 22:34:22
</td>
</tr>
</table>
</Body>
</html>
<br />
<html>
<Body>
<table border=1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
START_DATE :  Aug 6 22:35:12
</td>
</tr>
</table>
</Body>
</html>
<br />

output end time
Code:
<html>
<Body>
<table border = 1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
END_DATE:       Aug 6 22:34:35
</td>
</tr>
</table>
</Body>
</html>
<br />
<br />
<html>
<Body>
<table border = 1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
END_DATE:       Aug 6 22:35:23
</td>
</tr>
</table>
</Body>
</html>
<br />
<br />

it is also giving data output repeatedly

my input file
Code:
cc_inq_mob.csv

Code:
1,1,http://abc:123/defd/FixedAccInquiryAction.do?accountId=123456&fromCMS=null&operation=3&searchPage=SOHACCOUNTIDINQ
1,1,http://abc:345/defh/FixedAccInquiryAction.do?accountId=123456&fromCMS=null&operation=3&searchPage=SOHACCOUNTIDINQ

please advise
sorry for the trouble
Nohup apending data each time i run script-capture-jpg  


Last edited by mirwasim; 08-06-2016 at 03:10 PM.. Reason: insert image
# 11  
Old 08-07-2016
You said you're using bash as your shell, but /bin/sh is an old Bourne shell; not a bash shell on SunOS systems. The following suggestion uses /bin/ksh instead of /bin/sh so we can use the more modern form of command substitution and various variable expansions that are not available in a traditional Bourne shell. (This allows us to avoid invoking several external utilities by using shell variable expansions.) The code used in the following suggestions will work with both bash and ksh, but you'll probably find that ksh on SunOS system is a little faster than bash.

You have said that you're using nohup and cron to run your script sometimes and I asked what the crontab entries look like when you're using them. You still have not supplied any indication of how your script is invoked when run by cron so I can't make any guesses as to whether or not my following suggestions are sufficient to work in the way you run your script from cron. If you are using nohup and any output is being saved in nohup.out, you will need to do something to ensure that any scripts invoked using nohup place any output those scripts produce in unique files (rather than having each of those invocations destroy output produced by other invocations or having output from all of those invocations intermingled in a single output file).

You still have not shown us the contents of the script /path/jmeter, so updating that script to produce files that do not destroy files it produces when multiple versions of it are run concurrently are left as an exercise for the submitter. (AND, if this script is run multiple times concurrently, even with the changes shown in the suggestion below; it will not work because all invocations of /path/jmeter apparently produce output in the single file /path/acc_inq_mob.csv, or /location/path/acc_inq_mob.csv, or /location/path/path/acc_inq_mob.csv (all three of which are created, read, or both) by various commands in your script, but I assume that only one of them is correct.

You have a comment in your code:
Code:
##i remove all the files in the beginning of the script before it run

but that is just a comment and you have lots of files in your script that are appended to (thereby creating the history you talked about). And if you are running multiple copies of this script simultaneously, each invocation of your script is writing to the same file while you need to have each invocation write to a different set of files. The following suggestion assumes that all of the files for which you have defined variables to reference their pathnames that they are temporary files that need to be created as distinct for each invocation of your script and should be removed when your script exits.

In addition to that there is no need to run date and then extract three fields from the output it produces by also running nawk, just invoking date with a format operand is a lot more efficient.

And, instead of using echo to write one line at a time into your temporary files, it would be much more efficient to execute a single command to write the entire contents which reduces not only the number of commands your script executes but also greatly reduces the number of times your script opens and closes your various temporary files.

Why do you define various variables naming your temporary files halfway through your script (after using the full text dozens of times before you define the variables and once or twice after defining them)?

You have lots of references to the directory path:
Code:
ATTACH1="/path/ERROR_FILE.html"
ATTACH2="/path/date.html"
ATTACH3="/path/date_end.html"
ATTACH4="/path/SUCCESS.html"
ATTACH5="/path/ERROR.html"
ATTACH6="/path/output.html"
cd path
sh /path/jmeter -n -t path/jmeter.jmx -l log.jtl ## this script i am calling her
e and all the log which it create i am doing further scripting on that
nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv   > path/acc_inq_mob.log

some of which are absolute referencing the directory /path and the rest of which are relative expanding to:
Code:
cd /location/path
sh /path/jmeter -n -t /location/path/jmeter.jmx -l log.jtl ## this script i am calling her
e and all the log which it create i am doing further scripting on that
nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv   > /location/path/acc_inq_mob.log

I would guess that many, if not all, of the relative pathnames above are incorrect, but the following code sample uses the pathnames used in your script.

Other than some of the inconsistencies mentioned above, the following suggestion might work if you make similar changes in /path/jmeter and clean up the references to the path directory that are not given as absolute pathnames, and you either invoke it with ksh instead of sh or (since it now has a #!/bin/ksh at the start of the first line in the script) you make your script executable and invoke it by its name instead of explicitly using a shell to invoke it:
Code:
#!/bin/ksh

# Remove temporary file when the script exits.
trap 'rm -f "$ATTACH1" "$ATTACH2" "$ATTACH3" "$ATTACH4" "$ATTACH5" "$ATTACH6" "$ERROR_LOG"' EXIT

# Define variables.
SUBJECT="SANITY TEST RESULT FOR $SCRIPT_NAME_1"
ATTACH1="/path/ERROR_FILE.html.$$"
ATTACH2="/path/date.html.$$"
ATTACH3="/path/date_end.html.$$"
ATTACH4="/path/SUCCESS.html.$$"
ATTACH5="/path/ERROR.html.$$"
ATTACH6="/path/output.html.$$"
ATTACH_NAME1=${ATTACH1##*/}
##CC="email@gmail.com"
ERROR_LOG="ERROR_FILE.log.$$"
FROM="server"
MAILTO="email@gmail.com"
SCRIPT_NAME_1="ACCOUNT INQUIRY FOR MOBILE"

# Get start date and time.
DATE_1=$(date '+%b %e %T')

cd /location
> acc_inq_mob.csv.$$	# Should this line be moved after the "cd path" below?

cd path		# Should this be "/path"?


ksh /path/jmeter -n -t path/jmeter.jmx -l log.jtl ## this script i am calling here and all the log which it create i am doing further scripting on that
# Should above line be:
# ksh /path/jmeter -n -t /path/jmeter.jmx -l log.jtl
#    or
# ksh jmeter -n -t jmeter.jmx -l log.jtl


nawk -F, '/false/ && match ($0, /http:\/\/[^\/]*\/[^\/]*/) {print $3, substr ($0, RSTART, RLENGTH)}' log.jtl > "$ERROR_LOG"

# Note that the temp files produced by the next three commands are not removed
# when this script completes, the three following commands could be done by a
# single nawk command (but without a sample of the expected input and output I
# am NOT going to attempt to do that.
nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv.$$   > path/acc_inq_mob.log.$$

nawk '{gsub (/<[^>]*>/,_);sub (/\/[^\/]*$/, _, $1); print $1}' FS="[?;]" acc_inq_mob.log.$$ > acc_inq_output1.log.$$
cut -d',' -f3 acc_inq_output1.log.$$ > acc_inq_output.log.$$

# Get end date and time.
DATE_END_1=$(date '+%b %e %T')

printf '%s\n' "<html>
<Body>
<table border=\"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#7FFFD4\">
START_DATE :  $DATE_1
</td>
</tr>
</table>
</Body>
</html>
<br />"  > "$ATTACH2"

printf '%s\n' "<html>
<Body>
<table border = \"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#7FFFD4\">
END_DATE:       $DATE_END_1
</td>
</tr>
</table>
</Body>
</html>
<br />
<br />" > "$ATTACH3"

printf '%s\n' "<br />
<html>
<Body>
<table border = \"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#008000\">
<b>
<font color = \"#FFFFFF\">
SUCCESS RESULT
</font>
</b>
</td>
</tr>
</table>
</Body>
</html>
<br />" > "$ATTACH4"

printf '%s\n' "<br />
<html>
<Body>
<table border = \"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#FF0000\">
<b>
<font color = \"#FFFFFF\">
ERROR RESULT
</font>
</b>
</td>
</tr>
</table>
</Body>
</html>
<br />" > "$ATTACH5"

printf '%s\n' "<br />
<html>
<Body>
$(nawk 'BEGIN{print "<table border=\"1\">"} 
          {print "<tr>";
                  for(i=1;i<=NF;i++)print "<td>" $i "</td>";
                  print "</tr>"} END{print "</table>"}' acc_inq_output.log.$$)
</Body>
</html>
<br />" > "$ATTACH6"

printf '%s\n' "<html>
<Body>
$(nawk 'BEGIN{print "<table border=\"1\">"}
       {print "<tr>";
       print "<TD>";
       for(i=1;i<NF;i++) printf("%s ",  $i);
       print "</TD>";
       print "<TD>" $NF "</TD>";
       print "</tr>"}
     END{print "</table>"}' "$ERROR_LOG")
</Body>
</html>" > "$ATTACH1"

COUNT_ERROR_FILE=$(wc -l < "$ERROR_LOG")
OUNT_ERROR_FILE=${COUNT_ERROR_FILE##* }

if [  $COUNT_ERROR_FILE -eq 0 ]
then (	echo "From: $FROM"
	echo "To: $MAILTO"
	echo "CC: $CC"
	echo "Subject: $SUBJECT"
	echo "MIME-Version: 1.0"
	echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
	echo '---q1w2e3r4t5'
	echo "Content-Type: text/html"
	echo "Content-Disposition: inline"
	cat "$ATTACH2" 
	echo
	cat "$ATTACH3"
	echo
	cat "$ATTACH4"
	echo
	cat "$ATTACH6"
	echo
     )
else (
	echo "From: $FROM"
	echo "Cc: $CC"
	echo "To: $MAILTO"
	echo "Subject: $SUBJECT"
	echo "MIME-Version: 1.0"
	echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
	echo
	echo '---q1w2e3r4t5'
	echo "Content-Type: text/html"
	echo "Content-Disposition: inline"
	echo
	cat "$ATTACH2" 
	echo
	cat "$ATTACH3"
	echo
	echo 
	cat "$ATTACH5"
	echo 
	cat "$ATTACH1"
	echo
	cat "$ATTACH4"
	echo
	cat "$ATTACH6"
	echo
	echo '---q1w2e3r4t5'
	echo 'Content-Type: application; name="'${ATTACH_NAME1}'"'
	echo "Content-Transfer-Encoding: uuencode"
	echo 'Content-Disposition: attachment; filename="'${ATTACH_NAME1}'"'
	echo
	uuencode "$ATTACH1" "$ATTACH1"
	echo
     )
fi | /usr/sbin/sendmail $MAILTO $CC

Note, of course, that with no sample input, no example of the real desired output, no indication of what /path/jmeter does, and no belief that the references to path and /path are correct in the given script; none of this has been tested.

Last edited by Don Cragun; 08-07-2016 at 03:43 AM.. Reason: Add note.
The Following User Says Thank You to Don Cragun For This Useful Post:
mirwasim (08-08-2016)
# 12  
Old 08-08-2016
Dear D,

Thanks for your support and time.
from all this i learn to make my script small and i learn
Code:
trap

command

my script issue is fixed now. issue was with the declared variables.

i have fixed it now
Login to Reply

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Find gaps in time data and replace missing time value and column 2 value by interpolation in awk malandisa Programming 4 01-16-2015 10:11 AM
Resample time series data with replacement any way to do this in awk or just bash script malandisa Programming 3 03-21-2014 11:21 AM
Saving nohup output to a file other than nohup.out kraljic Shell Programming and Scripting 7 12-08-2013 04:33 PM
Nohup in conjunction with time BearCheese UNIX for Dummies Questions & Answers 1 05-01-2013 10:43 AM
Help needed for nohup time saswati16 UNIX for Dummies Questions & Answers 2 03-16-2011 08:35 AM
nohup output on screen for some time axes Shell Programming and Scripting 3 01-11-2011 08:38 AM
Apending a list of users to a file Jimmy_c UNIX for Dummies Questions & Answers 1 10-22-2010 12:10 PM
Run process with nohup every certain time pepeli30 UNIX for Dummies Questions & Answers 6 12-22-2009 06:40 AM
shell script data & time validation tal Shell Programming and Scripting 4 05-18-2009 07:53 AM
Iterate a min/max awk script over time-series temperature data jgourley UNIX for Dummies Questions & Answers 5 01-09-2009 02:41 PM


All times are GMT -4. The time now is 02:57 AM.

Unix & Linux Forums Content Copyright©1993-2018. All Rights Reserved.
UNIX.COM Login
Username:
Password:  
Show Password