Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Nohup apending data each time i run script

Shell Programming and Scripting


Tags
shell script, solved

Closed    
 
Thread Tools Search this Thread Display Modes
    #8  
Old Unix and Linux 08-06-2016   -   Original Discussion by mirwasim
RudiC's Unix or Linux Image
RudiC RudiC is offline Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 21 April 2018, 1:54 PM EDT
Location: Aachen, Germany
Posts: 12,504
Thanks: 401
Thanked 3,877 Times in 3,564 Posts
You are quite tight when it comes to answering questions intended to help, aren't you? Except for OS and shell, none of the questions asked in various posts have been answered. How do you expect anybody to give you a meaningful hint?

Last edited by RudiC; 08-06-2016 at 05:50 AM..
Sponsored Links
    #9  
Old Unix and Linux 08-06-2016   -   Original Discussion by mirwasim
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 21 April 2018, 6:35 PM EDT
Location: San Jose, CA, USA
Posts: 11,208
Thanks: 633
Thanked 3,903 Times in 3,341 Posts
Quote:
Originally Posted by mirwasim View Post
my script is of 230 lines including email.
shall i post whole script here?
If you would like for us to look at your script and try to help you find out what is going wrong, yes! Or, you can choose not to post your script and we will have to assume that you don't want our help.
Quote:
and in side my script i am calling other .jmx script also and my script send result on the basis of out put from that script.

which is fine and run as expected which i run

Code:
sh -x script.sh

but when i schedule in cron it mess it up. it post old data also with time and data.
it is keeping old data some where is buffer.
Again, if you want us to look at your code and try to help you find out what is going wrong, we need to see your code. If you don't want to show us your code, we will have to assume that you don't want our help.
Quote:
i am using SUNOS

output in email, can i attach the same here?
i am using bash
We would prefer that you copy and paste the output here in code tags, but if the output is big, you can attach it. (Just be aware that fewer people will try to help if they have to download big files to do so.)
The Following User Says Thank You to Don Cragun For This Useful Post:
mirwasim (08-06-2016)
Sponsored Links
    #10  
Old Unix and Linux 08-06-2016   -   Original Discussion by mirwasim
mirwasim's Unix or Linux Image
mirwasim mirwasim is offline
Registered User
 
Join Date: Dec 2012
Last Activity: 23 February 2018, 8:07 AM EST
Posts: 175
Thanks: 34
Thanked 1 Time in 1 Post
Dear Don,

Thanks for your time and i am sorry if i am not able to provide adequate information sometime.
below is my script, input file which is getting generated within a script and out file which is messed up, taking date and time twice also is taking output.
i dont know where it is saving the old information
Main script which i am running in nohup.
each time i am running it is appending information.


Code:
DATE_1=`date | awk '{print $2,$3,$4}'`
SCRIPT_NAME_1="ACCOUNT INQUIRY FOR MOBILE"

##i remove all the files in the begning of the script before it run

cd /location
> acc_inq_mob.csv

cd path


sh /path/jmeter -n -t path/jmeter.jmx -l log.jtl ## this script i am calling here and all the log which it create i am doing further scripting on that

nawk -F, '/false/ && match ($0, /http:\/\/[^\/]*\/[^\/]*/) {print $3, substr ($0, RSTART, RLENGTH)}' log.jtl > ERROR_FILE.log

nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv   > path/acc_inq_mob.log

nawk '{gsub (/<[^>]*>/,_);sub (/\/[^\/]*$/, _, $1); print $1}' FS="[?;]" acc_inq_mob.log > acc_inq_output1.log
cut -d',' -f3 acc_inq_output1.log > acc_inq_output.log


DATE_END_1=`date | awk '{print $2,$3,$4}'`

echo "<html>" >> date.html
echo "<Body>" >> date.html
echo "<table border="1">" >> date.html
echo "<tr>" >> date.html
echo "<td colspan="0" bgcolor="#7FFFD4">" >> date.html
echo "START_DATE : " $DATE_1  >> date.html ## start time details
echo "</td>" >> date.html
echo "</tr>" >> date.html
echo "</table>" >> date.html
echo "</Body>" >> date.html
echo "</html>" >> date.html
echo "<br />"  >> date.html

echo "<html>" >> date_end.html
echo "<Body>" >> date_end.html
echo "<table border = "1">" >> date_end.html
echo "<tr>" >> date_end.html
echo "<td colspan="0" bgcolor="#7FFFD4">"  >> date_end.html
echo "END_DATE:      " $DATE_END_1  >> date_end.html ## END time details
echo "</td>" >> date_end.html
echo "</tr>" >> date_end.html
echo "</table>" >> date_end.html
echo "</Body>" >> date_end.html
echo "</html>" >> date_end.html
echo "<br />"  >> date_end.html
echo "<br />"  >> date_end.html

echo "<br />"  >> SUCCESS.html
echo "<html>" >> SUCCESS.html
echo "<Body>" >> SUCCESS.html
echo "<table border = "1">" >> SUCCESS.html
echo "<tr>" >> SUCCESS.html
echo "<td colspan="0" bgcolor="#008000">"  >> SUCCESS.html
echo "<b>" >> SUCCESS.html
echo "<font color = "#FFFFFF">" >> SUCCESS.html
echo "SUCCESS RESULT" >> SUCCESS.html ## PRINTS SUCCESS TAG
echo "</font>" >> SUCCESS.html
echo "</b>" >> SUCCESS.html
echo "</td>" >> SUCCESS.html
echo "</tr>" >> SUCCESS.html
echo "</table>" >> SUCCESS.html
echo "</Body>" >> SUCCESS.html
echo "</html>" >> SUCCESS.html
echo "<br />"  >> SUCCESS.html

echo "<br />"  >> ERROR.html
echo "<html>" >> ERROR.html
echo "<Body>" >> ERROR.html
echo "<table border = "1">" >> ERROR.html
echo "<tr>" >> ERROR.html
echo "<td colspan="0" bgcolor="#FF0000">" >> ERROR.html
echo "<b>" >> ERROR.html
echo "<font color = "#FFFFFF">" >> ERROR.html
echo "ERROR RESULT" >> ERROR.html ## PRINTS ERROR TAG
echo "</font>" >> ERROR.html
echo "</b>" >> ERROR.html
echo "</td>" >> ERROR.html
echo "</tr>" >> ERROR.html
echo "</table>" >> ERROR.html
echo "</Body>" >> ERROR.html 
echo "</html>" >> ERROR.html
echo "<br />"  >> ERROR.html


echo "<br />"  >> output.html
echo "<html>" >> output.html
echo "<Body>" >> output.html
nawk 'BEGIN{print "<table border="1">"} 
          {print "<tr>";
                  for(i=1;i<=NF;i++)print "<td>" $i"</td>";
                  print "</tr>"} END{print "</table>"}' acc_inq_output.log >> output.html ##URLs with ports which are euccessfully tested
echo "</Body>" >> output.html
echo "</html>" >> output.html
echo "<br />"  >> output.html


echo "<html>" >> ERROR_FILE.html
echo "<Body>" >> ERROR_FILE.html
nawk 'BEGIN{print "<table border="1">"}
       {print "<tr>";
       print "<TD>";
       for(i=1;i<NF;i++) printf("%s ",  $i);
       print "</TD>";
       print "<TD>" $NF "</TD>";
       print "</tr>"}
     END{print "</table>"}'  ERROR_FILE.log >> ERROR_FILE.html

echo "</Body>" >> ERROR_FILE.html
echo "</html>" >> ERROR_FILE.html

COUNT_ERROR_FILE=`wc -l ERROR_FILE.log  |sed 's/\|/ /'|awk '{print $1}'`

FROM="server"
SUBJECT="SANITY TEST RESULT FOR $SCRIPT_NAME_1"
ATTACH1="/path/ERROR_FILE.html"
ATTACH2="/path/date.html"
ATTACH3="/path/date_end.html"
ATTACH4="/path/SUCCESS.html"
ATTACH5="/path/ERROR.html"
ATTACH6="/path/output.html"
ATTACH_NAME1=${ATTACH1##*/}
MAILTO="email@gmail.com"
##CC="email@gmail.com"


if [  $COUNT_ERROR_FILE -eq 0 ]
then
(
 echo "From: $FROM"
 echo "To: $MAILTO"
 echo "CC: $CC"
 echo "Subject: $SUBJECT"
 echo "MIME-Version: 1.0"
 echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
 echo '---q1w2e3r4t5'
 echo "Content-Type: text/html"
 echo "Content-Disposition: inline"
 cat "$ATTACH2" 
 echo
 cat "$ATTACH3"
 echo
 cat "$ATTACH4"
 echo
  cat "$ATTACH6"
 echo
) | /usr/sbin/sendmail $MAILTO $CC
exit
else
(
 echo "From: $FROM"
 echo "Cc: $CC"
 echo "To: $MAILTO"
 echo "Subject: $SUBJECT"
 echo "MIME-Version: 1.0"
 echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
 echo
 echo '---q1w2e3r4t5'
 echo "Content-Type: text/html"
 echo "Content-Disposition: inline"
 echo
 cat "$ATTACH2" 
 echo
 cat "$ATTACH3"
 echo
 echo 
 cat "$ATTACH5"
 echo 
 cat "$ATTACH1"
 echo
 cat "$ATTACH4"
 echo
 cat "$ATTACH6"
 echo
 echo '---q1w2e3r4t5'
 echo 'Content-Type: application; name="'${ATTACH_NAME1}'"'
 echo "Content-Transfer-Encoding: uuencode"
 echo 'Content-Disposition: attachment; filename="'${ATTACH_NAME1}'"'
 echo
 uuencode $ATTACH1 $ATTACH1
 echo
) | /usr/sbin/sendmail $MAILTO $CC
fi

output it is giving statrt time . it have created more that 2 also



Code:
<html>
<Body>
<table border=1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
START_DATE :  Aug 6 22:34:22
</td>
</tr>
</table>
</Body>
</html>
<br />
<html>
<Body>
<table border=1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
START_DATE :  Aug 6 22:35:12
</td>
</tr>
</table>
</Body>
</html>
<br />

output end time


Code:
<html>
<Body>
<table border = 1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
END_DATE:       Aug 6 22:34:35
</td>
</tr>
</table>
</Body>
</html>
<br />
<br />
<html>
<Body>
<table border = 1>
<tr>
<td colspan=0 bgcolor=#7FFFD4>
END_DATE:       Aug 6 22:35:23
</td>
</tr>
</table>
</Body>
</html>
<br />
<br />

it is also giving data output repeatedly

my input file

Code:
cc_inq_mob.csv



Code:
1,1,http://abc:123/defd/FixedAccInquiryAction.do?accountId=123456&fromCMS=null&operation=3&searchPage=SOHACCOUNTIDINQ
1,1,http://abc:345/defh/FixedAccInquiryAction.do?accountId=123456&fromCMS=null&operation=3&searchPage=SOHACCOUNTIDINQ

please advise
sorry for the trouble
Attached Thumbnails
Nohup apending data each time i run script-capture-jpg  

Last edited by mirwasim; 08-06-2016 at 03:10 PM.. Reason: insert image
    #11  
Old Unix and Linux 08-07-2016   -   Original Discussion by mirwasim
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 21 April 2018, 6:35 PM EDT
Location: San Jose, CA, USA
Posts: 11,208
Thanks: 633
Thanked 3,903 Times in 3,341 Posts
You said you're using bash as your shell, but /bin/sh is an old Bourne shell; not a bash shell on SunOS systems. The following suggestion uses /bin/ksh instead of /bin/sh so we can use the more modern form of command substitution and various variable expansions that are not available in a traditional Bourne shell. (This allows us to avoid invoking several external utilities by using shell variable expansions.) The code used in the following suggestions will work with both bash and ksh, but you'll probably find that ksh on SunOS system is a little faster than bash.

You have said that you're using nohup and cron to run your script sometimes and I asked what the crontab entries look like when you're using them. You still have not supplied any indication of how your script is invoked when run by cron so I can't make any guesses as to whether or not my following suggestions are sufficient to work in the way you run your script from cron. If you are using nohup and any output is being saved in nohup.out, you will need to do something to ensure that any scripts invoked using nohup place any output those scripts produce in unique files (rather than having each of those invocations destroy output produced by other invocations or having output from all of those invocations intermingled in a single output file).

You still have not shown us the contents of the script /path/jmeter, so updating that script to produce files that do not destroy files it produces when multiple versions of it are run concurrently are left as an exercise for the submitter. (AND, if this script is run multiple times concurrently, even with the changes shown in the suggestion below; it will not work because all invocations of /path/jmeter apparently produce output in the single file /path/acc_inq_mob.csv, or /location/path/acc_inq_mob.csv, or /location/path/path/acc_inq_mob.csv (all three of which are created, read, or both) by various commands in your script, but I assume that only one of them is correct.

You have a comment in your code:


Code:
##i remove all the files in the beginning of the script before it run

but that is just a comment and you have lots of files in your script that are appended to (thereby creating the history you talked about). And if you are running multiple copies of this script simultaneously, each invocation of your script is writing to the same file while you need to have each invocation write to a different set of files. The following suggestion assumes that all of the files for which you have defined variables to reference their pathnames that they are temporary files that need to be created as distinct for each invocation of your script and should be removed when your script exits.

In addition to that there is no need to run date and then extract three fields from the output it produces by also running nawk, just invoking date with a format operand is a lot more efficient.

And, instead of using echo to write one line at a time into your temporary files, it would be much more efficient to execute a single command to write the entire contents which reduces not only the number of commands your script executes but also greatly reduces the number of times your script opens and closes your various temporary files.

Why do you define various variables naming your temporary files halfway through your script (after using the full text dozens of times before you define the variables and once or twice after defining them)?

You have lots of references to the directory path:


Code:
ATTACH1="/path/ERROR_FILE.html"
ATTACH2="/path/date.html"
ATTACH3="/path/date_end.html"
ATTACH4="/path/SUCCESS.html"
ATTACH5="/path/ERROR.html"
ATTACH6="/path/output.html"
cd path
sh /path/jmeter -n -t path/jmeter.jmx -l log.jtl ## this script i am calling her
e and all the log which it create i am doing further scripting on that
nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv   > path/acc_inq_mob.log

some of which are absolute referencing the directory /path and the rest of which are relative expanding to:


Code:
cd /location/path
sh /path/jmeter -n -t /location/path/jmeter.jmx -l log.jtl ## this script i am calling her
e and all the log which it create i am doing further scripting on that
nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv   > /location/path/acc_inq_mob.log

I would guess that many, if not all, of the relative pathnames above are incorrect, but the following code sample uses the pathnames used in your script.

Other than some of the inconsistencies mentioned above, the following suggestion might work if you make similar changes in /path/jmeter and clean up the references to the path directory that are not given as absolute pathnames, and you either invoke it with ksh instead of sh or (since it now has a #!/bin/ksh at the start of the first line in the script) you make your script executable and invoke it by its name instead of explicitly using a shell to invoke it:


Code:
#!/bin/ksh

# Remove temporary file when the script exits.
trap 'rm -f "$ATTACH1" "$ATTACH2" "$ATTACH3" "$ATTACH4" "$ATTACH5" "$ATTACH6" "$ERROR_LOG"' EXIT

# Define variables.
SUBJECT="SANITY TEST RESULT FOR $SCRIPT_NAME_1"
ATTACH1="/path/ERROR_FILE.html.$$"
ATTACH2="/path/date.html.$$"
ATTACH3="/path/date_end.html.$$"
ATTACH4="/path/SUCCESS.html.$$"
ATTACH5="/path/ERROR.html.$$"
ATTACH6="/path/output.html.$$"
ATTACH_NAME1=${ATTACH1##*/}
##CC="email@gmail.com"
ERROR_LOG="ERROR_FILE.log.$$"
FROM="server"
MAILTO="email@gmail.com"
SCRIPT_NAME_1="ACCOUNT INQUIRY FOR MOBILE"

# Get start date and time.
DATE_1=$(date '+%b %e %T')

cd /location
> acc_inq_mob.csv.$$	# Should this line be moved after the "cd path" below?

cd path		# Should this be "/path"?


ksh /path/jmeter -n -t path/jmeter.jmx -l log.jtl ## this script i am calling here and all the log which it create i am doing further scripting on that
# Should above line be:
# ksh /path/jmeter -n -t /path/jmeter.jmx -l log.jtl
#    or
# ksh jmeter -n -t jmeter.jmx -l log.jtl


nawk -F, '/false/ && match ($0, /http:\/\/[^\/]*\/[^\/]*/) {print $3, substr ($0, RSTART, RLENGTH)}' log.jtl > "$ERROR_LOG"

# Note that the temp files produced by the next three commands are not removed
# when this script completes, the three following commands could be done by a
# single nawk command (but without a sample of the expected input and output I
# am NOT going to attempt to do that.
nawk '!/ACCOUNT_ID/' /path/acc_inq_mob.csv.$$   > path/acc_inq_mob.log.$$

nawk '{gsub (/<[^>]*>/,_);sub (/\/[^\/]*$/, _, $1); print $1}' FS="[?;]" acc_inq_mob.log.$$ > acc_inq_output1.log.$$
cut -d',' -f3 acc_inq_output1.log.$$ > acc_inq_output.log.$$

# Get end date and time.
DATE_END_1=$(date '+%b %e %T')

printf '%s\n' "<html>
<Body>
<table border=\"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#7FFFD4\">
START_DATE :  $DATE_1
</td>
</tr>
</table>
</Body>
</html>
<br />"  > "$ATTACH2"

printf '%s\n' "<html>
<Body>
<table border = \"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#7FFFD4\">
END_DATE:       $DATE_END_1
</td>
</tr>
</table>
</Body>
</html>
<br />
<br />" > "$ATTACH3"

printf '%s\n' "<br />
<html>
<Body>
<table border = \"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#008000\">
<b>
<font color = \"#FFFFFF\">
SUCCESS RESULT
</font>
</b>
</td>
</tr>
</table>
</Body>
</html>
<br />" > "$ATTACH4"

printf '%s\n' "<br />
<html>
<Body>
<table border = \"1\">
<tr>
<td colspan=\"0\" bgcolor=\"#FF0000\">
<b>
<font color = \"#FFFFFF\">
ERROR RESULT
</font>
</b>
</td>
</tr>
</table>
</Body>
</html>
<br />" > "$ATTACH5"

printf '%s\n' "<br />
<html>
<Body>
$(nawk 'BEGIN{print "<table border=\"1\">"} 
          {print "<tr>";
                  for(i=1;i<=NF;i++)print "<td>" $i "</td>";
                  print "</tr>"} END{print "</table>"}' acc_inq_output.log.$$)
</Body>
</html>
<br />" > "$ATTACH6"

printf '%s\n' "<html>
<Body>
$(nawk 'BEGIN{print "<table border=\"1\">"}
       {print "<tr>";
       print "<TD>";
       for(i=1;i<NF;i++) printf("%s ",  $i);
       print "</TD>";
       print "<TD>" $NF "</TD>";
       print "</tr>"}
     END{print "</table>"}' "$ERROR_LOG")
</Body>
</html>" > "$ATTACH1"

COUNT_ERROR_FILE=$(wc -l < "$ERROR_LOG")
OUNT_ERROR_FILE=${COUNT_ERROR_FILE##* }

if [  $COUNT_ERROR_FILE -eq 0 ]
then (	echo "From: $FROM"
	echo "To: $MAILTO"
	echo "CC: $CC"
	echo "Subject: $SUBJECT"
	echo "MIME-Version: 1.0"
	echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
	echo '---q1w2e3r4t5'
	echo "Content-Type: text/html"
	echo "Content-Disposition: inline"
	cat "$ATTACH2" 
	echo
	cat "$ATTACH3"
	echo
	cat "$ATTACH4"
	echo
	cat "$ATTACH6"
	echo
     )
else (
	echo "From: $FROM"
	echo "Cc: $CC"
	echo "To: $MAILTO"
	echo "Subject: $SUBJECT"
	echo "MIME-Version: 1.0"
	echo 'Content-Type: multipart/mixed; boundary="-q1w2e3r4t5"'
	echo
	echo '---q1w2e3r4t5'
	echo "Content-Type: text/html"
	echo "Content-Disposition: inline"
	echo
	cat "$ATTACH2" 
	echo
	cat "$ATTACH3"
	echo
	echo 
	cat "$ATTACH5"
	echo 
	cat "$ATTACH1"
	echo
	cat "$ATTACH4"
	echo
	cat "$ATTACH6"
	echo
	echo '---q1w2e3r4t5'
	echo 'Content-Type: application; name="'${ATTACH_NAME1}'"'
	echo "Content-Transfer-Encoding: uuencode"
	echo 'Content-Disposition: attachment; filename="'${ATTACH_NAME1}'"'
	echo
	uuencode "$ATTACH1" "$ATTACH1"
	echo
     )
fi | /usr/sbin/sendmail $MAILTO $CC

Note, of course, that with no sample input, no example of the real desired output, no indication of what /path/jmeter does, and no belief that the references to path and /path are correct in the given script; none of this has been tested.

Last edited by Don Cragun; 08-07-2016 at 03:43 AM.. Reason: Add note.
The Following User Says Thank You to Don Cragun For This Useful Post:
mirwasim (08-08-2016)
Sponsored Links
    #12  
Old Unix and Linux 08-08-2016   -   Original Discussion by mirwasim
mirwasim's Unix or Linux Image
mirwasim mirwasim is offline
Registered User
 
Join Date: Dec 2012
Last Activity: 23 February 2018, 8:07 AM EST
Posts: 175
Thanks: 34
Thanked 1 Time in 1 Post
Dear D,

Thanks for your support and time.
from all this i learn to make my script small and i learn

Code:
trap

command

my script issue is fixed now. issue was with the declared variables.

i have fixed it now LinuxLinuxLinuxLinux
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
run shell script under nohup directly [solved] johninweb Shell Programming and Scripting 0 03-15-2012 01:20 AM
Run process with nohup every certain time pepeli30 UNIX for Dummies Questions & Answers 6 12-22-2009 06:40 AM
Run Script With nohup Command On A Different Server King Nothing Shell Programming and Scripting 6 05-22-2009 09:33 AM
last run time of any script RahulJoshi Shell Programming and Scripting 7 10-16-2008 07:42 PM
run script for given time looza Shell Programming and Scripting 0 07-08-2008 11:12 PM



All times are GMT -4. The time now is 07:23 PM.