Error when executing script


 
Thread Tools Search this Thread
Operating Systems Linux Red Hat Error when executing script
# 8  
Old 05-05-2017
I don't know if the method I am using is complex, but the aim of this script is to generate reports every 1st, 7th and 15th of every month. However, the reports won't be generated until a few days after these dates. Like the reports for the 1st will be generated on the 5th of the month for example. And reports for the 7th will be generated on the 9th or 10th. And reports for the 15th will be generated on the 17th. These days of course will have to fall on a weekday, and not a weekend, which is why we cannot put a fixed date, for each batch.

For the reports on the 1st, I have to copy files dated 2nd onwards from the main directory that it was generated in, into another directory.
This is why I am cracking my head on how to get this done.

For example, these are the files listed in the main directory :

Code:
-rw-r-----  1 Debian-exim adm  1705192 May  1 06:25 mainlog.5.gz
-rw-r-----  1 Debian-exim adm 16737648 May  2 06:25 mainlog.4.gz
-rw-r-----  1 Debian-exim adm 11031310 May  3 06:33 mainlog.3.gz
-rw-r-----  1 Debian-exim adm  8258806 May  4 06:28 mainlog.2.gz
-rw-r-----  1 Debian-exim adm 37434708 May  5 06:26 mainlog.1
-rw-r-----  1 Debian-exim adm 14248899 May  5 17:43 mainlog
root@L28mustang:/var/log/exim4#

I have to copy these files, as you can see, dated May 2nd onwards till the latest (in this case, the 5th) to another directory. How do I create a bash script to do this? These are however not the only files here. There are older files from previous months also here. I need to copy over only the latest months' files dated 2nd onwards to another directory. The script has to be intelligent enough to recognize the files needed for that particular batch (1st, 7th, or 15th), and copy it over to another directory.

Then once it has been copied over to the other directory, this command has to be executed :

Code:
exigrep L28stream1 mainlog.2.gz | egrep -v "jiun.shyong.hor@ericsson.com|chander.c.shekher@ericsson.com|nagios|L28eagle" > mainlog.2.filtered

Means the oldest mainlog file will be labelled with the higher number. In the example above, the date of the oldest file was 16th (this was for batch 15). The report was generated on the 17th. Therefore the name of the oldest mainlog file will be "mainlog.2". Then there were 2 mainlog files generated on the 17th, which was mainlog.1 and mainlog.

This is the script that I have created so far. Any corrections and additional advise will definitely be very appreciated :

Code:
#!/bin/bash

DAY=$(date + %d)
MONTH=$(date +%b)
YEAR=$(date +%Y)
BC01="Blast_BC01"
BC15="Blast_BC15"
DIR1=$MONTH$YEAR_$BC01
DIR2=$MONTH$YEAR_$BC15
FOLDER=$YEAR

DIGMON=$(date +%m)

D=$(stat -c %z mainlog.* | awk -F"[ -]" '{print $3}')
FDAY=$(date -d "$D" '+%d')
FMON=$(date -d "$D" '+%m')
FYEAR=$(date -d "$D" '+%Y')

FILEDATE=$YEAR"-"$DIGMON"-"$i

cd /var/log/exim4

if [ ! -d "$YEAR" ]; then
    mkdir $YEAR
        if [ $DAY < 15 ];then
            mkdir -p /var/log/exim4/$YEAR/$DIR1
            for (( $D=2; $D<=$DAY; $D++ ))
                cp -p mainlog* ./$YEAR/$DIR1
            
        else
            mkdir -p /var/log/exim4/$YEAR/$DIR2
            for (( $D=16; $D<=$DAY; $D++ ))
                cp -p mainlog* ./$YEAR/$DIR2
        fi
else
    cd $FOLDER
        if [ $DAY < 15 ];then
            mkdir -p /var/log/exim4/$YEAR/$DIR1
            for ($D=2; $D<$DAY; $D++)
                cp -p mainlog* ./$YEAR/$DIR1
                    cd /$YEAR/$DIR1
                    count=$(ls -lrt | wc -l)
                    for (i=;i<=$count;i++)
                    exigrep L28stream1 mainlog.$count.gz | egrep -v "jiun.shyong.hor@ericsson.com|chander.c.shekher@ericsson.com|nagios|L28eagle" > mainlog.$count.filtered
            
        else
            mkdir -p /var/log/exim4/$YEAR/$DIR2
            for ($D=16; $D<$DAY; $D++)
                cp -p mainlog* ./$YEAR/$DIR2
        fi
fi

---------- Post updated at 06:32 PM ---------- Previous update was at 06:18 PM ----------

For TESTING purposes ONLY, however, I tried running some commands from the terminal to copy files from the directory /home/emokheng to /home/emoaigin :

These are the files in /home/emokheng :

Code:
[root@L28tstream1 emokheng]# ls -lrt
total 8
-rw-rw-r-- 1 emokheng emokheng 328 Aug 18  2015 foreign.pl
-rw-rw-r-- 1 emokheng emokheng  91 Aug 18  2015 afterinstallforeign.pl

As you can see the dates of the files are August 18th. For report generation this will be considered batch 15. These are the steps I have performed :

Code:
[root@L28tstream1 emokheng]# D="$(stat -c %z * | awk -F"[ -]" '{print $3}')"
[root@L28tstream1 emokheng]# echo "$D"
18
18
[root@L28tstream1 emokheng]# for (( "$D"=16; "$D"<=20; "$D"++ ));
> do
> cp -r * /home/emoaigin/
> done
-bash: ((: 18
18=16: syntax error in expression (error token is "18=16")
[root@L28tstream1 emokheng]#

What is wrong in these steps? Why does it not recognize files dated 18th and copy it over?
# 9  
Old 05-05-2017
You can't use it like that: for (( "$D"=16; "$D"<=20; "$D"++ ));. Use unexpanded variable names like for (( i=1; i<5; i++ )); do echo $i; done. $D is already expanded by the shell.
# 10  
Old 05-05-2017
But I want to copy files with a certain date over from one dir to another.
How do i do this?

I tried using the for loop like this :

Code:
[root@L28tstream1 emokheng]# for (( i=16; i<20; i++ ));
> do
> cp -r * /home/emoaigin/;
> done
cp: overwrite `/home/emoaigin/afterinstallforeign.pl'? y
cp: overwrite `/home/emoaigin/foreign.pl'? y
cp: overwrite `/home/emoaigin/afterinstallforeign.pl'? y
cp: overwrite `/home/emoaigin/foreign.pl'? y
cp: overwrite `/home/emoaigin/afterinstallforeign.pl'? y
cp: overwrite `/home/emoaigin/foreign.pl'? y
cp: overwrite `/home/emoaigin/afterinstallforeign.pl'? y
cp: overwrite `/home/emoaigin/foreign.pl'? y
[root@L28tstream1 emokheng]#

As you can see, it does the loop for times, (i.e copying 2 files 4 times).
This is not I want. What I want is for the script to recognize the files dated from the 16th to the latest date, and copy it over to the other directory, without repeating or overwriting like above.

Is this possible?
# 11  
Old 05-05-2017
Again, the $D is an argument and need to be in quotes.
Code:
echo $D

lets the shell do variable substitution then word splitting then globbing (replace wildcards with matching filenames). The word splitting reformats the line breaks to spaces.
Code:
echo "$D"

lets the shell do variable substitution only.
# 12  
Old 08-09-2017
Thank you so much for all the comments and suggestions.

I have now managed to compile the whole script, but still needs a bit more tweaking as there are few more errors.

Part of my script does this :

Code:
new=1
for f in `find . -name '*.filtered' | sort -r`; do                                                                  
> eximstats -nr "$f" > "eximstats_day"$new"_"$eximBC01.txt";
> eximstats -xls -nr  "$f" > "eximstats_day"$new"_"BC01.xls";
> ((new++));
> done

This returns errors like this :

Code:
bash: eximstats_day1_01082017.txt;
eximstats -xls -nr  ./mainlog.7.filtered > eximstats_day1_BC01.xls: No such file or directory
bash: eximstats_day2_01082017.txt;
eximstats -xls -nr  ./mainlog.6.filtered > eximstats_day2_BC01.xls: No such file or directory
bash: eximstats_day3_01082017.txt;
eximstats -xls -nr  ./mainlog.5.filtered > eximstats_day3_BC01.xls: No such file or directory
bash: eximstats_day4_01082017.txt;
eximstats -xls -nr  ./mainlog.4.filtered > eximstats_day4_BC01.xls: No such file or directory
bash: eximstats_day5_01082017.txt;
eximstats -xls -nr  ./mainlog.3.filtered > eximstats_day5_BC01.xls: No such file or directory
bash: eximstats_day6_01082017.txt;
eximstats -xls -nr  ./mainlog.2.filtered > eximstats_day6_BC01.xls: No such file or directory

However, when I execute the code directly on the terminal, there is no error (without the for loop) :

Code:
# eximstats -nr  mainlog.7.filtered > eximstats_day1_01082017.txt
# eximstats -xls -nr  mainlog.7.filtered > eximstats_day1_01082017.xls

Is there something wrong with the syntax?

---------- Post updated 08-09-17 at 06:24 PM ---------- Previous update was 08-08-17 at 07:19 PM ----------

I found out that the error was variable substitution. I corrected the code like this, and it worked properly :

Code:
new=1
for f in `find . -name '*.filtered' | sort -r`; do
       eximstats -nr "$f" > eximstats_day"$new"_"$eximBC07".txt;
       eximstats -xls -nr  "$f" > eximstats_day"$new"_"$eximBC07".xls;
       ((new++));
done

However, I have another part of the script that has this line :

Code:
eximstats -merge eximstats_day*.txt > "eximstats.consolidated_$MONTH$YEAR.txt"

I did not put the "" for the $MONTH$YEAR, but it still produced the output I wanted. Why is this?
# 13  
Old 08-09-2017
So far you did fairly well. A few comments to your code:

Code:
new=1
for f in `find . -name '*.filtered' | sort -r`; do
       eximstats -nr "$f" > eximstats_day"$new"_"$eximBC07".txt;
       eximstats -xls -nr  "$f" > eximstats_day"$new"_"$eximBC07".xls;
       ((new++));
done

First, you should not use backticks for command substitution any more. If you are not using an original Bourne shell (which would only know the backticks and nothing else) you should use the POSIX-construct you had at the beginning of the thread:

Code:
for f in $(find . -name '*.filtered' | sort -r) ; do

Second, you need only one quote for a whole string. i.e. replace this:
Code:
eximstats -nr "$f" > eximstats_day"$new"_"$eximBC07".txt

with this:
Code:
eximstats -nr "$f" > "eximstats_day${new}_${eximBC07}.txt"

Third, you need no ";" at the end of a line. You only need them if you have more than one command on the same line:
Code:
command1 ; command2

but:
Code:
command1
command2

and a last point: when you use integer environments you should have spaces surrounding them. It is not only easier to read but some shells are quite picky about this:

Code:
((i=i+1))            # might work, but this
(( i=i+1 ))          # is safer and this:
(( i = i + 1 ))      # might even be easier to read

Finally, a suggestion: when you write scripts you should try to avoid relative pathes. If you use "." or ".." in a script it tends to work if called in one directory but fail if called in another. Better to avoid this sort of risk and only use absolute pathes.

So, here is like i would write it:

Code:
new=1
for f in $(find /some/directory -name '*.filtered' | sort -r) ; do
       outfile="/some/directory/eximstats_day${new}_${eximBC07}"
       eximstats       -nr "$f" > "${outfile}.txt"
       eximstats -xls -nr "$f" > "${outfile}.xls"
       (( new++ ))
done

Quote:
However, I have another part of the script that has this line :

Code:
eximstats -merge eximstats_day*.txt > "eximstats.consolidated_$MONTH$YEAR.txt"

I did not put the "" for the $MONTH$YEAR, but it still produced the output I wanted. Why is this?
A string is a string. Hence, this:

Code:
string="abc"def

and this:

Code:
string="abcdef"

produces the same. It would not produce the same only if in the part not covered by quotes would contain something the shell interprets: blanks, tabs, newlines, and so on. Therefore you can (and in fact should, as i said above) cover always your whole string into a single pair of double quotes:

Code:
var="with spaces in it"
echo "this is a string ${var} - and here it continues"

I hope this helps.

bakunin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Gettting error while executing script

getting error as below while executing script in linux. OS version: Linux VGP-3GPSDB-LX 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux ./imxtract.sh: line 395: unexpected EOF while looking for matching ``' ./imxtract.sh: line 402: syntax error:... (1 Reply)
Discussion started by: Riverstone
1 Replies

2. UNIX for Dummies Questions & Answers

Error executing the script

I have the following script test.sh owned by dwdev account and group dwdev, the permissions on the script are as follows. -rw-r-x--- 1 dwdev dwdev 279 Sep 17 13:19 test.sh Groups: cat /etc/group | grep dwdev dwdev:x:704:dwdev dwgroup:x:725:dwdev writers:x:726:dwdev User: cat /etc/passwd |... (3 Replies)
Discussion started by: Ariean
3 Replies

3. Linux

Libssh2 error while executing script

hi all, i am getting libssh2 error while executing script in RHEL 6, when i locate that file its not there below is the ouput of this # locate libssh2_agent_init # cat /etc/issue Red Hat Enterprise Linux Server release 6.1 (Santiago) Kernel \r on an \m how do i resolve this issue, i... (1 Reply)
Discussion started by: muzaffar.k
1 Replies

4. Shell Programming and Scripting

Error executing script

Please delete de thread. Thanks. (10 Replies)
Discussion started by: Rodrih92
10 Replies

5. Shell Programming and Scripting

getting the error 'not found' while executing the script

Hi, I am not able to figure out what the problem is: getting the following error sqltst.sh: 1: not found here is the script #!/bin/sh . /home/dev1/.profile . /home/dev1/.infenv `sqlplus -s $REPDB_LOGON << EOF SET SERVEROUT ON SET FEEDBACK OFF SET HEADING OFF SET TRIMSPOOL... (4 Replies)
Discussion started by: svajhala
4 Replies

6. Shell Programming and Scripting

Error while executing a script

Hi Please assist. Im getting an error while execuing the script name d "cdsnd.basel.cd_new " as siiadm user. Thanks. siiadm> ls -l total 64 -rwxr-xr-x 1 siiadm sboadm 1004 Sep 17 2008 cdsnd.basel.cd -rwxr-xr-x 1 siiadm sapsys 998 Nov 16 09:14 cdsnd.basel.cd_new... (1 Reply)
Discussion started by: samsungsamsung
1 Replies

7. UNIX for Dummies Questions & Answers

Error Executing the script.

Hi , I m getting an error after executing the script. My script. Script is used to find out the date on 8 different machines(mentioned in SERVERNAMES file). I have added public key to avoid ssh password and ssh without password working fine. #!/bin/sh fn_VMFind() { Date=`ssh -t... (5 Replies)
Discussion started by: pinga123
5 Replies

8. Shell Programming and Scripting

Error while executing the below script

I am executing the below in telnet #!/usr/bin/ksh File1=simple.txt # The file to check LogFile=simple.log # The log file DelayMax=30 # Timeout delay Tolerance=2 # BEGIN ############################## while true do StampNow=$(date +%s)/60 # stamp in minutes ... (3 Replies)
Discussion started by: chinniforu2003
3 Replies

9. Shell Programming and Scripting

Error while executing a script

My script is as below : #!/bin/sh for line in `cat Results.txt` do FEILD1=`echo $line |awk -F"|" '{print $1}'` FEILD2=`echo $line |awk -F"|" '{print $2}'` FEILD3=`echo $line |awk -F"|" '{print $3}'` FEILD4=`echo $line |awk -F"|" '{print $4}'` echo "$FEILD1 $FIELD2 $FIELD3 $FIELD4" done ... (15 Replies)
Discussion started by: shwetainnani
15 Replies

10. Shell Programming and Scripting

error while executing the script

Hello I am executing the following script nawk 'NR == 1 || substr($0,63,5) ~ /H... / && \ _++ == 2 { fn && close(fn); fn = "part_" ++c; _ = 1 } { print > fn }' sample.dat When i execute as it is it is executing fine. but when i execute the whole script as a single line like below ... (2 Replies)
Discussion started by: dsdev_123
2 Replies
Login or Register to Ask a Question