Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

[Solved] Grep in bash script

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers [Solved] Grep in bash script
# 1  
Old 10-14-2013
[Solved] Grep in bash script
Hi Experts,

I'm writing script to find out last files and its modified date - unfortunately am having problem with the below script.

Error message:
Code:
"grep: sales.txt: No such file or directory"

Code:
#!/bin/bash
var=1
var1=`awk '{n++} END {print n}' sales.txt`

while [[ ${var} -le  ${var1} ]]
do


prod=$var
member=`grep ^$prod, sales.txt | cut -d "," -f2 | tr -d "\15"`

due1=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f3 | tr -d "\15"`
FTP=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f4 | tr -d "\15"`
FolderName=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f5 | tr -d "\15"`
curdate=`date +%Y-%m-%d`
duedate1=$( date +%Y-%m-$due1 )
keyword=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f7 | tr -d "\15"`
#echo "$member","$due1","$FolderName","$keyword","$FTP"

if [[ "$FTP" == "ABC" ]]
then

echo $FolderName
cd /
cd /archive/$FolderName
file1_To_Copy=`ls -tr *"$keyword"* |tail -n 1`
last_mod_date1=`ls --time-style=+%Y-%m-%d -ld $file1_To_Copy | awk '{print $6}'`

else
echo "do nothing"
fi
echo "$member","$due1","$FolderName","$keyword","$FTP","$file_To_Copy","$last_mod_date1"

var=`expr $var + 1`

done

Any help is greatly appreciated
thank you
Last edited by parpaa; 10-14-2013 at 12:51 AM..
# 2  
Old 10-14-2013
Is "sales.txt" file present in the same directory as the script?

And, I couldn't help noticing this:
Quote:
Code:
var1=`awk '{n++} END {print n}' sales.txt`

You could fetch the total number of lines using wc -l instead of calling heavyweights like awk.
# 3  
Old 10-14-2013
Quote:
Originally Posted by parpaa
I'm writing script to find out last files and its modified date - unfortunately am having problem with the below script.

Error message:
Code:
"grep: sales.txt: No such file or directory"

It might help to properly indent your code. For the machine it doesn't matter but it makes for easier reading.

Code:
#!/bin/bash

var=1
var1=`awk '{n++} END {print n}' sales.txt`

while [[ ${var} -le  ${var1} ]] ; do
     prod=$var
     member=`grep ^$prod, sales.txt | cut -d "," -f2 | tr -d "\15"`

     due1=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f3 | tr -d "\15"`
     FTP=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f4 | tr -d "\15"`
     FolderName=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f5 | tr -d "\15"`
     curdate=`date +%Y-%m-%d`
     duedate1=$( date +%Y-%m-$due1 )
     keyword=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f7 | tr -d "\15"`
     #echo "$member","$due1","$FolderName","$keyword","$FTP"

     if [[ "$FTP" == "ABC" ]] ; then
          echo $FolderName
          cd /
          cd /archive/$FolderName
          file1_To_Copy=`ls -tr *"$keyword"* |tail -n 1`
          last_mod_date1=`ls --time-style=+%Y-%m-%d -ld $file1_To_Copy |\
                                  awk '{print $6}'`
     else
          echo "do nothing"
     fi
     echo  "${member}, ${due1}, ${FolderName}, ${keyword}, ${FTP}, ${file_To_Copy}, ${last_mod_date1}"

     var=`expr $var + 1`

done

OK, here are a few pointers to make your code better:

1) get rid of the backticks! Their use is strongly discouraged and they only exist for backwards compatibility. Just because the shell is lenient enough to understand them is no reason for you to use them. Use process substitution ("$( ... )") instead.

2) The shell can do integer arithmetics itself! There is no need to call upon external programs to do what the shell can do. This only slows down your script because calls to external programs require the fork()-system call which is expensive. Instead of "let", "expr" just use "(( ... ))".

3) Declare what you use! Yes, it is possible to use variables just like that. No, this still is not good practice. By first declaring your variables you can give them default values, make sure they are of the right type AND you have the chance of annotating your code. This way you get documentation automatically.

4) Quote your variables! To protect your code of shell interpretation if your variables contain white space you should routinely quote them. Instead of, for instance, grep ^$prod use grep "^${prod}"

5) While this all was not making your code fail the following probably was. I marked bold the problem:

Code:
due1=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f3 | tr -d "\15"`
FTP=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f4 | tr -d "\15"`
FolderName=`grep ^$prod, sales.txt | tail -1 | cut -d "," -f5 | tr -d

grep is called like this: "grep regexp /path/to/file" and the comma is probably not part of the regexp you look for.

Further, you could do yourself a favor and grep for the right line first, then store it to a variable instead of retrieving it over and over again:

Code:
#!/bin/bash

typeset -i var=1                                      # this value means?
typeset -i var1=$(wc -l sales.txt)                    # and this one means?
typeset -i prod=0                                     # this one does?
typeset    workline=""                                # line buffer
typeset    curdate=$(date +%Y-%m-%d)                  # current date
typeset    curdate2=$(date +%Y-%m)                    # current date without days

while [[ ${var} -le  ${var1} ]] ; do
     prod=$var

     workline=$(grep "^${prod}" sales.txt | tail -1 | tr -d "\15")

     member=$(echo "$workline" | cut -d "," -f2)
     due1=$(echo "$workline" | cut -d "," -f3)        # see annotation below
     FTP=$(echo "$workline" | cut -d "," -f4)
     FolderName=$(echo "$workline" | cut -d "," -f5)

     duedate1="${curdate2}-${due1}"                    # see annotation next line
     # duedate1="$(echo "$workline" | cut -d "," -f3)-${due1}"    # does the same as the two noted lines above

     keyword=$(echo "$workline" | cut -d "," -f7)
     #echo "$member","$due1","$FolderName","$keyword","$FTP"

     if [[ "$FTP" == "ABC" ]] ; then
          echo $FolderName
          # cd /                                       # this one is superfluous
          # cd /archive/$FolderName                    # this one too, see below

          file1_To_Copy=$( ls -tr /archive/$FolderName/*"$keyword"* |\
                           tail -n 1 \
                         )
          last_mod_date1=$( ls --time-style=+%Y-%m-%d -ld $file1_To_Copy |\
                            awk '{print $6}' \
                          )
     else
          echo "do nothing"
     fi
     echo  "${member}, ${due1}, ${FolderName}, ${keyword}, ${FTP}, ${file_To_Copy}, ${last_mod_date1}"

     (( var += 1 ))
done

Further suggestions: get some system into your variable naming. It is easy to lose track between upper case lower case, mixed case, etc. identifiers. Personally, i use a (stripped down) sort-of Hungarian Style Notation, but that is more a matter of taste. If i read in a record (like you do with what i called "workline") I would have the resulting variables named to reflect where they come from:

Code:
     chLine=$(grep "^${prod}" sales.txt | tail -1 | tr -d "\15")

     chLineMember=$(echo "$workline" | cut -d "," -f2)
     chLineDue1=$(echo "$workline" | cut -d "," -f3)
     chLineFtp=$(echo "$workline" | cut -d "," -f4)
     fLineFolder=$(echo "$workline" | cut -d "," -f5)

("ch" ("char") is prefix for strings, the last one uses "f" as a prefix because "f" ("file") means a path name for me, so i immediately recognize the variables content just by reading its name). As they all start with "Line" i know where they come from. The form a sort-of "structure" (to [mis-]use that term from the C language).

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
parpaa
# 4  
Old 10-14-2013
Invalid arithmetic operator
Hi bakunin,

Really appreciate for your help, when I tried running the modified script
This the error am facing. Any idea?
Code:
delete.sh: line 3: typeset: 32 sales.txt: syntax error: invalid arithmetic operator (error token is ".txt")

I think the problem is in this line
Code:
typeset -i var1=$(wc -l sales.txt)

# 5  
Old 10-14-2013
Try:
Code:
typeset -i var1=$(wc -l < sales.txt)

You can run the wc command on its own both ways (with and without <), to see the difference, and why it's giving the error.
This User Gave Thanks to Scott For This Post:
parpaa
# 6  
Old 10-14-2013
Thanks Bakunin and Scott for your speedy replies. Much appreciated - it worked
# 7  
Old 10-14-2013
Sorry for the typo in wc -l and thanks to Scott for the correction.

Moderator's Comments:
Mod Comment I changed the thread status to "Solved".


bakunin
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to use grep in a loop using a bash script?

Dear all, Please help with the following. I have a file, let's call it data.txt, that has 3 columns and approx 700,000 lines, and looks like this: rs1234 A C rs1236 T G rs2345 G T Please use code tags as required by forum rules! I have a second file, called reference.txt,... (1 Reply)
Discussion started by: aberg
1 Replies

2. Shell Programming and Scripting

[Solved] Unable to call a python script from bash

Hi, I am trying to run a python script embedded in bash script. But is throwing me an error. Please help. Script: #!/bin/bash nohup /usr/bin/python /opt/web/http.py & Error: /usr/bin/python: can't open file '/opt/web/http.py': No such file or directory Please help me on this. (6 Replies)
Discussion started by: maddy26615
6 Replies

3. Shell Programming and Scripting

[Solved] Help with grep script

Hi, I'm having trouble with a script to copy one line out of multiple files in a directory and copy to a file called test. I've tried the code below but it copies one line out of the first file multiple times not one line out of all the files. Would someone help? I'm very new to all this. ... (8 Replies)
Discussion started by: bob101
8 Replies

4. Shell Programming and Scripting

[Solved] Using SSH in bash script

Hello, I am writing a script that has to log in to X number of servers over a full C class range. I am only keyed on a certain number of these servers and this can change from time to time. The part of my script that deals with this is for i in $(cat $server_list); do ssh ... (0 Replies)
Discussion started by: colinireland
0 Replies

5. Shell Programming and Scripting

Problem using grep in bash script

When it comes to programing and UNIX, I know just enough to be really really dangerous. I have written a python script to parse through a file that contains ~1 million lines. Depending on whether a certain string is matched, the line is copied into a particular file. For the sake of brevity,... (4 Replies)
Discussion started by: errcricket
4 Replies

6. Shell Programming and Scripting

[Solved] perl and grep: get a script to look at more then one file

hi guys i have this very messy script, that looks in /var/log/messages.all for an error and reports if it finds the key works how can i get it to look at more then one file, i.e /var/log/message.all * so it looks in old logs as well thanks exit 0 if (isRenderNode(hostname)); my... (4 Replies)
Discussion started by: ab52
4 Replies

7. Shell Programming and Scripting

Bash script - Grep

Hi, I am very new to bash scripting and I need to write a bash script that takes two arguments, a string and a file. The output should be each line which matches the string *from the beginning of the line*. For example, given a string "ANA" the line starting with "ANABEL" will be printed, but... (9 Replies)
Discussion started by: jboy
9 Replies

8. Shell Programming and Scripting

[Solved] Output in bash script not captured in variable

I'm tring to write down a simple script that would execute a command and wait until it returns a specific result. This is what i did: bjobs_out=`bjobs` while ]; do bjobs_out=`bjobs` sleep 6 done It seems to work until the command 'jobs' return the list of jobs in execution, but... (4 Replies)
Discussion started by: lifedj
4 Replies

9. Shell Programming and Scripting

[SOLVED] how to use my aliases in a bash script

Hi, I confess to be stuck ... as explained here Aliases - Bash Reference Manual I added shopt -s expand_aliasesto my script ... but it didn't work. Later I've read I had to add this at the begning of my my script : . ~/.bash_aliases(of course my aliases are in this file) but again, the... (0 Replies)
Discussion started by: xib.be
0 Replies

10. Shell Programming and Scripting

Bash script (using find and grep)

I'm trying to make a simple search script but cannot get it right. The script should search for keywords inside files. Then return the file paths in a variable. (Each file path separated with \n). #!/bin/bash SEARCHQUERY="searchword1 searchword2 searchword3"; for WORD in $SEARCHQUERY do ... (6 Replies)
Discussion started by: limmer
6 Replies
Login or Register to Ask a Question