Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Count the number of files to delete doesnt match Post 302986097 by Scrutinizer on Sunday 20th of November 2016 01:46:40 AM
Old 11-20-2016
Further to Don's remarks, if you are using file name expansion with the for loop,

The first attempt might look like this:

Code:
for file in file_20160*.lis
do
  grep "$file" bplist.txt >>filetodelete.txt
done

It is important to test for the case when there are zero files that fit the pattern, otherwise you end up with an a variable that contains file_20160*.lis, which would then become a regular expression, since that is what grep, like so:

Code:
grep file_20160*.lis bplist.txt >>filetodelete.txt

which would then delete any file names that start with "file_2016" followed by zero or more zeroes and ".lis" from the file..

Now probably those files do not exist in your case, but it is best to avoid a possible loop hole altogether, by testing if a file exists and use string matching instead of regex matching, using grep's -F parameter. To avoid partial file name matches (where the pattern or string that grep is looking for is a subset of the filename) another important parameter would be the -x option, which forces line matches. A third thing would be to avid the possibility that files that start with a - sign could be interpreted as an option flag to grep. One way to stop this is by using the -- flag. Because you file pattern starts with file that will not be an issue here, but it is good practice to do that anyways, so that in future if you ever change the pattern so that it starts with an *, this will not break things.

So then it becomes:
Code:
for file in file_20160*.lis
do
  if [ -f "$file" ]; then 
    grep -Fx -- "$file" bplist.txt >>filetodelete.txt
  fi
done

Now that last thing here is that you are appending to the file here, probably out of necessity, otherwise the file would be overwritten with very loop. An alternative would be to redirect the loop itself so the file would only be opened once and you do not have to delete the file prior to running the loop:

Code:
for file in file_20160*.lis
do
  if [ -f "$file" ]; then
    grep -Fx -- "$file" bplist.txt
  fi
done > filetodelete.txt

One last thing. This is still an expensive way to do it because an external program in a subshell is used to perform the operations for every iteration in the for loop, which is resource intensive.

An alternative would be to use a pipe (|) and grep's - operator for stdin, which most grep's (but not all) will honor, together with the file flag -f
Code:
ls file_20160*.lis | grep -Fxf -  -- bplist.txt > filetodelete.txt

if there are not too many files in the directory.

Or use the more robust:
Code:
for file in file_20160*.lis
do
  if [ -f "$file" ]; then
    printf "%s\n" "$file"
  fi
done |
grep -Fxf -  -- bplist.txt > filetodelete.txt

Since the - operator for stdin is not universally supported in grep, another way would be to use process substitution ( <( ... ) ) that is used in for modern bash, ksh93 or zsh:
Code:
grep -Fxf <(ls file_20160*.lis) -- bplist.txt > filetodelete.txt

if there are not too many files in the directory.

Or again the more robust variety:
Code:
grep -Fxf <(
  for file in file_20160*.lis
  do
    if [ -f "$file" ]; then
      printf "%s\n" "$file"
    fi
  done
) -- bplist.txt > filetodelete.txt

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Grep, count and match two files

I am writing the below script to do a grep and count number of occurances between two tab delimited files. I am trying to achieve.. 1) Extract column 2 and column 3 from the S.txt file. Put it in a temp pattern file 2) Grep and count column 2 in D.txt file 3) Compare the counts between... (19 Replies)
Discussion started by: madhunk
19 Replies

2. UNIX for Dummies Questions & Answers

compare two files if doesnt match then display error message

hi , i have one file ,i need to search particular word from this file and if content is matched then echo MATCHED else NOT MATCHED file contains : mr x planned to score 75% in exam but end up with 74%. word to be searched id 75% please help me out . waiting for reply thanks in advance (2 Replies)
Discussion started by: atl@mav
2 Replies

3. Shell Programming and Scripting

Awk Array doesnt match for substring

Awk Array doesnt match for substring nawk -F"," 'FNR==NR{a=$2 OFS $3;next} a{print $1,$2,a}' OFS="," file1 file2 I want cluster3 in file1 to match with cluster3int in file2 output getting: Output required: Help is appreciated (8 Replies)
Discussion started by: pinnacle
8 Replies

4. UNIX for Dummies Questions & Answers

Comparing two files and count number of lines that match

Hello all, I always found help for my problems using the search option, but this time my request is too specific. I have two files that I want to compare. File1 is the index and File2 contains the data: File1: chr1 protein_coding exon 500 600 . + . gene_id "20532";... (0 Replies)
Discussion started by: DerSeb
0 Replies

5. Shell Programming and Scripting

Match and count the number of times

ile1 Beckham Ronaldo file2 Beckham Beckham_human Ronaldo Ronaldo_spain Ronaldo Ronaldo_brazil Beckham Beckham_manch Zidane Zidane_Fran Rooney Rooney_Eng Output shud be (1 Reply)
Discussion started by: cdfd123
1 Replies

6. UNIX for Dummies Questions & Answers

Count number of files in directory excluding existing files

Hi, Please let me know how to find out number of files in a directory excluding existing files..The existing file format will be unknown..each time.. Thanks (3 Replies)
Discussion started by: ammu
3 Replies

7. Shell Programming and Scripting

Count the delimeter from a file and delete the row if delimeter count doesnt match.

I have a file containing about 5 million rows, in the file there are some records which has extra delimiter at random position. (we dont know the positions), now we have to Count the delimeter from each row and if the count of delimeter is not matching then I want to delete those rows from the... (5 Replies)
Discussion started by: Akumar1
5 Replies

8. Shell Programming and Scripting

Count number of match words

Input: some random text SELECT TABLE1 some more random text some random text SELECT TABLE2 some more random text some random text SELECT TABLE3 some more random text some random text SELECT TABLE1 some more random text Output: 'SELECT TABLE1' 2 'SELECT TABLE2' 1 'SELECT TABLE3' 1 I... (5 Replies)
Discussion started by: chitech
5 Replies

9. Shell Programming and Scripting

How to count number of files in directory and write to new file with number of files and their name?

Hi! I just want to count number of files in a directory, and write to new text file, with number of files and their name output should look like this,, assume that below one is a new file created by script Number of files in directory = 25 1. a.txt 2. abc.txt 3. asd.dat... (20 Replies)
Discussion started by: Akshay Hegde
20 Replies

10. Shell Programming and Scripting

Count the pipes "|" in line and delete line if count greter then number.

Hello, I have been working on Awk/sed one liner which counts the number of occurrences of '|' in pipe separated lines of file and delete the line from files if count exceeds "17". i.e need to get records having exact 17 pipe separated fields(no more or less) currently i have below : awk... (1 Reply)
Discussion started by: ketanraut
1 Replies
All times are GMT -4. The time now is 05:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy