How to exclude some files from the loop on the directory?

Login or Register to Ask a Question and Join Our Community

How to exclude some files from the loop on the directory?

Tags

shell, shell scripts

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting How to exclude some files from the loop on the directory?

03-10-2017

Registered User

112, 3

Join Date: Jan 2012

Last Activity: 25 June 2018, 7:40 AM EDT

Posts: 112

Thanks Given: 17

Thanked 3 Times in 3 Posts

How to exclude some files from the loop on the directory?

I have one question.
On the directory I have many files start with

Code:

DB.DAILYxxxxxxx.YYYYMMDD.HHMMSS

and I have several files with other format, like

Code:

LET.20170310
daily.20170310
tba.20170310

How can I exclude from my loop DB.DAILY files?

I tried

Code:

ls *20170310* | while read line
do
done

But it takes all.

Thanks for contribution

digioleg54

View Public Profile for digioleg54

Find all posts by digioleg54

03-10-2017

Registered User

1,781, 705

Join Date: May 2008

Last Activity: 10 November 2021, 5:38 PM EST

Posts: 1,781

Thanks Given: 62

Thanked 705 Times in 653 Posts

Code:

ls *20170310* | grep -v DB.DAILY | while read filename
do
   ...
done

Last edited by Aia; 03-10-2017 at 11:29 PM..

Aia

View Public Profile for Aia

Find all posts by Aia

03-10-2017

Registered User

270, 99

Join Date: Feb 2017

Last Activity: 13 May 2020, 6:57 AM EDT

Location: United Kingdom

Posts: 270

Thanks Given: 16

Thanked 99 Times in 85 Posts

Hi,

You could try a script like this, perhaps.

Code:

#!/bin/bash
for file in `/bin/ls files/`
do
        case "$file" in
                DB.DAILY*)
                        #We want to skip this
                        continue
                        ;;
                *)
                        #But we want to process everything else
                        echo "$file"
                        ;;
        esac
done

So the idea is that for every file that's not got a name starting with "DB.DAILY" we echo out its filename. If its name does start with "DB.DAILY" we ignore it and move on to the next file.

Here's a sample of it being run, so you can see the output.

Code:

$ ls files
daily.20170310  DB.DAILYxxxxxxx.YYYYMMDD.HHMMSS  LET.20170310  tba.20170310
$ ./script.sh
daily.20170310
LET.20170310
tba.20170310
$

Hope this helps.

drysdalk

View Public Profile for drysdalk

Find all posts by drysdalk

03-10-2017

Registered User

12,315, 4,560

Join Date: Jul 2012

Last Activity: 22 November 2019, 4:29 PM EST

Location: San Jose, CA, USA

Posts: 12,315

Thanks Given: 952

Thanked 4,560 Times in 3,818 Posts

For the names listed, you could also just use:

Code:

for line in file/[^D]*
do	printf 'Processing file "%s"\n' "$line"
	# whatever else you want to do...
done

which with your sample files would produce the output:

Code:

Processing "files/LET.20170310"
Processing "files/daily.20170310"
Processing "files/tba.20170310"

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

03-11-2017

Registered User

1,781, 705

Join Date: May 2008

Last Activity: 10 November 2021, 5:38 PM EST

Posts: 1,781

Thanks Given: 62

Thanked 705 Times in 653 Posts

Quote:

Originally Posted by Don Cragun

For the names listed, you could also just use:

Code:

for line in file/[^D]*
do	printf 'Processing file "%s"\n' "$line"
	# whatever else you want to do...
done

which with your sample files would produce the output:

Code:

Processing "files/LET.20170310"
Processing "files/daily.20170310"
Processing "files/tba.20170310"

Of course, that would assume either that there are no other files or directories with different timestamp as part of their name or that any other name entry is wanted except those that do not start with upper case D. e.g. tba.20170307

Those assumptions could be dispersed with a few modifications.

Code:

for fname in [^D]*20170310*
do
  if [ -f "$fname" ]; then
	printf 'Processing file "%s"\n' "$fname"
	# whatever else you want to do...
  fi
done

Here's another alternative using Perl as the filter.

Code:

 perl -le '$,="\n";print grep {/^[^D].*20170310/ and -f} <*>' | while read f; do #do something with f; done

Last edited by Aia; 03-11-2017 at 01:00 AM..

Aia

View Public Profile for Aia

Find all posts by Aia

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Loop through files in directory

I am trying to loop through files in a directory, and sort each file. No matter what changes I make to the code, I get the following errors: 'aunch.sh: line 4: syntax error near unexpected token `do 'aunch.sh: line 4: `for f in ${FILES}/*; do #!/bin/bash FILES=$(pwd) for f in ${FILES}/*;...

2. Shell Programming and Scripting

Loop through files in directory

I am trying to loop through files in a directory, and sort each file. No matter what changes I make to the code, I get the following errors: 'aunch.sh: line 4: syntax error near unexpected token `do 'aunch.sh: line 4: `for f in ${FILES}/*; do #!/bin/bash FILES=$(pwd) for f in ${FILES}/*;...

3. Shell Programming and Scripting

Loop through files in a directory

Hello How do I loop through files in a specific directory ? This script is not working! #! /bin/bash FILES=/usr/desktop/input/* For f in $FILES; do awk '-v A="$a" -v B="$b" {$6=($1-64)/2 ;$7=((10^($6/10))/A)^(1/B) ; print}' OFS="\t" $f > /root/Desktop/output/$f.txt; done ...

4. UNIX for Dummies Questions & Answers

[Solved] Writing a loop to changing the names of files in a directory

Hi, I would like to write a loop to change the names of files in a directory. The files are called data1.txt through data1000.txt. I'd like to change their names to a1.txt through a1000.txt. How do I go about doing that? Thanks!

5. UNIX Desktop Questions & Answers

Copy all files in 1 directory to another usinge for-in loop

I was looking to get some help with copying files in one directory to another using a for-in loop. My script file is called copyfile and here is what I have: for file in $(ls -a $1) do cp $file ~/dir-2 done When I run copyfile dir-1 this is what I get cp: omitting directory `.'...

6. Shell Programming and Scripting

loop through files in directory

hi all i have some files present in a directory i want to loop through all the files in the directory each time i loop i should change the in_file parameter in the control file and load it into a table using sql loader there is only one table where i have to load alll the files ...

7. Shell Programming and Scripting

exclude lines in a loop

I use while do - done loop in my shell script. It is working as per my expectations. But I do not want to process all the lines. I am finding it difficult to exclude certain lines. 1) I do not want to process blank lines as well as lines those start with a space " " 2) I do not want to...

8. Shell Programming and Scripting

Loop certain code to all files within directory

Hi all, Can somebody help me with this problem pls. I need to extract one specific line from each files in a folder and put the all lines extracted in a unique output file in the following format. line extracted, respective name of file, date of file. I�m, trying the part to extract...

9. AIX

loop through the directory for files and sort by date and process the first file

hello i have a requirement where i have a direcotry in which i get files in the format STOCKS.20080114.dat STOCKS.20080115.dat STOCKS.20080117.dat STOCKS.20080118.dat i need to loop through the directory and sort by create date descending order and i need to process the first file. ...

10. Shell Programming and Scripting

Loop through files in a directory

Hi, I want to write bash script that will keep on looking for files in a directory and if any file exists, it processes them. I want it to be a background process, which keeps looking for files in a directory. Is there any way to do that in bash script? I can loop through all the files like...

Login or Register to Ask a Question