Sponsored Content
Top Forums Shell Programming and Scripting Bash - Find files excluding file patterns and subfolder patterns Post 303003829 by jim mcnamara on Thursday 21st of September 2017 07:13:58 AM
Old 09-21-2017
Maintaining something like that is hard - as you found out.
Consider splitting up the lists, make a file with the exclusion patterns, one pattern per line.
Call it exclusions. Changes in red.

Example:
Code:
find "$PATH1"  -type f \( -name "*" ! -iname "*~"  ! -iname "*.iso"  ! -iname "*.tar"  ! -iname "*.bz2"  ! -iname "*.gz"
  ! -iname "*.tgz"  ! -iname "*.7z"  ! -iname "*.sfx"  ! -iname "*.zip"  ! -iname "*.rpm" 
 ! -iname "*.gz.aa"  ! -iname "*.rpm"  ! -iname "*.pdf"  ! -iname "*.png"  ! -iname "*.jpg"  ! -iname "*.dll" 
 ! -iname "*.exe"  ! -iname "*.xsession*"  \) -a -type d \( -name "/etc/gconf/gconf.*" -o -name ".cache" 
-o -name ".Cache" -o -name ".mozilla" -o -name ".googleearth"  \) -prune  -o -print    | grep -v -f $HOME/exclusions > resultfile 

In order to test exclusions you can feed it output from a directory that had oddball file names
Code:
ls -a /path/to/somedir | grep  -f $HOME/exclusions   # note: no -v option

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To find 3 patterns in a file

hy i have a requirement in which my script needs to find 3 patterns in a file and if any pattern is missing it should sent a mail Patterns Interval60min_Daily_readings$a.txt Interval_Daily_readings$a.txt Daily_readings$a.txt Basically i want to test for the above Patterns in the... (2 Replies)
Discussion started by: ali560045
2 Replies

2. Shell Programming and Scripting

Searching patterns in 1 file and deleting all lines with those patterns in 2nd file

Hi Gurus, I have a file say for ex. file1 which has 3500 lines in it which are different account numbers and another file (file2) which has 230000 lines in it. I want to read all the lines in file1 and delete all those lines from file2 which has that same pattern as in file1. I am not quite... (4 Replies)
Discussion started by: toms
4 Replies

3. Shell Programming and Scripting

Find files that do not match specific patterns

Hi all, I have been searching online to find the answer for getting a list of files that do not match certain criteria but have been unsuccessful. I have a directory that has many jpg files. What I need to do is get a list of the files that do not match both of the following patterns (I have... (21 Replies)
Discussion started by: nikos-koutax
21 Replies

4. Shell Programming and Scripting

Excluding patterns from a list

I have the following code that takes the command line arguments. However I want to remove from the command line list the user options. For example, removing --quiet --shift=3 sort=4/5/6 I have written the following code to take care of this situation. set strLst = `echo $argv | tr '... (3 Replies)
Discussion started by: kristinu
3 Replies

5. UNIX for Dummies Questions & Answers

Find diff between two patterns in two files and append

Hi, I'm a newbie at programming in Unix, and I seem to have a task that is greater than I can handle. Trying to learn awk by the way (but in the end, i just need something that works). My goal is to compare two files and output the difference between the two. I've been reading, and I think I... (5 Replies)
Discussion started by: legato22
5 Replies

6. Shell Programming and Scripting

Find matched patterns in multiple files

Hi, I need help to find matched patterns in 30 files residing in a folder simultaneously. All these files only contain 1 column. For example, File1 Gr_1 st-e34ss-11dd bt-wwd-fewq pt-wq02-ddpk pw-xsw17-aqpp Gr_2 srq-wy09-yyd9 sqq-fdfs-ffs9 Gr_3 etas-qqa-dfw ddw-ppls-qqw... (10 Replies)
Discussion started by: redse171
10 Replies

7. Shell Programming and Scripting

Find matched patterns in a column of 2 files with different size and merge them

Hi, i have input files like below:- input1 Name Seq_ID NewID Scores MT1 A0QZX3 1.65 277.4 IVO A0QZX3 1.65 244.5 HPO A0QZX3 1.65 240.5 RgP A0Q3PP 5.32 241.0 GX1 LPSZ3S 96.1 216.9 MEL LPSS3X 4.23 204.1 LDD LPSS3X 4.23 100.2 input2 Fac AddName NewID ... (9 Replies)
Discussion started by: redse171
9 Replies

8. Shell Programming and Scripting

Find matched patterns and print them with other patterns not the whole line

Hi, I am trying to extract some patterns from a line. The input file is space delimited and i could not use column to get value after "IN" or "OUT" patterns as there could be multiple white spaces before the next digits that i need to print in the output file . I need to print 3 patterns in a... (3 Replies)
Discussion started by: redse171
3 Replies

9. Shell Programming and Scripting

Help needed in excluding certain word patterns

Hi, I need help with following. I need to exclude words that match following patterns a. more than length 4 (example SBRAP) b. contains mixture uppercase and lower case regardless of the length (example GSpD) File contains COFpC MCHX SP SNFCA GEH SBRAP DGICA JPMpE WFCpP GSpD AXL... (5 Replies)
Discussion started by: jakSun8
5 Replies

10. Shell Programming and Scripting

Find files not matching multiple patterns and then delete anything older than 10 days

Hi, I have multiple files in my log folder. e.g: a_m1.log b_1.log c_1.log d_1.log b_2.log c_2.log d_2.log e_m1.log a_m2.log e_m2.log I need to keep latest 10 instances of each file. I can write multiple find commands but looking if it is possible in one line. m file are monthly... (4 Replies)
Discussion started by: wahi80
4 Replies
funsky(1)							SAORD Documentation							 funsky(1)

NAME
funsky - convert between image and sky coordinates SYNOPSIS
funsky iname[ext] # RA,Dec (deg) or image pix from stdin funsky iname[ext] [lname] # RA, Dec (deg) or image pix from list funsky iname[ext] [col1] [col2] # named cols:units from stdin funsky iname[ext] [lname] [col1] [col2] # named cols:units from list OPTIONS
-d # always use integer tlmin conversion (as ds9 does) -r # convert x,y to RA,Dec (default: convert RA,Dec to x,y) -o # include offset from the nominal target position (in arcsec) -v # display input values also (default: display output only) -T # output display in rdb format (w/header,tab delimiters) DESCRIPTION
Funsky converts input sky coordinates (RA, Dec) to image coordinates (or vice versa) using the WCS information contained in the specified FITS file. Several calling sequences are supported in order to make it easy to specify coordinate positions in different ways. The first required argument is always the input FITS file (or extension) containing the WCS information in an extension header. Note that the data from this file is not used. By default, the program converts input RA and Dec values to X and Y using this WCS information. If the WCS is associated with a FITS image, then the X,Y values are image values. If the WCS is associated with a binary table, then the X, Y val- ues are physical values. To convert X,Y to RA and Dec, use the -r (reverse) switch. If no other command arguments are supplied, then the input positions are read from the standard input. Each line is assumed to contain a single coordinate position consisting of an RA in hours (or X in pixels) followed by a Dec in degrees (or Y in pixels). The usual delim- iters are supported (spaces, commas, tabs). For example: # read from stdin, default column names and units [sh] funsky snr.ev 22.982695 58.606523 # input RA (hrs), Dec(deg) 510.00 510.00 22.982127 58.607634 # input 512.00 510.50 22.981700 58.614301 # input 513.50 513.50 ^D # end of input If a second argument is supplied, this argument is assumed to be a file containing RA (X) and Dec (Y) positions. The file can either be an ASCII table or a FITS binary table. The order of columns is unimportant, if the table has a column header. In this case, the names of the columns must be one of "RA", "DEC", or "X", "Y" for sky to image and image to sky conversions, respectively. If the table has no header, then once again, RA (X) is assumed to first, followed by DEC (Y). For example: # read from file, default column names and units [sh] cat hd.in RA DEC --------- --------- 22.982695 58.606523 22.982127 58.607634 22.981700 58.614301 [sh] funsky snr.ev hd.in 510.00 510.00 512.00 510.50 513.50 513.50 If three arguments are supplied, then the input positions again are read from the standard input. Each line is assumed to contain a single coordinate position consisting of an RA (or X in pixels) followed by a Dec (or Y in pixels), with the usual delimiters supported. However, the second and third arguments now specify the column names and/or sky units using a colon-delimited syntax: [colname]:[h|d|r] If the colname is omitted, the names default to "RA", "DEC", "X", "Y", "COL1", or "COL2" as above. If the units are omitted, the default is hours for RA and degrees for Dec. When the -r switch is used (convert from image to sky) the units are applied to the output instead of the input. The following examples will serve to illustrate the options: # read from stdin, specifying column names (def. units: hours, degrees) [sh] cat hd.in MYRA MYDEC --------- --------- 22.982695 58.606523 22.982127 58.607634 22.981700 58.614301 [sh] funsky snr.ev MYRA MYDEC < hd.in 510.00 510.00 512.00 510.50 513.50 513.50 # read from stdin, specifying column names and units [sh] cat dd.in MYRA MYDEC --------- --------- 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 [sh] funsky snr.ev MYRA:d MYDEC:d < dd.in 510.00 510.00 512.00 510.50 513.50 513.50 # read stdin, convert image to sky, specifying output sky units [sh] cat im.in 510.00 510.00 512.00 510.50 513.50 513.50 [sh] cat im.in | funsky -r snr.ev :d :d 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 Finally, four command arguments specify both and input file and column names and/or units: [sh] cat dd.in MYRA MYDEC --------- --------- 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 [sh] funsky snr.ev dd.in MYRA:d MYDEC:d 510.00 510.00 512.00 510.50 513.50 513.50 # read file, convert image to sky, specifying output sky units [sh] cat im.in 510.00 510.00 512.00 510.50 513.50 513.50 [sh] funsky -r snr.ev im.in :d :d 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 By default, the output of funsky consists only of the converted coordinate position(s), one per output line. This makes parsing in shell scripts easy. Use the -v (verbose) switch to specify that the input coordinates should be pre-pended to each line. For example: [sh] cat dd.in MYRA MYDEC --------- --------- 344.740432 58.606523 344.731900 58.607634 344.725500 58.614301 [sh] funsky snr.ev dd.in MYRA:d MYDEC:d 510.00 510.00 512.00 510.50 513.50 513.50 [sh] funsky -v snr.ev dd.in MYRA:d MYDEC:d 344.740432 58.606523 510.00 510.00 344.731900 58.607634 512.00 510.50 344.725500 58.614301 513.50 513.50 In addition, a full starbase table can be output using the -T (table) switch. This switch can be used with or without the -v switch. If the -T and -v are both specified, then a descriptive header parameters are output before the table (mainly to remind you of the sky units): # output table in non-verbose mode [sh] funsky -T snr.ev dd.in MYRA:d MYDEC:d X Y ------------ ------------ 510.00 510.00 512.00 510.50 513.50 513.50 # output table in verbose mode [sh] funsky -T -v snr.ev dd.in MYRA:d MYDEC:d # IFILE = /Users/eric/data/snr.ev # ICOL1 = MYRA # ICOL2 = MYDEC # IUNITS1 = d # IUNITS2 = d # OCOL1 = X # OCOL2 = Y MYRA MYDEC X Y ------------ ------------ ------------ ------------ 344.740432 58.606523 510.00 510.00 344.731900 58.607634 512.00 510.50 344.725500 58.614301 513.50 513.50 Finally, the -d (ds9) switch mimicks ds9's use of integer TLMIN and TLMAX values for all coordinate transformations. FITS conventions seem to call for use of floating point TLMIN and TLMAX when the data are floats. This convention is followed by funsky but results in a small discrepancy with ds9's converted values for floating point data. We will remedy this conflict in the future, maybe. SEE ALSO
See funtools(7) for a list of Funtools help pages version 1.4.2 January 2, 2008 funsky(1)
All times are GMT -4. The time now is 06:12 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy