Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

File Move & Sort by Name - Kick out Bad File Names & More

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File Move & Sort by Name - Kick out Bad File Names & More
# 1  
Old 09-19-2016
File Move & Sort by Name - Kick out Bad File Names & More
I have a dilemma, we have users who are copying files to "directory 1." These images have file names which include the year it was taken. I need to put together a script to do the following:
  1. Examine the file naming convention, ensuring it's the proper format (e.g. test-1983_filename-123.tif) Ensuring the naming convention is in the correct format, doesn't include incorrect characters, filename out of order, etc. (this is in case a user adds a file/image with the incorrect naming convention)
  2. Rejects any images that are incorrectly named.
  3. Determine what year is in the filename (e.g. 1983)
  4. Search a different filesystem/location to see if that current directory exist (e.g. 1983 - year directory)
    1. If so, copying file to that location.
    2. If not, create directory (directory named year, e.g. 1983), then copy file to that location.


"Directory 1" will potentially have 1000's of files to process and sort.


I currently have a bash script that determines whether files exist within "directory 1," places them into a directory by today's date, then moves them to a different filesystem/location. I need the script to vet each filename, and move it to the respective directory named by the year. I'm most uncertain about how to vet the filenames. Any assistance is greatly appreciated!
Last edited by rbatte1; 09-19-2016 at 11:12 AM.. Reason: Converted text list into formatted numbered list with sub-lettered list and correcting spelling
# 2  
Old 09-19-2016
More specifically, to allow us to help you to create regular expression syntax and a script:
Quote:
Code:
1. what pattern elements are required - examples
    A. years: 1980 - 2016? i.e. is 1950 valid?
    B. file type the '.something' string at the end - .tif?
2. Does the create date of the file have to match the date in the file name?
3. Do you really want to delete or simply mv  a bad filename, rather than make it name-conforming?

Last edited by jim mcnamara; 09-19-2016 at 11:04 AM..
This User Gave Thanks to jim mcnamara For This Post:
Nvizn
# 3  
Old 09-19-2016
Quote:
Originally Posted by jim mcnamara
More specifically, to allow us to help you to create regular expression syntax and a script:
Absolutely! Thanks for your response.
    1. Years that would be valid = 1900-2000's and going forward.
    2. All files will have .tif file extension. Not valid, otherwise.
  2. Creation date doesn't matter. The date within the file name (e.g. name-2001_filename-123.tif) will be named by a human. This is where the script must check and confirm the naming convention formatting, is correct.

    Essentially, it will contain a year and some sort of file naming convention indicative to the image. Here's the format:

    locationabbreviation-year-orgnumber_imagenumber.tif

    examples:
    bldg1-1996-org101_F-12345.tif
    bldgzone-2001-org234_ZTW-12345.tif
    .
  3. Not delete bad file name, but move to a "bad file" directory or so. If the file doesn't follow this formatting, its kicked out to the "bad file" directory.
Last edited by rbatte1; 09-19-2016 at 12:53 PM.. Reason: Added CODE tags and formatted lists rather than text lists
# 4  
Old 09-19-2016
How about this as a starting point (needs a recent shell, e.g. bash):
Code:
ls *.tif | while read FN; do [[ "$FN" =~ [[:alnum:]]+-[[:digit:]]{4}-[[:alnum:]]+_[[:upper:]]+-[[:digit:]]+.tif ]] && echo OK $FN || echo NOK; done

Please note that the test file name in your first post would fail this check.
# 5  
Old 09-20-2016
Can you tell us what you mean by Rejects any images that are incorrectly named perhaps by providing examples of bad filenames...
# 6  
Old 09-21-2016
Quote:
Originally Posted by shamrock
Can you tell us what you mean by Rejects any images that are incorrectly named perhaps by providing examples of bad filenames...
Absolutely! I think RudiC may be on to something.

A bad filename could include anything that doesn't follow the naming schema.

Again, naming schema:

Code:
locationabbreviation-year-orgnumber_imagenumber.tif


Error examples:
  1. Any of these, out of order.
  2. Spaces within the name.
  3. Special characters. ( examples: / * ? & % #)
  4. A non-valid year, e.g. 1890. Any not exclusive to 1900's+.
  5. It could also be missing one of these 4 categories, e.g. included locationabbrev, year & image number, but left out org number.


Does this provide any clarity on the question?
Last edited by rbatte1; 09-21-2016 at 12:20 PM.. Reason: Converted to formatted number-list
# 7  
Old 09-21-2016
Quote:
Originally Posted by Nvizn
Absolutely! I think RudiC may be on to something.
.
.
.
To prevent misunderstandings - YOU should be on to something. With the starters given, you could test and adapt and fine tune them until suiting your needs, or come back here if you can't get to a satisfying solution. Also, the final solution could be posted so people can see/learn from them for future questions.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Want to sort a file using awk & sed to get required output

Hi All, Need Suggestion, Want to sort a file using awk & sed to get required, output as below, such that each LUN shows correct WWPN and FA port Numbers correctly: Required output: 01FB 10000000c97843a2 8C 0 01FB 10000000c96fb279 9C 0 22AF 10000000c97843a2 8C 0 22AF 10000000c975adbd ... (10 Replies)
Discussion started by: aix_admin_007
10 Replies

2. Shell Programming and Scripting

Sort a the file & refine data column & row format

cat file1.txt field1 "user1": field2:"data-cde" field3:"data-pqr" field4:"data-mno" field1 "user1": field2:"data-dcb" field3:"data-mxz" field4:"data-zul" field1 "user2": field2:"data-cqz" field3:"data-xoq" field4:"data-pos" Now i need to have the date like below. i have just... (7 Replies)
Discussion started by: ckaramsetty
7 Replies

3. Shell Programming and Scripting

Archive & move the file

I have a set of files in path /ifx01/etldata/lmt/ a.xml b.xml I have to archive & move these files to path /ifx01/etldata/archive ,so that not even the original files should exists. After acrhiving there should be no files in path /ifx01/etldata/lmt/ (6 Replies)
Discussion started by: jagadeeshn04
6 Replies

4. Shell Programming and Scripting

replace & with & xml file

Hello All I have a xml file with many sets of records like this <mytag>mydata</mytag> <tag2>data&</tag2> also same file can be like this <mytag>mydata</mytag> <tag2>data&</tag2> <tag3>data2&amp;data3</tag3> Now i can grep & and replace with &amp; for whole file but it will replace all... (4 Replies)
Discussion started by: lokaish23
4 Replies

5. UNIX for Dummies Questions & Answers

Search for & edit rows & columns in data file and pipe

Dear unix gurus, I have a data file with header information about a subject and also 3 columns of n rows of data on various items he owns. The data file looks something like this: adam peter blah blah blah blah blah blah car 01 30 200 02 31 400 03 57 121 .. .. .. .. .. .. n y... (8 Replies)
Discussion started by: tintin72
8 Replies

6. UNIX for Dummies Questions & Answers

WebDav/davfs mounted file & directory names in all UPPERCASE

Hey, I have a WebDav directory mounted and everything seems fine except for one thing. All file/directory names appear in all UPPERCASE, when in actual fact they are lowercase on the remote machine. For example: foo/bar/baz.html on the remote host, appears on my local machine as... (0 Replies)
Discussion started by: MrMoney
0 Replies

7. Shell Programming and Scripting

cp && rm command, rm: <file> not removed. No such file or directory

Hi, I am running this is korn shell cp $source/$fname $dest/dir && rm $source/$fname This was returned: rm: /dir/file not removed: No such file or directory The file could be found in the $dest directory which meant the cp was success. The above code is used in a for loop to move... (5 Replies)
Discussion started by: Leion
5 Replies

8. Shell Programming and Scripting

Sort & Split records in a file

Hi, I am new to scripting. I need a script to sort and the records in a file and then split them into different files. For example, the file is: H1...................... H2...................... D2.................... D2.................... H1........................... (15 Replies)
Discussion started by: Sunitha_edi82
15 Replies

9. Shell Programming and Scripting

Finding & Moving Oldest File by Parsing/Sorting Date Info in File Names

I'm trying to write a script that will look in an /exports folder for the oldest export file and move it to a /staging folder. "Oldest" in this case is actually determined by date information embedded in the file names themselves. Also, the script should only move a file from /exports to... (6 Replies)
Discussion started by: nikosey
6 Replies

10. Shell Programming and Scripting

File Compare & Move between 2 servers

Greetings - I am a newbie in shell scripts. I have been thru the whole forum but there has been no similar query posed. The objective of my system is to have a unified filebase system. I am using RSync to synchronise files between the location & central server with both of them having the... (4 Replies)
Discussion started by: evolve
4 Replies
Login or Register to Ask a Question