Sort and move multiple files by modification date


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sort and move multiple files by modification date
# 1  
Old 11-30-2014
Sort and move multiple files by modification date

Hi I have a problem, I have a large group of archive files in a folder some are later versions of the same archive, the only difference btween them is that the archiving program we use appends the name with a code for it to keep track of in its data base, and the modification date.
I am starting to accumulate a large number of duplicate archive files am trying to find a way to isolate the files by name and move only the latest version to a new folder.
If any one has any ideas how I might accomplish this I would greatly appreciate if you could point me in the right direction.
I had thought that I could use list in some way and create a command from that with sed to move the correct files to another folder.
Thank you very much
# 2  
Old 11-30-2014
Can you sort the files by it creation date? old files should be older,
can you place here an example to the files names (old+new)

Igal
# 3  
Old 11-30-2014
And, tell us what OS and shell you're using...
and, show us what you have tried so far to solve your problem...
and, tell us what the pathname is for you new folder...
and, explain why you think sed would be a good tool to identify the files you want to move?
# 4  
Old 12-01-2014
Hi again, I apologize for just getting back to you now.
The shell I am using is the Bash shell and the bsd version of sed, I'm on a mac.
An example of the file name I have to work with is;
20113_Zebra_WindowCat2014_93f7842-d959-46f1-b1f6-af652fc0f57.bkf
20113_Zebra_WindowCat2014_8015911e-0d8d-4dcf-b2d5-45733d7adb6e.bkf
The part of the string after the 2014 portion is added by the archive program we use to keep track of the latest version in its database, there is no logic that I can see for the string that the archive program adds.
I was thinking that I could list the the directory where the archives reside by date to a text file with the most recent files listing first.
I could use sed to eliminate duplicate lines so that only the most recent entry would be left in the list.
With sed I can reformat the list to a command to move the most recent files I want to a temporary directory while I clean up the original archive directory. Once the original archive directory is cleaned I can move only the most recent archive files back.
I think I'm a little clearer on what I need to do now that I have explained it you though. If you know of a different approach to solve this problem please let me know I would greatly appreciate your input.
Thanks again
# 5  
Old 12-01-2014
Not highly efficient, but pretty simple...
Code:
ls -t | awk -F_ '/.bkf*/ && !s[$1 FS $2 FS $3]++' | while read f
do	echo mv "$f" dest_dir
done

Change dest_dir to the pathname of your destination directory. Remove the echo shown in red and rerun the script to actually move the files if the mv commands it shows you look right.

If you want to try this on a Solaris/SunOS system, change awk to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk.
# 6  
Old 12-01-2014
Hi again, the script works like a charm, It really is a thing of beauty.
I am so grateful for your generousity with your time and talents.
Thanks so very much, you and Unix.com are the very best!!
Merry Christmas
# 7  
Old 12-01-2014
I'm glad my suggested script is working for you.

Could you please explain how you would use sed to eliminate related files? I don't normally think of sed as an appropriate tool for that type of operation (especially when the files to be skipped over are not necessarily contiguous with the file that is to be kept).
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

2. Shell Programming and Scripting

Move files from one date to another

Hi I have hundreds of files with the following format I need to move this 1000 files from above to a different date Is there a way to do that with a script or command (3 Replies)
Discussion started by: dsravanam
3 Replies

3. UNIX for Dummies Questions & Answers

Move Multiple Files adding date timestamp before file type

There are files in a directory and I have to move multiple files adding datetimestamp before the file type. /Data/ abc.csv def.csv ghi.csv I have to move this files to archive directory adding datatimestamp before .csv /archive/ abc_YYYYMMDDHHMMSS.csv def_YYYYMMDDHHMMSS.csv... (7 Replies)
Discussion started by: eskay
7 Replies

4. Shell Programming and Scripting

Move all files except sys date (today) files in Solaris 10

I want to move all files from one directory to another directory excluding today (sysdate files) on daily basis. file name is in pattern file_2013031801, file_2013031802 etc (2 Replies)
Discussion started by: khattak
2 Replies

5. Shell Programming and Scripting

Move files from one directory to another based on creation/modification date

Hi All, Really stuck up with a requirement where I need to move a file (Lets say date_Employee.txt--the date will have different date values like 20120612/20120613 etc) from one directory to another based on creation/modification dates. While visiting couple of posts, i could see we can... (3 Replies)
Discussion started by: dsfreddie
3 Replies

6. Shell Programming and Scripting

Rename old files with last modification date

Hi everyone, I have files like file1_Mod.txt, file2_Mod.txt. I want to rename the old files with the last modification date. I write the below script to rename with current date, but I donīt know how to use "date -r" to get the last modification date with the same format I have below... (5 Replies)
Discussion started by: cgkmal
5 Replies

7. UNIX for Advanced & Expert Users

Help with sorting files according to modification date

Hi, I was very surprised to not be able to find an answer to this question despite my best efforts in Google and elsewhere. Maybe it's a good thing as it forced me to finally become a member in this great forum that i use frequently. Ok my question: I want to be able to sort files inside a... (3 Replies)
Discussion started by: stavros
3 Replies

8. Shell Programming and Scripting

List the file or files with last modification date

hi. I need help my programing friends :p I need to list all the files with a certain name (for example FileName) by last modification date but only the one with the last date. If there are two files with the same name and same modification date it should print the both. For example in this set... (6 Replies)
Discussion started by: KitFisto
6 Replies

9. Shell Programming and Scripting

Copy files based on modification date

How to copy files from a location to a directory <YYMM> based on last modification date? This will need to run daily. I want to copy those file for May to 0905 and Jun to 0906. Appreciate your guidance.:) Thanks. -rw-rw-rw- 1 ttusr tgrp 4514 May 29 21:49 AB24279J.lot_a... (17 Replies)
Discussion started by: KhawHL
17 Replies

10. Shell Programming and Scripting

Sort and move files

Here is the situation: I have three folders: a. Storage ( have 50 files for processing) b. Process (should move ONLY ONE file for processing from storage) c. Backup (processed files should be moved here from Process folder) can you help me out to wirte a shell script please. Process: From... (1 Reply)
Discussion started by: subs2me
1 Replies
Login or Register to Ask a Question