How to copy files from one location to another based on a priority?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to copy files from one location to another based on a priority?
# 1  
Old 10-16-2013
How to copy files from one location to another based on a priority?

Hi Gurus,

I am a newbie to shell scripting and I am facing a problem right now.I have to automate the copy of files based on a priority.The scenario is as below:

1) There will be files from Mon-Fri with Mon file being named as abc_def_01_YYYYMMDD and Tue file being abc_def_02_YYYYMMDD and so forth till Friday's file which would be abc_def_YYYYMMDD.

2) I have to set a priority such that if on any given day if i have more than one file, I have to copy the earlier file first.For instance if on Thursday i have monday's file and wednesday's file I have to copy the Monday's file first, then after a pre-defined delay send Wednesday's file-delay- and then send the thursday's file.

Please let me know how could i do it..Thanks in anticipation.
# 2  
Old 10-16-2013
Quote:
Originally Posted by vikramgk9
if on any given day if i have more than one file
From this I assume that the source files are being removed periodically by something? i.e. you won't have old files hanging around that you've already processed?
# 3  
Old 10-16-2013
Yes Carlo.Thanks for the quick response.The old files are being periodically removed.Once the file is copied is to the target location, they should be deleted.
Further, the file is the trigger to the script.
For some reason if the four files do not come on mon-thu and on fri if all 5 files come at once they should be sent based on the priority described above.
can this be done please?

Last edited by vikramgk9; 10-16-2013 at 04:16 PM..
# 4  
Old 10-16-2013
1. Is it a typo that Friday's file is missing the 05 i.e. should abc_def_YYYYMMDD above be abc_def_05_YYYYMMDD

2. if the files are sorted by the YYYYMMDD part of the file name will this do the trick in getting the order right.


Say your filenames are like this:

Code:
221_170_01_20131014
rtn_17_02_20131015
trm_01_03_20131016
xxx_yyy_04_20131017
abc_def_05_20131018

You could try doing something like this:

Code:
LOOK_DIR=/var/spool/dumpdir
PROC_DIR=/var/spool/ready
WAIT=120

ls $LOOK_DIR | sort -t_ -k 4 | while read file
do
    cp "$LOOK_DIR/$file" "$PROC_DIR/$file"
    sleep $WAIT
done

This User Gave Thanks to Chubler_XL For This Post:
# 5  
Old 10-19-2013
Hi Chubler! Thanks a lot! It worked perfectly!! Smilie Smilie
Yes it was a typo.The friday's file shoudl have been abc_def_05_YYYYMMDD.

Can you please explain me:

Code:
ls $LOOK_DIR | sort -t_ -k 4 | while read file
do
    cp "$LOOK_DIR/$file" "$PROC_DIR/$file"
    sleep $WAIT
done

Again! Thanks a ton! Smilie Smilie

Last edited by vikramgk9; 10-19-2013 at 03:59 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy data at specified location from multiple files

Hello everyone, Im super new to coding but increasingly in need of it at work. Im have task stacked because of this problems, that I cannot figure out how to solve looking on the internet after trying many many things that looked similar to me. I have multiple data files of the form (see below).... (2 Replies)
Discussion started by: Xfiles_fan
2 Replies

2. Shell Programming and Scripting

Copy files based on specific word in a file name & its extension and putting it in required location

Hello All, Since i'm relatively new in shell script need your guidance. I'm copying files manually based on a specific word in a file name and its extension and then moving it into some destination folder. so if filename contains hyr word and it has .md and .db extension; it will move to TUM/HYR... (13 Replies)
Discussion started by: prajaktaraut
13 Replies

3. Shell Programming and Scripting

Use find with cp and sed in ksh to copy files to a slightly different location

Hello there wonderful people, I am running on Solaris 10 and with the following ksh version: strings /bin/ksh | grep Version | tail -2 @(#)Version M-11/16/88i Suppose I want to copy files that end in _v2 from underneath /dir1/dir2/save directory to /dir1/dir2. Basically, what I’m... (12 Replies)
Discussion started by: ejianu
12 Replies

4. Shell Programming and Scripting

How to copy files from one location to another using xargs??

Hello Experts, I need to copy files from one location to another using xargs. Tried something like this (In Ubuntu & Solaris ). mkdir -p 1234; find /home/emd/Desktop/n007/M007/ -type f -name "A2014*" | xargs -0 cp -r {} /home/emd/Desktop/1234 But every time i run this, a weird error... (6 Replies)
Discussion started by: Saidul
6 Replies

5. Shell Programming and Scripting

Consolidate files based on priority

please help, I would like to merge 4 files, all of them same format. The first col is the key for all files,followed by variable number of fields so I want to check if the key is present in the files according to priority and then also add a column saying it came from which file . So the... (4 Replies)
Discussion started by: ritakadm
4 Replies

6. Shell Programming and Scripting

Copy files from one location to another

I have below files in one location /test/files and also for each dates there are similar files A20130924.0000-0005_file1 A20130924.0000-0005_file2 A20130924.0005-0010_file1 A20130924.0005-0010_file2 . . . A20130924.2355-0000_file1 A20130924.2355-0000_file2 If i execute the script like... (4 Replies)
Discussion started by: Saidul
4 Replies

7. UNIX for Dummies Questions & Answers

how to copy files and record original file location?

:EDIT: I think my post name should have been labeled: how to copy files and record original file location. not "retain". Hello, this is my first post! I searched the forums a lot before posting, but was unable to answer my question. Here's my problem: There are several hundred text files... (4 Replies)
Discussion started by: willie8605
4 Replies

8. Shell Programming and Scripting

Copy files in thumbnail folder to a secondary location for Amazon S3

Hello all! I am trying to create a script that will copy files from one location, to another but only folders that are filled with thumbnails to an exact directory replica in the second location. For example: /images/2012/01/19/Event/Photographer/thumbnails to ... (4 Replies)
Discussion started by: Buzzman25
4 Replies

9. Shell Programming and Scripting

Shell Script for Copy files from one location to another location

Create a script that copies files from one specified directory to another specified directory, in the order they were created in the original directory between specified times. Copy the files at a specified interval. (2 Replies)
Discussion started by: allways4u21
2 Replies

10. UNIX for Advanced & Expert Users

copy files from one location to similar location

I need help in forming a script to copy files from one location which has a sub directory structure to another location with similar sub directory structure, say location 1, /home/rick/tmp_files/1-12/00-25/ here 1-12 are the number of sub directories under tmp_files and 00-25 are sub... (1 Reply)
Discussion started by: pharos467
1 Replies
Login or Register to Ask a Question