Bash folder manipulation - selecting/copying specified files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash folder manipulation - selecting/copying specified files
# 1  
Old 06-01-2010
Bash folder manipulation - selecting/copying specified files

Bash/scripting newbie here - I feel this might be a trivial problem, but I'm not sure how to tackle it. I've got a folder of a year's worth of files, with some random number of files generated every day of the year (but at least one per day). I'm writing a script to automatically grab the file with the earliest daily timestamp and copy it to another folder. For example, if three files were generated on Jan 1st at 01:15, 01:30, and 01:50, I want to copy the one timestamped 01:15 into another folder and move on to Jan 2nd.

I've figured out the very basic bash commands to list files in a directory and copy them, but not how to read those filenames & their timestamps into a variable so I can perform operations on them, or how to go back from the variable to actually copy the files (in my other programming life, mostly using Java, file I/O is enough of a general mess that I'm making a foray into scripting ...). Any suggestions?

Thanks!
-Goose
# 2  
Old 06-01-2010
When you do
Code:
ls -lrt  *

does the order of the files come out matching the order of the timestamps i.e.,
file_1:15
file_1:30

and so on. It should if the timestamps in the filename mean anything. Please show an example ls -lrt output for a few days worth of files.
# 3  
Old 06-01-2010
Do you want to copy the files with earliest creation date or with earliest modification date?
Can you post one line of ls -l output?
# 4  
Old 06-01-2010
Here's the ls -l output for 10 files over 6 days. I'm just interested in the timestamp in the filename, which doesn't correspond much if at all to the file modification date. So I guess I need to figure out some way of parsing out the date (ie, 2009-01-01) and the time (ie, 12-09-44-58) and comparing them - and I think I have examples of how to do that in other scripting languages (R & perl). But I'm still not sure how to go from the output of something like that back to telling the file system which actual file to copy to a different directory.

Quote:
-rwxrwxrwx 1 root root 67007 2009-01-01 12:21 bartlett_2009-01-01_12-09-44-58.jpg
-rwxrwxrwx 1 root root 71627 2009-01-02 12:06 bartlett_2009-01-02_12-00-31-84.jpg
-rwxrwxrwx 1 root root 70099 2009-01-02 12:21 bartlett_2009-01-02_12-10-31-71.jpg
-rwxrwxrwx 1 root root 60966 2009-01-03 12:21 bartlett_2009-01-02_12-15-33-89.jpg
-rwxrwxrwx 1 root root 75890 2009-01-04 12:06 bartlett_2009-01-03_12-00-35-25.jpg
-rwxrwxrwx 1 root root 76765 2009-01-04 12:21 bartlett_2009-01-03_12-10-35-13.jpg
-rwxrwxrwx 1 root root 73274 2009-01-05 12:21 bartlett_2009-01-04_12-10-34-02.jpg
-rwxrwxrwx 1 root root 76386 2009-01-06 12:06 bartlett_2009-01-05_12-00-35-73.jpg
-rwxrwxrwx 1 root root 78089 2009-01-06 12:21 bartlett_2009-01-05_12-10-35-59.jpg
-rwxrwxrwx 1 root root 41768 2009-01-07 12:21 bartlett_2009-01-06_12-10-37-98.jpg
Thanks!
# 5  
Old 06-05-2010
Code:
ls -l | awk '!a[$6]++' | while read line; do cp $(echo "$line" | awk '{print $8}') /target/dir; done

# 6  
Old 06-08-2010
I ended up using
Code:
date=$(echo $n | cut -f 2-4 -d '_')

to extract the date from the filename, storing it as the previous date, then comparing the next file date to it to determine if they differed. It'll be a while before I can come up with nice one-liners like pseudocoder's, but I'm pretty happy to have my first running bash script! The archives on here helped a lot.

Last edited by WildGooseChased; 06-08-2010 at 11:02 AM.. Reason: my horrible grammer
# 7  
Old 06-08-2010
Quote:
Originally Posted by WildGooseChased
I'm just interested in the timestamp in the filename, which doesn't correspond much if at all to the file modification date.
Code:
awk '{split($NF,a,"_");if(a[2]!=$6){print "cp",$NF,"/dir/"}}' file | sh

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash: copying lines with specific character to files with same name as copied line.

I am trying to make my script as simple as a possible but, I am not sure if the way I am approaching is necessarily the most efficient or effective it can be. What I am mainly trying to fix is a for loop to remove a string from the specified files and within this loop I am trying to copy the lines... (2 Replies)
Discussion started by: Allie_gastrator
2 Replies

2. Shell Programming and Scripting

Need help in finding and copying list of files using bash shell script

Dear All, I have a situation where I want to copy some files of type .txt. These files are o/p from one program. Some of the files are named as fileName .txt instead of fileName.txt after fileName by mistake I have specified "space". Now I want to move these files as follows. mv fileName*... (13 Replies)
Discussion started by: linuxUser_
13 Replies

3. Shell Programming and Scripting

Copying files from various folders to similar folder structure in another location

Hi, I need to write a script the has to copy the files from folders and subfolders to the same folder structure located in another location. Ex: mainfolder1 file1,file2,file3 subfolder1(file1,etc) subfolder2(file1,etc) to another folder location of same folder structure. rsync is not... (7 Replies)
Discussion started by: Raji Perumal
7 Replies

4. Shell Programming and Scripting

Copying large files in a bash script stops execution

Hello, I'm new to this forum and like to first of all say hello to everyone. I've got a really annoying problem at the moment. I'm trying to rsync some files (about 200MB with one file of 120MB) from a Raspberry PI with raspbian to a debian server via rsync. This procedure is stored in a... (3 Replies)
Discussion started by: wex_storm
3 Replies

5. Programming

Selecting files in regular intervals from a folder

Hi, I need your expertise in selecting files from a folder. I have files named with convention: filename.i.j where j is an interger from 1 to 16, for each i which is an integer from 1 to 2000. I would like to select the files with i in regular interval of 50 like filename.1.j,... (2 Replies)
Discussion started by: rpd25
2 Replies

6. Shell Programming and Scripting

copying random Jpg files to different folder

Hi, I have 200 pictures in a folder and I would like move 10 random pictures every week to given folder automatically. I have this server on 1and1.com. So I tried the following using Bash script for manual copy and paste for testing #!/bin/bash mapfile -t -n 3 files < <(find... (13 Replies)
Discussion started by: raamkum
13 Replies

7. OS X (Apple)

Automated command ; extracting files from folders and copying them into a single folder

Hello everyone, I'm running Mac OS X Leopard (10.5.8) and I want to use the Terminal to help automate this tedious and laborious command for me: I need to extract all of the .m4p files in my "iTunes Music" folder which reside in folders of the artist, and then subfolders for the albums and... (2 Replies)
Discussion started by: qcom
2 Replies

8. Shell Programming and Scripting

bash script for selecting grib files with cdo

hi, i am new with scripts and have a little problem. i have a file with dates in the form YYYYMMDD, in average about 40 days per year. now i shall cut those days with cdo selday from monthly files. the script shall look up the days, forward them to the cdo operator and write the new grib... (1 Reply)
Discussion started by: jumkae
1 Replies

9. Shell Programming and Scripting

script for Finding files in a folder and copying to another folder

Hi all, I have a folder '/samplefolder' in which i have some files like data0.txt, data1.txt and data2.txt. I have to search the folder for existence of the file data0.txt first and if found have to copy it to some other file; next i have to search the folder for existence of file... (5 Replies)
Discussion started by: satish2712
5 Replies
Login or Register to Ask a Question