Sorting and moving file sequence with gaps


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sorting and moving file sequence with gaps
# 1  
Old 06-28-2011
Sorting and moving file sequence with gaps

Hello,

I have lots of sequentially numbered files which make up an image sequence.

I'm trying to do two things with it:

#1: Find gaps in the sequence and move each range of sequencial files into their own subfolder.

#2: Designate a starting point (file) and move every 24th file into another subfolder.

The files are named like this:

AAAA_000001
AAAA_000002
... (continuous)
AAAA_120001
AAAA_120002
AAAA_125000 (sequence break)
AAAA_125001
... (continuous)
AAAA_150001
AAAA_150002
etc...

I figure the best is to put it into two scripts, but I'm can't find a good starting point for either task. I don't know how to identify the range of the sequence so I can make a list to move the files.

Any ideas?


Many thanks
# 2  
Old 06-28-2011
I dont understand your requirements...so try to be clear and post sample input and output.
# 3  
Old 06-28-2011
Plus, what if the missing file is the last? Is there a clue in the first of the #? What if #1 is missing? Generic solution: any is clue to #1, #1 is clue to all. Start with every prefix "find . . . | sed '. . .' | sort -u | while read prefix, do, look for ${prefix}00001, get clue, look for all the children, done".
# 4  
Old 06-28-2011
The missing files are not so important, in fact they are not really missing.

Let me try another way of explaining it, and let me focus on just the sequence organization.

I have a folder with the following file sequence ranges:
/media/images/
AAAA_[0000-1000].jpg
AAAA_[2000-3000].jpg
AAAA_[4000-5000].jpg
AAAA_[6000-8000].jpg

So 4 groups total, only I don't know how long they are, what number they start at, or how many digits the numbers have. The prefix and extension are set.

I want to create folders using the first file number in sequence:

/media/images/AAAA_0000
/media/images/AAAA_2000
/media/images/AAAA_4000
/media/images/AAAA_6000

and move each sequence into its folder.

Is this more clear?
# 5  
Old 06-28-2011
Sure, ksh/bash not sed can do same:
Code:
find ... -name AAAA_*00.jpg | while read f
do
 mkdir ${f%.jpg}
 mv ${f%00.jpg}*.jpg ${f%.jpg}/
done

More speed?
Code:
find ... -name AAAA_*00.jpg | sed 's/\.jpg$//' | xargs -n999 mkdir
find ... -name AAAA_*00.jpg | sed 's/\(\(.*\)00\)\.jpg$/mv \2*.jpg \1/' | sh

Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Gaps and frequencies

I have this infile: >GHL8OVD01BNNCA Freq 10 TAGATGTGCCCGTGGGTTTCCCGTCAACACCGGATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTCGTGGGTTTCCCGTCAACACCGGCAAATAGT-GCAGCA-TA >GHL8OVD01CMQVT Freq 1 TTGATGTGCCAGTTTCCCGTCTAGCAGCACTACCAGGACCTTCGC-TA >GHL8OVD01CMQVW Freq 1... (1 Reply)
Discussion started by: Xterra
1 Replies

2. Shell Programming and Scripting

Adding gaps to a string in bash

I have the following string, and want to introduce additional spaces between the two %s. This will be done by specifying the gap between the %s. Example having gap=8 will put 8 spaces between the two %s. frmt_k1d1_test="%s %s\n" I am doing the script in bash. ---------- Post updated at... (4 Replies)
Discussion started by: kristinu
4 Replies

3. Shell Programming and Scripting

find common entries and match the number with long sequence and cut that sequence in output

Hi all, I have a file like this ID 3BP5L_HUMAN Reviewed; 393 AA. AC Q7L8J4; Q96FI5; Q9BQH8; Q9C0E3; DT 05-FEB-2008, integrated into UniProtKB/Swiss-Prot. DT 05-JUL-2004, sequence version 1. DT 05-SEP-2012, entry version 71. FT COILED 59 140 ... (1 Reply)
Discussion started by: manigrover
1 Replies

4. Shell Programming and Scripting

Searching for Gaps in Time

I am very new to shell scripting. We use C-Shell here and I know the issues that surround it. I hope a solution can be created using awk, sed, etc... instead of having to write a program. I have an input file that is sorted by date and time in ascending order ... (2 Replies)
Discussion started by: jclanc8
2 Replies

5. UNIX for Advanced & Expert Users

help with sorting sequence in Unix C:sort -t ':' +0 -1 -n +1 -2 +2 -3 -o list list

Hi List is 000|2008-07-17|556543|RTJ|35-RTGJ|EYT 465|2008-11-10|567789|GHJ|45-DGHH|ETU 533|2008-09-06|567789|GHJ|45-DGHH|ETU How does it do it? sort -t ':' +0 -1 -n +1 -2 +2 -3 -o list list (6 Replies)
Discussion started by: gurvinder
6 Replies

6. Shell Programming and Scripting

Finding & Moving Oldest File by Parsing/Sorting Date Info in File Names

I'm trying to write a script that will look in an /exports folder for the oldest export file and move it to a /staging folder. "Oldest" in this case is actually determined by date information embedded in the file names themselves. Also, the script should only move a file from /exports to... (6 Replies)
Discussion started by: nikosey
6 Replies

7. Shell Programming and Scripting

Sorting Files by date and moving files in date order

I need to build a k shell script that will sort files in a directory where files appear like this "XXXX_2008021213.DAT. I need to sort by date in the filename and then move files by individual date to a working folder. concatenate the files in the working folder then start a process once... (2 Replies)
Discussion started by: rebel64
2 Replies

8. Linux

Searching for gaps in huge (2.2G) log file?

I've got a 2.2 Gig syslog file from our Cisco firewall appliance. The problem is that we've been seeing gaps in the syslog for anywhere from 10 minutes to 2 hours. Currently I've just been using 'less' and paging through the file to see if I can find any noticeable gaps. Obviously this isn't the... (3 Replies)
Discussion started by: deckard
3 Replies
Login or Register to Ask a Question