Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Split 200.000 files into different subfolders

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Split 200.000 files into different subfolders
# 1  
Old 06-19-2010
Question Split 200.000 files into different subfolders
Dear UNIX-Community,

can help me doing 2 things in Debian 5.0?

1.) Create 100 folders

Format: ./0/0, ./0/1, ./0/2, ...,./0/9, ./1/0/, ..., ./9/9

2.) Move over 200.000 files into the subdirectories according to their last digits.

12398123.dat -> ./3/2/12398123.dat
48161.dat -> ./1/6/48161.dat

Thank you very much!

Regards,
Diadas
# 2  
Old 06-19-2010
With bash (ksh93 supports that syntax too):

Code:
mkdir -p {0..9}/{0..9}

for f in *dat; do
  fn=${f%.dat} 
    mv -v -- "$f" "${fn: -1}/${fn: -2:1}"
done

You should add error checking.
# 3  
Old 06-19-2010
Code:
$ for i in 0 1 2 3 4 5 6 7 8 9; do
for ii in 0 1 2 3 4 5 6 7 8 9; do
mkdir -p $i/$ii; done; done
$ 
$ for i in *.dat ; do
subdir=$(echo $i | sed 's/.*\(.\)\(.\)\.dat/\1/')
maindir=$(echo $i | sed 's/.*\(.\)\(.\)\.dat/\2/')
echo "$i" $maindir/$subdir
done
$

Replace echo with mv or mv -v if your satisfied with the output.
Handle with care. Ensure that you have a fresh and working backup before doing this for the case that something goes wrong.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with split a list of records into each line with 200 coordinate at a time

Input File: E 3359799 3360148 350 X D 3471287 3471607 321 X E 3359799 3360740 942 X E 4359790 4360039 250 X . . . Desired Output File: E 3359799 3359998 200 X E 3359999 3360148 150 X D 3471287 3471486 200 X D 3471487 3471607 121 X E 3359799 3359998 200 X E 3359999 3360198 200 X... (1 Reply)
Discussion started by: perl_beginner
1 Replies

2. UNIX for Beginners Questions & Answers

Find for files within a list of subfolders

Helo Is there a better way to search within a list of subfolders : A_START_PATH="/data_1/data_2" # # dir2, dir3, dir6, ..... dir59 exists # A_LIST="$A_START_PATH/dir1 $A_START_PATH/dir4 $A_START_PATH/dir5" find "$A_LIST" -type f -name"*.txt" Now searching for all files in any subdirs... (2 Replies)
Discussion started by: jcdole
2 Replies

3. Shell Programming and Scripting

Splitting a folder containing different files into subfolders

I have a folder with 4000 (*3) files like gr_q4_gb-1.anc gr_q4_gb-1.anc_cdr_st.txt gr_q4_gb-1.anc_cdr_tr.txt gr_q4_gb-2.anc gr_q4_gb-2.anc_cdr_st.txt gr_q4_gb-2.anc_cdr_tr.txt gr_q4_gb-3.anc gr_q4_gb-3.anc_cdr_st.txt gr_q4_gb-3.anc_cdr_tr.txt . . gr_q4_gb-4000.anc... (6 Replies)
Discussion started by: sammy777888
6 Replies

4. UNIX for Dummies Questions & Answers

Take 100MB worth files from 200,000 Files

Hi, I have a process which creates almost 200K files. Each file ranging from 1kb to 5kb. This is almost 2GB of data in all files. I have a requirement where the business needs only 100MB worth of files. Is there a way to get files around 100MB (doesn't have to be exactly 100MB) from all the... (2 Replies)
Discussion started by: grep_me
2 Replies

5. Shell Programming and Scripting

List all the files in the present path and Folders and subfolders files also

Hi, I need a script/command to list out all the files in current path and also the files in folder and subfolders. Ex: My files are like below $ ls -lrt total 8 -rw-r--r-- 1 abc users 419 May 25 10:27 abcd.xml drwxr-xr-x 3 abc users 4096 May 25 10:28 TEST $ Under TEST, there are... (2 Replies)
Discussion started by: divya bandipotu
2 Replies

6. UNIX for Dummies Questions & Answers

Need help in checking for files in subfolders

Hi, I am trying to print a listing of files from the top level directory, check to see if any files have the same name as the top level directory name and if so, cd to that file and list the files under it. Don't know how to check for the file in the next level. What I have so far: ... (6 Replies)
Discussion started by: tes218
6 Replies

7. Shell Programming and Scripting

Compare 200,000 of rows in two text files

Friends, I have two very large plain text files with pipe delimited as below. Both files are not sorted. Both files have 200,000 of rows. FName|LName|Address|HPhNumber Is perl or shell script feasible for this task? Thanks, Prashant (1 Reply)
Discussion started by: ppat7046
1 Replies

8. UNIX for Dummies Questions & Answers

rename files in subfolders

Hello i have this script : foreach f ($1/*.cpp ) mv $f $f:r.c end that renames me files in dir , how can i change it so it will rename me also in subdirectorys? thanks (0 Replies)
Discussion started by: umen
0 Replies

9. News, Links, Events and Announcements

Microsoft "Donates" $3,000,000,000 to Feds

Surreal quote from the news link below: http://www.washingtonpost.com/wp-dyn/articles/A44615-2002Nov12.html (0 Replies)
Discussion started by: Neo
0 Replies
Login or Register to Ask a Question