Simple script for separating files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Simple script for separating files
# 1  
Old 09-08-2008
Simple script for separating files

Hi all,

I was wondering if someone could help me writing a simple script on separating files into separate folders.

I have 92 files called various things and I want to separate these folders like so:
Create a folder called "1" and put files 1-23 in it
Create a folder called "2" and put files 24-46 in it
Create a folder called "3" and put files 47-69 in it
Create a folder called "4" and put files 70-92 in it

Again the files aren't names 1 thru 92, so is there a way to say take the first 23, and then the next 23 and so on....
How would I go about doing this in csh?


Thanks in advance!
-m
# 2  
Old 09-08-2008
It all depends on how you want them sorted.

Do you have a hard limit at 23? If so, count all of your files, devide by 23, round up to the nearest whole number. You need this many directories.

You can list your directory in order by date (or reverse order) / alphabetically (or reverse alpha) / etc.

Pipe your list into a simple do/while script with a counter inside. While the counter is less than 23 (or whatever number you require), move the next file in the list into a subdirectory.

This looks suspiciously like homework, so please don't be offended if I don't write the script for you. Give it a try, post back here with your results. At least make an effort.
# 3  
Old 09-08-2008
Thanks for your suggestions. I can assure you it isn't homework. I work in a fMRI lab and we use linux/unix based software for doing analysis and I just started and am trying to slowly learn how to write scripts to separate our dicom files to make my life easier.

Most of your suggestions are still foreign to me. So I will try to work on it, but it might take me a while to even get up to that point.

Thanks,

Megan
# 4  
Old 09-08-2008
Hi Megan,

First off, sorry for the assumption - I'm still new here, and trying hard not to break any rules myself!

Here is some code that I've thrown together that should resolve your issue (no guarantees!)

NOTE: I did not include directory creation to this script. You can do it, but I didn't think to include it before firing it up here.

Code:
#!/usr/bin/csh

# Here are the variables that you will need to change:
################################################################################
set number_of_files = 23           # This is the number of files that you expect
                                   # to store in each directory
set source_dir = /tmp/testing      # You'll change this to your source directory
set target_dir = /tmp/folder_      # This is the prefix that you'll use for your
                                   # folder tree. You'll likely wish to change it

# Here are the 'static' variables that you will not change:
################################################################################
#this number will increment the count cap
set count = 0
#iteration counters
set folder_counter = 1  # Starts at one, such that multiplication issues don't arise
set counter = 0


# Get your list of files - I'm taking it from an ls -tr
# This provides an historic list, with the oldest files first

foreach file ( `ls -tr $source_dir` )
   @ counter = $counter + 1
   @ count = $number_of_files * $folder_counter   #First time 'round, this is 3, then 6

   # First, check to see if you should move on to the next folder or not...
   #  - adding a 'move' request to ensure that the file is not skipped
   if ($counter > $count) then
      @ folder_counter++
      mv $source_dir/$file $target_dir$folder_counter
   endif


   # Since you didn't need to increment the folder_counter - move the file as if nothing happened...
   if ($counter <= ($count)) then
      mv $source_dir/$file $target_dir$folder_counter
   endif

end

Please let me know if it works. I'm sure that there are other MUCH MORE elegant solutions Smilie


edit:changed a comment (wrong comment included in paste)
# 5  
Old 09-08-2008
Thanks for all your help! That did help a bit, at least got me started on the right foot. So I changed things a bit (see below). However, when I run it the first 23 files don't end up in a directory, and it all ends up in a directory called DTI5. Is there anyway to count the files and after counting them say move file #1 thru #23 to directory DTI1, move file #24 thru #46 to directory DTI2, and so on AND THEN tell it to stop counting at file number 92 if there happens to be more files in the source directory???

#!/bin/csh

# Here are the variables that you will need to change:
################################################################################
mkdir DTI1
mkdir DTI2
mkdir DTI3
mkdir DTI4

set number_of_files = 23 # This is the number of files that you expect
# to store in each directory
set source_dir = ./ # changed this to the current directory I want it run in (has all 92 files)
set target_dir = DTI # This is the prefix that you'll use for your
# folder tree. You'll likely wish to change it

# Here are the 'static' variables that you will not change:
################################################################################
#this number will increment the count cap
set count = 0
#iteration counters
set folder_counter = 1 # Starts at one, such that multiplication issues don't arise
set counter = 0


# Get your list of files - (I don't want them by date, they are already in order so I just list them)
Smilie
foreach file ( `ls $source_dir` )
@ counter = $counter + 1
@ count = $number_of_files * $folder_counter #First time 'round, this is 3, then 6

# First, check to see if you should move on to the next folder or not...
# - adding a 'move' request to ensure that the file is not skipped
if ($counter > $count) then
@ folder_counter++
mv $source_dir/$file $target_dir$folder_counter
endif


# Since you didn't need to increment the folder_counter - move the file as if nothing happened...
if ($counter <= ($count)) then
mv $source_dir/$file $target_dir$folder_counter
endif

end
# 6  
Old 09-09-2008
k...

Changed it up a bit, and added the directory testing... (Remember to NOT have the script in the same directory, or it will get moved along with 91 of your 92 files)

Cheers!

Code:
#!/bin/csh

# Here are the variables that you will need to change:
################################################################################
set number_of_files = 23              # This is the number of files that you
                                      # expect to store in each directory
set total_files_to_move = 92          # This is the total number of files that
                                      # you will move with this utility
set source_dir = ./                   # set to 'current' directory
set target_dir = ./DTI                # This is the prefix that you'll use for
                                      # the folder tree.

# Here are the 'static' variables that you will not change:
################################################################################
#this number will increment the count cap
set count = 0
#iteration counters
set folder_counter = 1  # Starts at one, such that multiplication issues don't arise
set counter = 0

# Get your list of files
foreach file ( `ls -tr $source_dir` )
   @ counter = $counter + 1
   @ count = $number_of_files * $folder_counter

   # Test to see if the first target directory exists
   if (! -d $target_dir$folder_counter ) then
      mkdir $target_dir$folder_counter
   endif

# Test to see if the maximum number of files to move have been moved
   if ($counter > ($total_files_to_move)) then
      exit
   endif


   # Check to see if you should move on to the next folder or not...
   if ($counter > $count) then
      @ folder_counter++

      # Test to see if subsequent target directory exists
      if (! -d $target_dir$folder_counter ) then
         mkdir $target_dir$folder_counter
      endif

      mv $source_dir/$file $target_dir$folder_counter/$file
   endif

   # Since you didn't need to increment the folder_counter - move the file as if nothing happened...
   if ($counter <= ($count)) then
      mv $source_dir/$file $target_dir$folder_counter/$file
   endif
end

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to generate a csv files by separating the values from the input file based on position?

Hi All, I need help for doing the following. I have a input file like: aaaaaaaaaabbbbbbbbbbbbbbbbbbbb cccbbbbbaaaaaadddddaaaabbbbbbb now I am trying to generate a output csv file where i will have for e.g. 0-3 chars of each line as the first column in the csv, 4-10 chars of the line as... (3 Replies)
Discussion started by: babom
3 Replies

2. Programming

Separating two classes in two files

I have a file Map.hh shown below. I want to put the two classes Phase and Map in two different files Phase.hh and Map.hh. I have forward declaration before the Map class. How can I tackle this situation? ////////////////////////////////////////////////////////////////////////// #ifndef... (3 Replies)
Discussion started by: kristinu
3 Replies

3. Shell Programming and Scripting

Separating list of input files (*.file) with a comma in bash script

Hi all, I'm trying to get a bash script working for a program (bowtie) which takes a list of input files (*.fastq) and assembles them to an output file (outfile.sam). All the .fastq files are in one folder in my home directory (~/infiles). The problem is that the 'bowtie' requires that... (7 Replies)
Discussion started by: TuAd
7 Replies

4. UNIX for Dummies Questions & Answers

Simple version control script for text files

HI guys, Could you help me writing a simple version control script for a text files. the format could be ./version_control <file(s)> (I want it to be able to work with more than 1 file at the same time) commands are add and get, add means you add new file(s) to the archive, get means you... (4 Replies)
Discussion started by: s3270226
4 Replies

5. Shell Programming and Scripting

Separating Pattern Into Separate Files

I am trying to separate a specific pattern match into separate files. Sometimes there is only one pattern match, but other times there could be multiple (up to 6 or 8). Pattern is as follows - its starts with NYZ or VTZ and ends with $$. Again looking to get those blocks of data from one big... (17 Replies)
Discussion started by: Double-E
17 Replies

6. Shell Programming and Scripting

Simple script to find common strings in two files

Hi , I want to write a simple script. I have two files file1: BCSpeciality Backend CB CBAPQualDisp CBCimsVFTRCK CBDSNQualDisp CBDefault CBDisney CBFaxMCGen CBMCGeneral CBMCQualDisp file2: CSpeciality Backend (8 Replies)
Discussion started by: ramky79
8 Replies

7. Shell Programming and Scripting

Simple script to count files

Hello, I am new to shell scripting and I need your help. I have found similar scripts in the forum but I need further assistance. I am building a script to use hourly in cron to mailx me if the number of files in a path is less than e.g 100 I have started with the following: #!/bin/sh... (2 Replies)
Discussion started by: drbiloukos
2 Replies

8. UNIX for Dummies Questions & Answers

any script for joining files based on simple conditions

Condition1; If NPID and IndID of both input1 and input2 are same take all the vaues relevant to them and print together as output Condition2; IDNo in output: Take the highly repeated same letter of similar NPID-IndID as *1* Second highly repeated same letter... (0 Replies)
Discussion started by: stateperl
0 Replies

9. Shell Programming and Scripting

Merging files into a single tab delimited file with a space separating

I have a folder that contains say 50 files in a sequential order: cdf_1.txt cdf_2.txt cdf_3.txt cdf_3.txt . . . cdf_50.txt. I need to merge these files in the same order into a single tab delimited file. I used the following shell script: for x in {1..50}; do cat cdf_${x}.txt >>... (3 Replies)
Discussion started by: Lucky Ali
3 Replies

10. Shell Programming and Scripting

Simple script uploading *.dem files to an ftp

Hello.. i want to create a simple script that's upload all the *.dem files from one directory to ftp and then delete them. Any help? (3 Replies)
Discussion started by: TuXaKoS
3 Replies
Login or Register to Ask a Question