pasting two files in every directory (+100 directories)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting pasting two files in every directory (+100 directories)
# 1  
Old 01-23-2012
pasting two files in every directory (+100 directories)

Hi,

I have around 400 directories each one named as hour_1/ , hour_2/ .....hour_400/ and each of these contains two files, namely:

File1: hour_1.txt (in hour_1/) , hour_2.txt (in hour_2/) ....hour_400.txt (in hour_400/) etc...


File2: client_list_hour_1.txt (in hour_1/), client_list_hour_2.txt (in hour_2/).....client_list_hour_400.txt (in hour_400/) etc...

and i wish to paste (by using the unix command paste) File 2 with File 1.

is there a quick way of doing this?

at the moment i've tried:

Code:
#!usr/bin/sh
find /home/my_dir/ -name "client_list_*" -type f | \
while read fname
do
     fileBaseName=`basename "${fname}"`
     fileDirName=`dirname "${fname}"`

     find /home/my_dir/ -name "hour_*" -type f | \
     while read fn
     do
        fBaseN =`basename "${fn}"`
        fDirName=`dirname "${fname}"`
      paste ${fname} ${fn} > ${fDirName}\stats.txt 
     done
done


and pasting is not done properly as I expected...any suggestions?

thanks in advance
# 2  
Old 01-23-2012
In what way is pasting not done how you expected? Smilie

Is there any pattern to these directories that would help find them?

The backslashes you have everywhere aren't necessary. Pipes act to join statements together by themselves.

Code:
find /home/my_dir/ -name "prefix_*" -type d |
while read DIR
do
        paste "$DIR"/hour_1/hour_1.txt "$DIR"/hour_2/hour_2.txt > "$DIR"/stats.txt
done


Last edited by Corona688; 01-23-2012 at 01:44 PM..
# 3  
Old 01-23-2012
Hi Corona and thank you for your quick response

Let me elaborate a bit more on it.
For instance:

the hour_1/clients_hour_1.txt is a single-column file with 700 records (i.e. 700 lines containing only one field per line)

I was expecting after pasting their corresponding stats residing in the 3 column file hour_1/hour_1.txt (that has exactly the same record number-i.e. 700)

the resulting file (hour_1/stats.txt) should be a 4 column with all the fields filled...but is not


also, the code you provided does not seem to work for all the 400 directories..but only for the first 2...i need to have in every directory a stats.txt file.

thank you again
# 4  
Old 01-23-2012
Quote:
Originally Posted by amarn
the resulting file (hour_1/stats.txt) should be a 4 column with all the fields filled...but is not
All right... So what is it, then? Smilie

Are these DOS text files? Windows has a strange habit of adding carriage-returns to everything. You may need to dos2unix these files before you can process them using UNIX utilities.

Quote:
also, the code you provided does not seem to work for all the 400 directories..but only for the first 2...
I expected you'd have to modify it to your tastes, because you didn't tell me what any of the directories were named so my code probably wouldn't match them... And you still haven't told me even though I asked... Smilie

So what are these directories named?
# 5  
Old 01-23-2012
ok,

for example, the client_hour_1.txt is composed by clientIDs formatted as below:

Quote:
id_555
id_472
...
...
...
the hour_1.txt contains some stats for each of the clients in the previous file and looks like:

Quote:
0.33000 44.3232 99.0424
88.4350 12.6462 45.0931
......... .......... .........
the desired stats.txt should look like
Quote:
id_555 0.33000 44.3232 99.0424
id_472 88.4350 12.6462 45.0931
......
......
I'm under cygwin on Windows...the hour_*.txt files were generated in MATLAB (saved in ASCII format) and the client_hour_*.txt files were created using awk...(i actually extracted them from another file using awk)

so, the root directory is /home/my_dir/

and this directory contains 400 directories , e.g. /home/my_dir/hour_1/ , /home/my_dir/hour_2/ ,etc..etc..

each of these hour_*/ directories contains the files i described above....and the ideal is to have for every directory a stats.txt file , ideally i will try to have for every directory a stats_hour_*.txt file...(e.g. for hour_1/ i would like a stats_hour_1.txt file)

thank you again
# 6  
Old 01-23-2012
You could try this:

Code:
for num in {1..400}
do
   echo "Processing directory #${num}"
   cd /home/my_dir/hour_${num}
   dos2unix -q -n hour_${num}.txt tmp.txt
   paste tmp.txt client_hour_${num}.txt > stats_hour_${num}.txt
   rm tmp.txt
done

This User Gave Thanks to Chubler_XL For This Post:
# 7  
Old 01-23-2012
Quote:
Originally Posted by amarn
I'm under cygwin on Windows
Next time note that at the beginning. Not actually using UNIX, and dealing with files in a system so incompatible with UNIX, is a bit of a game-changer when trying to use UNIX utilities in the usual way.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. AIX

How to backup a directory (sub-directories/files) files from one server on to other ?

Hello, Server A: /directory1/ Server B: /Backups/ i wanted to backup contents of /directory1 from "server A" on to "Server B" every 1 hour. If there is any change in (only new/differences) contents on serverA (directory1/) supposed to be backeup on next run. I did used rsync command to... (5 Replies)
Discussion started by: System Admin 77
5 Replies

2. UNIX for Advanced & Expert Users

Find all files in the current directory excluding hidden files and directories

Find all files in the current directory only excluding hidden directories and files. For the below command, though it's not deleting hidden files.. it is traversing through the hidden directories and listing normal which should be avoided. `find . \( ! -name ".*" -prune \) -mtime +${n_days}... (7 Replies)
Discussion started by: ksailesh1
7 Replies

3. Shell Programming and Scripting

How to list all the files, directories and sub-directories in the current path except one directory?

Can anyone come up with a unix command that lists all the files, directories and sub-directories in the current directory except a folder called log.? Thank you in advance. (7 Replies)
Discussion started by: Manjunath B
7 Replies

4. Shell Programming and Scripting

going through files in a directory, and it's sub-directories

i have a c-shell script. and i pass it one or two arguments the first argument is the directory the second is the "-r" now, if i pass it only one, it searches the files ending in txt in that directory if i pass it two parameters, like "temp" and "-r" i want it to go through all the files and... (1 Reply)
Discussion started by: pantelis
1 Replies

5. Shell Programming and Scripting

compare files in two directories and output changed files to third directory

I have searched about 30 threads, a load of Google pages and cannot find what I am looking for. I have some of the parts but not the whole. I cannot seem to get the puzzle fit together. I have three folders, two of which contain different versions of multiple files, dist/file1.php dist/file2.php... (4 Replies)
Discussion started by: bkeep
4 Replies

6. UNIX for Dummies Questions & Answers

Renaming files after their directory name in multiple sub directories

So I am not sure if this should go in the shell forum or in the beginners. It is my first time posting on these forums. I have a directory, main_dir lets say, with multiple sub directories (one_dir through onehundred_dir for example) and in each sub directory there is a test.txt. How would one... (2 Replies)
Discussion started by: robotsbite
2 Replies

7. UNIX for Advanced & Expert Users

Extracting the different files from directory & its sub directories

Hi Everyone, It would be helpful if someone helps me on this. Requirement: I have a directory which includes different types of files(for example *.java,*.class),but not restricted for only these types. I need to find the same types of file extensions from its directories and subdirectories... (3 Replies)
Discussion started by: rcvasu
3 Replies

8. Shell Programming and Scripting

mget * (obtein files from current directory but not the files form sub-directories)

Hello, Using the instruction mget (within ftp) and with "Interactive mode off", I want to get all files from directory (DirAA), but not the files in sub-directories. The files names don't follow any defined rule, so they can be just letters without (.) period Directory structure example: ... (0 Replies)
Discussion started by: Peter321
0 Replies

9. Shell Programming and Scripting

how many directories and files are there in a directory

I want to know how many directories and files are there in a directory and if the sub directory have any files i also need that also . I have done this far .... (4 Replies)
Discussion started by: smongam
4 Replies

10. UNIX for Dummies Questions & Answers

List files that are not directories from current directory

I can't manage to list all files that are not directories from current directory. (2 Replies)
Discussion started by: beni22sof
2 Replies
Login or Register to Ask a Question