cat files from subdirectories output using same filename


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers cat files from subdirectories output using same filename
# 1  
Old 08-12-2010
Data cat files from subdirectories output using same filename

Hi,
I need to concatenate data files with a .mp extension that are stored in directories by year. I want to keep the same filename as an output for example:

for the file name p030.mp, which resides in the following subdirectories:

/2000/p030.mp
/2001/p030.mp
/2002/p030.mp

I want to:
concatenate p030.mp for 2000-2002 and output the concatenated file in a new directory, say:
/ifeeldumb/p030.mp

I can get this to work for a single file:
cat /2000/p030.mp /2001/p030.mp /2002/p030.mp > p030.mp

but I get lost when I try to put it in a loop and extract individual filenames for use with output names.

any help would be greatly appreciated, THANKS!!!!
# 2  
Old 08-12-2010
Is there any pattern to p030? If you need to do, say, 001 through 999:

Code:
for ((N=1; N<1000; N++))
do
        FILENAME=`printf "p%03d.mp" "$N"`
        cat /200[0-2]/${FILENAME} > /ifeeldumb/${FILENAME}
done


Last edited by Corona688; 08-12-2010 at 07:46 PM.. Reason: reduce the range of years
This User Gave Thanks to Corona688 For This Post:
# 3  
Old 08-12-2010
more info on cat files post

Hi Corona688--

Thanks for your reply. To be more specific, p030.mp is the name of a station for GPS data. So, in each subdirectory by year, I have probably 50 different station names with the .mp extension. The only thing the station names (filenames) have in common, besides the .mp extension, is that they can only be 4 letter/number combos in length proceeding the .mp extension:

so they could be:
p030.mp
abcd.mp
abc0.mp
iid2.mp

To make it more confusing, I have data from 2006-2010 but some stations only have data from 2008-onward.

In terms of actual content within the station files, they may have different lines (different days of year present). Not sure if that matters.

Thanks for your help! I really appreciate it.
# 4  
Old 08-12-2010
Something like this should work (assuming that your present working directory contains the 20?? directories.)

Code:
#!/usr/bin/env ksh

target=/tmp/cats
find 20?? -name "*.mp" | while read file
do
        base=${file##*/}    # ditch the directory portion of the filename
        cat $file >>$target/$base.new    # I like to add .new to prevent accidental overwriting while testing!
done

Change target to the directory you want to put the output files into and you're good to go.
These 2 Users Gave Thanks to agama For This Post:
# 5  
Old 08-13-2010
thanks!

Thanks, agama. The code you wrote worked like a charm! Smilie
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash to calculate average of all files in directory and output by part of filename

I am trying to use awk to calculate the average of all lines in $2 for every file in a directory. The below bash seems to do that, but I cannot figure out how to capture the string before the _ as the output file name and have it be tab-delimeted. Thank you :). Filenames in... (3 Replies)
Discussion started by: cmccabe
3 Replies

2. UNIX for Dummies Questions & Answers

Find all files containing string not following symlinks CAT (modified) output content to /filename

This should recursively walk through all dirictories and search for a specified string in all present files, if found output manicured content (eg some regex) with CAT into a specified directory (eg /tmp/) one by one, keeping the original names This is what I have so far, which seems to... (1 Reply)
Discussion started by: lowmaster
1 Replies

3. Shell Programming and Scripting

Cat files listed in text file and redirect to new directory with same filename

I have a directory that is restricted and I cannot just copy the files need, but I can cat them and redirect them to a new directory. The files all have the date listed in them. If I perform a long listing and grep for the date (150620) I can redirect that output to a text file. Now I need to... (5 Replies)
Discussion started by: trigger467
5 Replies

4. Shell Programming and Scripting

Append string to all the files inside a directory excluding subdirectories and .zip files

Hii, Could someone help me to append string to the starting of all the filenames inside a directory but it should exclude .zip files and subdirectories. Eg. file1: test1.log file2: test2.log file3 test.zip After running the script file1: string_test1.log file2: string_test2.log file3:... (4 Replies)
Discussion started by: Ravi Kishore
4 Replies

5. Shell Programming and Scripting

Bash script deleting my files, and editing files in subdirectories question

#!/bin/bash # name=$1 type=$2 number=1 for file in ./** do if then filenumber=00$number elif then filenumber=0$number fi tempname="$name""$filenumber"."$type" if (4 Replies)
Discussion started by: TheGreatGizmo
4 Replies

6. Shell Programming and Scripting

Getting output from a file similar to cat output

I have a file # cat /root/llll 11 22 33 44 When I cat this file content to a variable inside a shell script and echo that shell script, it does not show up as separate lines. I need echo output similar to cat. cat /root/shell_script.sh #!/bin/bash var=`cat /root/llll` echo $var (2 Replies)
Discussion started by: anil510
2 Replies

7. Shell Programming and Scripting

Recursively cat files in a directory with filename printed first.

I want to recursively cat the content of files in a directory e.g. find /etc -type f -exec cat {} \; But I want it to print the file name first and then the content. For example let's say /etc/statetab and /etc/colord.conf will be printed first then I want the output to look something like; ... (6 Replies)
Discussion started by: lewk
6 Replies

8. Shell Programming and Scripting

wget same filename from subdirectories and rename or concat

I would like to wget a file "index.html" from a site which is lies in different subdirectories and is different in size. The index.html shall be concatenated or renamed to index01.html index02.html .... I would like to store this file in just one directory and use the option -nd. Though i can't get... (0 Replies)
Discussion started by: sdf
0 Replies

9. Shell Programming and Scripting

Filename from splitting files to have the same filename of the original file with counter value

Hi all, I have a list of xml file. I need to split the files to a different files when see the <ko> tag. The list of filename are B20090908.1100-20090908.1200_CDMA=1,NO=2,SITE=3.xml B20090908.1200-20090908.1300_CDMA=1,NO=2,SITE=3.xml B20090908.1300-20090908.1400_CDMA=1,NO=2,SITE=3.xml ... (3 Replies)
Discussion started by: natalie23
3 Replies

10. Shell Programming and Scripting

cat and output filename

Hi, how I can display a file with cat and printing the filename before each line. e.g. file1 { one two three four five } Output: file1:one file2:two file1:three file1:four file1:five (12 Replies)
Discussion started by: Timmää
12 Replies
Login or Register to Ask a Question