Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

CAT multiple files according to file name

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers CAT multiple files according to file name
# 1  
Old 08-07-2014
CAT multiple files according to file name
I have a folder that contains a number of files with file names as follows:
Code:
XX.YYYY..ZZZ.2014.001.000000
XX.YYYY..ZZZ.2014.001.000400
XX.YYYY..ZZZ.2014.001.000800
XX.YYYY..ZZZ.2014.001.001200
XX.YYYY..ZZZ.2014.001.001600
.....
XX.YYYY..ZZZ.2014.002.000000
XX.YYYY..ZZZ.2014.002.000400
XX.YYYY..ZZZ.2014.002.000800
XX.YYYY..ZZZ.2014.002.001200
XX.YYYY..ZZZ.2014.002.001600
....

I would like to cat all files XX.YYYY..ZZZ.2014.001.* into one file named XX.YYYY..ZZZ.2014.001, all files XX.YYYY..ZZZ.2014.002.* into one file named XX.YYYY..ZZZ.2014.002.

I am new to the Forums, so apologies if the post ended up in the wrong place.
Thank you for helping.
Cheers


Moderator's Comments:
Mod Comment Please use code tags next time for your code and data. Thanks
Last edited by vbe; 08-07-2014 at 11:33 AM..
# 2  
Old 08-07-2014
What do they contain, appending files is easy, but then is the order important?
# 3  
Old 08-07-2014
The shell will match all files that begin with XX.YYYY..ZZZ.2014.001. using "*", cat will concatenate the matching files, then redirect the output ">" to a file.
Code:
cat XX.YYYY..ZZZ.2014.001.* > XX.YYYY..ZZZ.2014.001

This User Gave Thanks to xbin For This Post:
quakesrus
# 4  
Old 08-07-2014
Because of the file names they already appear in the correct order.
They are acoustic data from a microphone and the filename is:
XX.YYYY..ZZZ.year.dayofyear.hoursminuteseconds

Ideally I would like to cat them in the order they are listed but this is not necessary.
Thanks for your reply.
Cheers
Last edited by rbatte1; 08-14-2014 at 08:08 AM..
# 5  
Old 08-07-2014
xbin is right and basically the command he gave should be sufficient.

There might be several problems, though:

1. Does order matter (like vbe mentioned)?

2. Are the files well-formed? Some have missing end-of-line markers or end-of-file markers and concatenating them might produce this:

file 1:
Code:
file1line1
file1line2
file1line3

file 2:
Code:
file2line1
file2line2
file2line3

after concatenation:

file 1:
Code:
file1line1
file1line2
file1line3file2line1
file2line2
file2line3

3. How many files are there involved? If you issue a wildcard which expands to a really, really big number of files (several thousands) you might get over the maximum line length of the shell (because the wildcard is expanded to a list of filenames) and the command might fail for this reason. This is a remote chance, but it is there.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
quakesrus
# 6  
Old 08-07-2014
xbin suggestion worked fine. I now just need to script around this command so that I can loop through all groups of files (XX.YYYY..ZZZ.2014.001.*, XX.YYYY..ZZZ.2014.002.*,... XX.YYYY..ZZZ.2014.365.*) that I need to concatenate instead of having to cat each group manually.
Thank you for help.
# 7  
Old 08-07-2014
How about:

Code:
$ touch XX.YYYY..ZZZ.2014.001.12345
$ touch QQ.RRRR..ZZZ.2014.001.12345
$ find . -type f | awk -F"." -v OFS="." '{ NF=7; A[$0] } END { for(X in A) { print X }'

./QQ.RRRR..ZZZ.2014.001
./XX.YYYY..ZZZ.2014.001

$

Code:
$ find . -type f | awk -F"." -v OFS="." '{ NF=7; A[$0] } END { for(X in A) { print X } }' |
while read LINE
do
        echo cat "${LINE}"*
done

cat ./QQ.RRRR..ZZZ.2014.001.12345
cat ./XX.YYYY..ZZZ.2014.001.12345

$

Remove the 'echo' once you've tested it and are sure it does what you want. And add > "${LINE}.complete" or something. Ideally you'd want to create them in a different folder.
Last edited by Corona688; 08-07-2014 at 12:23 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge multiple columns into one using cat

I will like to merge several files using 'cat', but I observe the output is not consistent. the merge begins at the last line of the first file. file1.txt: 1234 1234 1234 file2.txt: aaaa bbbb cccc dddd cat file1.txt file2.txt > file3.txt file3.txt: 1234 1234 1234aaaa bbbb cccc... (13 Replies)
Discussion started by: geomarine
13 Replies

2. Shell Programming and Scripting

Cat files listed in text file and redirect to new directory with same filename

I have a directory that is restricted and I cannot just copy the files need, but I can cat them and redirect them to a new directory. The files all have the date listed in them. If I perform a long listing and grep for the date (150620) I can redirect that output to a text file. Now I need to... (5 Replies)
Discussion started by: trigger467
5 Replies

3. UNIX for Dummies Questions & Answers

Is there any way to cat multiple files and show filenames?

Hi, Is there any way to do a cat * where it shows the name of each file in the process? Similar to what more does below? $ more ?.sql :::::::::::::: 1.sql :::::::::::::: set linesize 200 select db_unique_name, cast( from_tz( cast(... (5 Replies)
Discussion started by: newbie_01
5 Replies

4. UNIX for Dummies Questions & Answers

Cat files based on file name

Bros, I have list of files 20140916_registeredshop.csv 20140916_datavisit.csv 20140915_registeredshop.csv 20140915_datavisit.csv 20140914_registeredshop.csv 20140914_datavisit.csv 20140913_registeredshop.csv 20140913_datavisit.csv 20140912_registeredshop.csv 20140912_datavisit.csv ... (1 Reply)
Discussion started by: radius
1 Replies

5. UNIX Desktop Questions & Answers

trying to cat multiple pairs of files

I have a number of files in a directory named like this: fooP1, fooN1, fooP2, fooN2 ... fooP(i), fooN(i). I'd like to know how to combine each P and N pair into a single file, foo(i) TIA John Balwit (1 Reply)
Discussion started by: balwit
1 Replies

6. Shell Programming and Scripting

.sh script / Check for file in two directories then cat the files

Hi, Here is what i'm trying to do, -Check in two directories for a user inputed filename -Then cat all the version found of the file to the current screen I am a total nembie to programming, here what i have done so far.... #!/bin/bash #Check in /home/loonatic and /var/named/master... (2 Replies)
Discussion started by: Loonatic
2 Replies

7. Shell Programming and Scripting

bash: cat multiple files together except first line?

Hopefully the title summarized what I need help with. I have multiple files that I would like to concatenate in bash. ie: cat file1 file2 file3 > bigfile except I do not want to include the first line from each file (). Any help? Thanks. (6 Replies)
Discussion started by: sanimfj
6 Replies

8. Shell Programming and Scripting

cat certain files in directories to files named after the dir?

Hi all, I have a directory with many subdirectories each named like so: KOG0001, KOG0002, ...KOG9999. Each of these subdirectories contain a variable number two kinds of files (nuc and prot) named like so: Capitella_sp_nuc_hits.fasta (nuc) and Capitella_sp_prot_hits.fasta (prot). The... (2 Replies)
Discussion started by: kmkocot
2 Replies

9. UNIX for Dummies Questions & Answers

cat multiple files questions

Hi-- I'm trying to figure out how to use cat more wisely. I have the following command, which works, but I'd like to understand how to get it to work more clearly and efficiently. cat 'my file.001' 'my file.002' 'my file.003' 'my file.004' 'my file.005' 'my file.006' 'my file.007' 'my... (6 Replies)
Discussion started by: rlinsurf
6 Replies

10. Shell Programming and Scripting

Cat'ing a multiple line file to one line

I am writing a script that is running a loop on one file to obtain records from another file. Using egrep, I am finding matching records in file b, then outputing feilds of both into another file. **************************** filea=this.txt fileb=that.txt cat $filea | while read line do... (1 Reply)
Discussion started by: djsal
1 Replies
Login or Register to Ask a Question