appending data from similar files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting appending data from similar files
# 1  
Old 12-08-2011
appending data from similar files

I am familiar with scripting, but I am trying to see if there is an easy way to append files from similar files into one file. For example, if there is file1_20121201, file1_20121202, file1_20121203,
file2_20121201, file2_20121202, file2_20121203

I want to be able to combine all the data from file1_* into file1_output and file2_* into file2_output.

Basically, appending the data from all similar files into one file. I can probably try doing the hard way of looping through all the files, but I am sure there will be much easier and flexible way (not depending on the number of files based on a certain pattern, etc).

Any help on this is greatly appreciated.

Thanks
# 2  
Old 12-08-2011
One way:

Code:
for x in 1 2
do
    (ls file${$x}_* | xargs cat ) >file${x}_output
done

Not tested!!
This User Gave Thanks to agama For This Post:
# 3  
Old 12-08-2011
Sorry, I should have been more specific...I am not sure how many such patterns exist. In other words, it could be file1_*, file2_*, file3_* ...filen_*
how can I do this dynamically?
The actual situation in my case is the timestamp is appended to file1, file2, etc. I would like to combine all such (file1_*, file2_*) from a current day (or whatever the correct range of files I determine) to be appended into one file. But if I get a general idea, I can proceed with the actual implementation, except that I am stuck how to do this dynamically.
# 4  
Old 12-09-2011
Code:
for x in `ls | egrep "file[0-9]+.*"`; do n=`echo $x | cut -d_ -f1 | sed 's/file//'`; cat $x >> file${n}_output; done

This User Gave Thanks to balajesuri For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to format string to collate data of similar IP address.

I have a string which has IP and data of files for that ip which is printed using the below code: Sample string code below: str1="10.9.11.128\n-rwxr-xr-x user1 2019-12-29 17:53 /var/branch/custom/tg.xml 286030210\n10.9.12.129\n-rwxr-xr-x user1 2019-12-29 17:53 /app/branch/custom/tg.xml... (6 Replies)
Discussion started by: mohtashims
6 Replies

2. Shell Programming and Scripting

How to group similar data?

Hello everybody: I want rearrange about 5 million rows (with 300 columns) into groups. Data looks like the following: where there were various experiments (column 2) conducted at different locations (column headers in top row column 4 onwards) in different years (column 1) using instruments... (0 Replies)
Discussion started by: sheetalk
0 Replies

3. Shell Programming and Scripting

Help with data appending to a file

Hi I have a file called text.txt contains x y z when i run a command i will get output like below x 20 z 30 i want to insert x, z value in text.txt file and should be like this x 20 y 0 z 30 can anyone help me please? (1 Reply)
Discussion started by: siva kumar
1 Replies

4. Shell Programming and Scripting

Appending data to the end of a line

I have searched the forms and I can not find info on appending each line of one file to the same line of another file. I know that I can cat one file to another or append the 2nd file to the end of the 1st but not quite sure how to append one line of data to another. For example File 1 has ... (2 Replies)
Discussion started by: scw132
2 Replies

5. Shell Programming and Scripting

appending previous data.

I have on file abc.txt abc.txt: 20090807 Now I want to delete empty lines which has tap/whit spaces from abc.txt .and store the date value in the file into variable.some processs will update the this file with some date . if the process updtes thiis file with empty string , write the... (3 Replies)
Discussion started by: Gopal_Engg
3 Replies

6. Shell Programming and Scripting

reversing and appending data in multiple files

Hello, I have a some files that look like this: 0 3 1 5 2 8 3 7 I want to reverse and append the data so it looks like this: 3 7 2 8 1 5 0 3 0 3 1 5 2 8 3 7 I first thought about using cat and tac cleverly with some redirection and pipe in a one-liner but I couldn't get it to... (1 Reply)
Discussion started by: bigfoot
1 Replies

7. Shell Programming and Scripting

Appending data into a variable

Hi, I would like to know if it's possible to append data into a variable, rather than into a file. Although I can write information into a temporary file in /tmp, I'd rather if possible write into a variable, as I don't like the idea that should my script fail, I'll be polluting the server with... (5 Replies)
Discussion started by: michaeltravisuk
5 Replies

8. Shell Programming and Scripting

appending data to file

Hi. I wrote a very simple script and it doesn't work :( It is supposed to go to a certain directory, execute some command and append the output to the file "expo.dat" what it does is that it writes to the file only one entery. I dont know if Im using the write synthax for "append". Here is... (3 Replies)
Discussion started by: Enigma08
3 Replies

9. UNIX for Dummies Questions & Answers

Appending text to a number of similar filenames

Hi, I was wondering if there was a way to append something to filenames based on a wildcard. For example, if I have the following files in a directory: blah1 blah2 blah3 blah4 blah5 I want to rename these all to: blah1.txt blah2.txt blah3.txt blah4.txt blah5.txt Is there a... (4 Replies)
Discussion started by: Djaunl
4 Replies

10. UNIX for Dummies Questions & Answers

help on appending data to existing data

I need to know how to record the hostname, date/time and all of the process and send it all to one file. I know that the commands I need are hostname, date and ps but I don't know how to do them all and send them all to the same file. Please help! (1 Reply)
Discussion started by: precious51980
1 Replies
Login or Register to Ask a Question