Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

want to combine multiple file into one

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting want to combine multiple file into one
# 1  
Old 08-12-2010
want to combine multiple file into one
Below are the list of files I have
I want to consolidate the monthly log in the one file

Eg : all the july log to be moved to log_july_full.txt inside the script


Path : /home/user/data/log/

Code:
Jun  6 04:08 data_log-2010.05.30-10:04:08.txt
Jun 13 01:38 data_log-2010.06.06-10:04:08.txt
Jun 13 04:00 data_log-2010.06.06-10:04:22.txt
Jun 20 02:03 data_log-2010.06.13-10:04:09.txt
Jun 20 04:01 data_log-2010.06.13-10:04:25.txt
Jun 27 02:06 data_log-2010.06.20-10:04:09.txt
Jun 27 04:05 data_log-2010.06.20-10:04:23.txt
Jul  4 02:04 data_log-2010.06.27-10:04:09.txt
Jul  4 04:04 data_log-2010.06.27-10:04:08.txt
Jul 10 09:43 data_log-2010.07.04-10:04:09.txt
Jul 11 04:04 data_log-2010.07.04-10:04:21.txt
Jul 18 02:02 data_log-2010.07.11-10:04:08.txt
Jul 18 04:02 data_log-2010.07.11-10:04:19.txt
Jul 25 02:09 data_log-2010.07.18-10:04:08.txt
Jul 25 04:01 data_log-2010.07.18-10:04:18.txt
Aug  1 02:05 data_log-2010.07.25-10:04:31.txt
Aug  1 04:02 data_log-2010.07.25-10:04:11.txt
Aug  8 02:03 data_log-2010.08.01-10:04:07.txt
Aug  8 04:05 data_log-2010.08.01-10:04:27.txt

I want the script should go in the path and search the previous month log in " /home/user/data/log/" and consolidate
Last edited by Scott; 08-12-2010 at 01:01 PM.. Reason: Added code tags
# 2  
Old 08-12-2010
if u are searching for algorithm, here it is:
1. go to the specific dir, and
2. get the files that start with particular month ( say jul )
3. sort the files based up on the date in that month,
4. concatenate the contents to the file whichever you want.
5. exit.


Chose a scripting language you are familiar with, and do it in that !!!
# 3  
Old 08-13-2010
i know the algotham..
but i want the specific grep or awk command clearly
so that i can combine the montly files
# 4  
Old 08-13-2010
Code:
#!/bin/bash

file=$1
month_list=`awk '{print $1}' $file|sort -u`

for month in $month_list
do
    grep ^$month $file >>log_$month.txt
done

Pass file name argument 1.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Combine multiple commands

I have the following sh-script: konsole -T todo -e vi todo.txt & konsole -T window1 -e ssh user@server & konsole -T window2 -e ssh user@server2 -e cd directory & The first two lines are working fine. The first opens a txt-file, the second opens a ssh-connection. The third line... (6 Replies)
Discussion started by: andre666
6 Replies

2. UNIX for Dummies Questions & Answers

Combine multiple sed

Would like to convert it to one line sed -i -e '/./,$!d' term1.txt sed -i '2d' term1.txt sed -i '/^$/Q' term1.txt cat term1.txt>> test1.txt Appreciate if someone could point out the error sed -i -e '/./,$!d' -e '2d' -e '/^$/Q' term1.txt cat term1.txt>> $test1.txt ... (6 Replies)
Discussion started by: w020637
6 Replies

3. Shell Programming and Scripting

Compare multiple files, identify common records and combine unique values into one file

Good morning all, I have a problem that is one step beyond a standard awk compare. I would like to compare three files which have several thousand records against a fourth file. All of them have a value in each row that is identical, and one value in each of those rows which may be duplicated... (1 Reply)
Discussion started by: nashton
1 Replies

4. Shell Programming and Scripting

Combine multiple unique lines from event log text file into one line, use PERL or AWK?

I can't decide if I should use AWK or PERL after pouring over these forums for hours today I decided I'd post something and see if I couldn't get some advice. I've got a text file full of hundreds of events in this format: Record Number : 1 Records in Seq : ... (3 Replies)
Discussion started by: Mayday22
3 Replies

5. Shell Programming and Scripting

Combine Multiple Files into Single One File One after other

I am trying to combine 4 .dat files into one single Output file Inputs are:- file123.dat, file256.dat, file378.dat & file490 Expected Output:- FileName=file1 {text from file1} EOF {blank line} FileName=file2 {text from file2} EOF {blank line} FileName=file3 {text from file3} EOF... (4 Replies)
Discussion started by: lancesunny
4 Replies

6. Shell Programming and Scripting

Combine multiple lines in file based on specific field

Hi, I have an issue to combine multiple lines of a file. I have records as below. Fields are delimited by TAB. Each lines are ending with a new line char (\n) Input -------- ABC 123456 abcde 987 890456 7890 xyz ght gtuv ABC 5tyin 1234 789 ghty kuio ABC ghty jind 1234 678 ght ... (8 Replies)
Discussion started by: ratheesh2011
8 Replies

7. Emergency UNIX and Linux Support

Combine multiple Files into one big file

Hi Ppl, I have a requirement like i will be getting files of huge size daily and if the file size is so huge ,the files will be split into many parts and sent.The first file will have the header details followed by detail records and the consecutive files will have detail records and the last... (11 Replies)
Discussion started by: ganesh_248
11 Replies

8. UNIX for Dummies Questions & Answers

Combine multiple files with common string into one new file.

I need to compile a large amount of data with a common string from individual text files throughout many directories. An example data file is below. I want to search for the following string, "cc_sectors_1" and combine all the data from each file which contains this string, into one new... (2 Replies)
Discussion started by: GradStudent2010
2 Replies

9. Shell Programming and Scripting

Combine multiple columns from multiple files

Hi there, I was wondering if someone can help me with this. I am trying the combine multiple columns from multiple files into one file. Example file 1: c0t0d0 c0t2d0 # hostname vgname c0t0d1 c0t2d1 # hostname vgname c0t0d2 c0t2d2 # hostname vgname c0t1d0 c0t3d0 # hostname vgname1... (5 Replies)
Discussion started by: martva
5 Replies

10. Shell Programming and Scripting

combine multiple .xl file

Hi Friends, Is it possible in Unix ? The requirement is ============= Create a unix shellscript that combines multiple excel files into one. Note : it is possible for data files.but the requirement is for excel file (4 Replies)
Discussion started by: bikas_jena
4 Replies
Login or Register to Ask a Question