Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need Header for all splitted files - awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need Header for all splitted files - awk
# 1  
Old 06-01-2010
Need Header for all splitted files - awk
Input file: i have a file and need to split into multiple files based on first column. i need the header for all the splitted files. I'm unable to get the header.
Code:
$ cat log.txt
id,mailtype,value
1252468812,yahoo,3.5
1252468812,hotmail,2.4
1252468819,yahoo,1.2
1252468812,msn,8.9
1252468923,gmail,12
1252468819,live,3.4
1252468929,yahoo,9.0
1252468929,msn,1.2

Required:
Split the above files based on the first field (i.e. lines with same first field should go to the same file)

The awk one liner:
Code:
$ awk -F "," '{close(f);f=$1}{print > f".txt"}' log.txt

Output:
Above file is splited into the following sub-files.
Code:
$ cat 1252468812.txt
1252468812,yahoo,3.5
1252468812,hotmail,2.4
1252468812,msn,8.9
 
$ cat 1252468819.txt
1252468819,yahoo,1.2
1252468819,live,3.4
 
$ cat 1252468923.txt
1252468923,gmail,12
 
$ cat 1252468929.txt
1252468929,yahoo,9.0
1252468929,msn,1.2

Last edited by Franklin52; 06-01-2010 at 02:32 PM.. Reason: Please use code tags
# 2  
Old 06-01-2010
Try:
Code:
awk -F "," '
!h{h=$0}
!a[$1]{a[$1]=$1; print h > $1 ".txt"}
{print > $1 ".txt"}' log.txt

# 3  
Old 06-02-2010
Thanks for your quick response, but i need header for all the splitted files other than first column. How to acheive that.

Thanks in Advance!

Code:
awk -F "|" '
!h{h=$0}
!a[$1]{a[$1]=$1; print h > $1 ".txt"}
{print $2 $3 $4 $5 $6 $7 $8 $9 $10 $11 $12 $13 $14 > $1 ".txt"}' test_file

Last edited by Franklin52; 06-02-2010 at 02:00 PM.. Reason: Please use code tags
# 4  
Old 06-02-2010
What should be the header? Anyway, play around with something like this:
Code:
awk -F "|" '
!a[$1]{a[$1]=$1; print h > $1 ".txt"}
{print $2 $3 $4 $5 $6 $7 $8 $9 $10 $11 $12 $13 $14 > $1 ".txt"}' h="co1 col2 .." test_file

# 5  
Old 06-02-2010
I need output like below

Input File:

id,mailtype,value

1252468812,yahoo,3.5
1252468812,hotmail,2.4
1252468819,yahoo,1.2
1252468812,msn,8.9
1252468923,gmail,12
1252468819,live,3.4
1252468929,yahoo,9.0
1252468929,msn,1.2

Output File1:

$ cat 1252468812.txt

mailtype,value
yahoo,3.5
hotmail,2.4
msn,8.9

Output File2:

$ cat 1252468819.txt
mailtype,value
yahoo,1.2
live,3.4

Output File3:

$ cat 1252468923.txt

mailtype,value
gmail,12

Output File4:

$ cat 1252468929.txt

mailtype,value
yahoo,9.0
msn,1.2
# 6  
Old 06-02-2010
Code:
awk -F "|" '
!a[$1]{a[$1]=$1; print h > $1 ".txt"}
{print $2 $3 $4 $5 $6 $7 $8 $9 $10 $11 $12 $13 $14 > $1 ".txt"}' h="mailtype,value" test_file

# 7  
Old 06-02-2010
When i run that, its not coming out...its going to some loop i guess
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Append header with awk

I have to append the header (ie "START OF LINE") to a file only if there is data in it. Below command is showing header though there is no data in a file, can you suggest how to correct.. awk '{NR!=0} BEGIN{print "START OF LINE"}; {print}' file (3 Replies)
Discussion started by: JSKOBS
3 Replies

2. AIX

Mail subject splitted

Hi I have a script that is sending mail and it was working fine. But the last time it run, it gave some errors. It look like the subject was split into many receiver. I am on AIX 6.1. #!/bin/ksh # HOSTNAME=`which hostname` MACHINE=`${HOSTNAME}` # E-mail to inform hci of a stop echo "###... (9 Replies)
Discussion started by: jacquesT
9 Replies

3. Shell Programming and Scripting

Assistance with an awk code to split files but keep the header

---------- Post updated at 11:48 AM ---------- Previous update was at 11:46 AM ---------- Hello all I have an awk code that successfully creates separate text files based on the first six letters of the second field. What it doesn't do is preserve the header into each resulting file. ... (6 Replies)
Discussion started by: colecandoo
6 Replies

4. Shell Programming and Scripting

Combine splitted low & high byte files into one file

Hi all, i have a binary file splitted into 2 chunks, first part with all high bytes and the second part with all low bytes. I need to combine the two chunks into one binary file like (eg. exactly the reverse of the splitting method solved in the thread # 130940) Hi bytes file content:... (7 Replies)
Discussion started by: mzs
7 Replies

5. Shell Programming and Scripting

Script to multi-transfer splitted files via scp

Hey :3 I am moving some stuff between different servers. I do it like this: scp -r -P 22 -i ~/new.ppk /var/www/bigfile.tar.gz user@123.123.123.123:/var/www/bigfile.tar.gz Lets say, this file is 50 GiB. I would like to know, if its possible to split the file in different parts,... (2 Replies)
Discussion started by: Keenora
2 Replies

6. Shell Programming and Scripting

Adding header to sub files after splitting the main file using AWK

Hi Folks, I have a file like: mainfile.txt: ------------- file1 abc def xyz file1 aaa pqr xyz file2 lmn ghi xyz file2 bbb tuv xyz I need output having two files file1 and file2. file1: ------ Name State Country abc def xyz aaa pqr xyz file2: (3 Replies)
Discussion started by: tanmay.gemini
3 Replies

7. Shell Programming and Scripting

splitting newfile.txt file and executing each splitted files

split -l $split_count newfile.txt for i in $split_files* do if test -s $workingdir/$split_files* then ./<$i.out> fi done ... (4 Replies)
Discussion started by: sanjay mn
4 Replies

8. UNIX for Dummies Questions & Answers

Merge all csv files in one folder considering only 1 header row and ignoring header of all others

Friends, I need help with the following in UNIX. Merge all csv files in one folder considering only 1 header row and ignoring header of all other files. FYI - All files are in same format and contains same headers. Thank you (4 Replies)
Discussion started by: Shiny_Roy
4 Replies

9. Shell Programming and Scripting

Merging words splitted into characters with awk

I have an OCR output with some words splitted into single characters separated by blank spaces, and I want the same text with these words written correctly. Example: This is a text w i t h some s p l i t e d W o r d s . The regular expression for matching splitted words could be something... (5 Replies)
Discussion started by: dokamo
5 Replies

10. UNIX for Dummies Questions & Answers

How to restore splitted DD file

I backup a file to tape by using "dd", since the size is too large it took two tape for the backup, the question is how to restore the said file? The "dd" doesn't ask me insert next tape while restoration. (3 Replies)
Discussion started by: coolmans
3 Replies
Login or Register to Ask a Question