Split file by column value, each with header


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split file by column value, each with header
# 8  
Old 11-14-2016
Hello senhia83,

Could you please try following and let me know if this helps you(though honestly saying I am not completly sure about your query).
Code:
awk -F, 'NR==1{print > "TATA34052016__testcase.csv"; print > "TATA79802016_testcase.csv"} ($NF ~ /TATA3405/){print >> "TATA34052016__testcase.csv"} ($NF ~ /TATA7980/){print >> "TATA79802016_testcase.csv"}'  Input_file

Output it will create 2 files named TATA34052016__testcase.csv and TATA79802016_testcase.csv.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
# 9  
Old 11-14-2016
Quote:
Originally Posted by senhia83
Aia,

This is the error i get. Am I running it wrong?
Code:
$ perl spltwheader.pl testcase tmp6

Type of arg 1 to keys must be hash (not hash element) at spltwheader.pl line 16, near "}) "
Execution of spltwheader.pl aborted due to compilation errors.

Oh, my...my! Picky version of Perl you have!
Please, replace
Code:
 for my $year (keys $data{$fname}) {

for
Code:
 for my $year (keys %{$data{$fname}}) {

This User Gave Thanks to Aia For This Post:
# 10  
Old 11-15-2016
@Ravinder ...your solution is too specific and hardcoded, I`m looking for something generic and flexible like Aia and rdrtx1. Thanks a lot for your help.

@ Aia , the edits worked ! running now..many thanks..
# 11  
Old 11-15-2016
Try also
Code:
awk '
NR==1   {HD = $0
         next
        }

!FN[$5] {FN[$5] = $5 "2016__testcase.csv"
         print HD  >  FN[$5]
        }

        {print $0  >  FN[$5]
        }
' FS="," file

This User Gave Thanks to RudiC For This Post:
# 12  
Old 11-15-2016
@ RudiC:

Can you please explain the last part ?

Code:
awk '
NR==1   {HD = $0 # Get first line ie header
         next    # Dont run any other commands. ie read next line.
        }

!FN[$5] {FN[$5] = $5 "2016__testcase.csv" # here set array FN with index set as value of $5 . it is true twice for given input sample.
         print HD  >  FN[$5] # redirect header from HD to file
        }

        {print $0  >  FN[$5] # Print the current line to file. How this differentiates two different $5 , lines and print them to respective files ?
        }
' FS="," file

# 13  
Old 11-15-2016
The output is redirected to the respective file, controlled by $5, as requested in post#1. If a new (i.e. not yet registered) $5 shows up in a line, the new file name is constructed in FN[$5], and then, all lines with the same $5 are assembled in the same file. The number of files (= different unique $5) is "unlimited" (except for system factors e.g. memory).
These 3 Users Gave Thanks to RudiC For This Post:
# 14  
Old 11-15-2016
Quote:
Originally Posted by RudiC
The output is redirected to the respective file, controlled by $5, as requested in post#1. If a new (i.e. not yet registered) $5 shows up in a line, the new file name is constructed in FN[$5], and then, all lines with the same $5 are assembled in the same file. The number of files (= different unique $5) is "unlimited" (except for system factors e.g. memory).
Hello Rudi,

Thank you for nice explanation, I would like to mention here wouldn't be it good to close the opened files or if there are too many files opened it may be an issue with it. Could you please put some light on this too, will be grateful to you sir.

Thanks,
R. Singh
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split large xml into mutiple files and with header and footer in file

Split large xml into mutiple files and with header and footer in file tried below it splits unevenly and also i need help in adding header and footer command : csplit -s -k -f my_XML_split.xml extrfile.xml "/<Document>/" {1} sample xml <?xml version="1.0" encoding="UTF-8"?><Recipient>... (36 Replies)
Discussion started by: karthik
36 Replies

2. Shell Programming and Scripting

Split a file based on encountering header

I need to split a file based on headers found Input file file1 content: ADD john mickey DROP matt sam output of file F1 john mickey output of file F2 matt sam (5 Replies)
Discussion started by: Diddy
5 Replies

3. Shell Programming and Scripting

Sort and Split file with header and custom name

Hi, I am using SUN SOLARIS (SunOS sun4v sparc SUNW, T5240). I have a huge data file with header and trailer. This file gets used into an ETL process. ETL skips the header record (which is the first record of the file) and loads the rest of the record. The file can be delimited (comma,... (5 Replies)
Discussion started by: Saanvi1
5 Replies

4. Shell Programming and Scripting

Split and add header and trailer from input file

I need to split the file based on pattern from position 34-37 while retaining the header and trailer records in each individual split file Also is it possible to output the TOM and PAT records in the same output file ? I need the output file names same as xyz_pattern_Datetimestamp.txt ... (23 Replies)
Discussion started by: techedipro
23 Replies

5. UNIX for Dummies Questions & Answers

Column Header in the Spool file

Hi All, I have a problem with the column heading while spooling the data from the database. Since i want the column header, therefore i am not using SET HEADING OFF, and i am getting the header, but for the longer column name, some of the character from the column name is missing. for... (7 Replies)
Discussion started by: Pramod_009
7 Replies

6. UNIX for Dummies Questions & Answers

Rename a header column by adding another column entry to the header column name

Hi All, I have a file example.csv which looks like this GrpID,TargetID,Signal,Avg_Num CSCH74_1_1,2007,61,256 CSCH74_1_1,212007,647,679 CSCH74_1_1,12007,3,32 CSCH74_1_1,207,299,777 I want the output as GrpID,TragetID,Signal-CSCH74_1_1,Avg_Num CSCH74_1_1,2007,61,256... (1 Reply)
Discussion started by: Vavad
1 Replies

7. Shell Programming and Scripting

Rename a header column by adding another column entry to the header column name URGENT!!

Hi All, I have a file example.csv which looks like this GrpID,TargetID,Signal,Avg_Num CSCH74_1_1,2007,61,256 CSCH74_1_1,212007,647,679 CSCH74_1_1,12007,3,32 CSCH74_1_1,207,299,777 I want the output as GrpID,TragetID,Signal-CSCH74_1_1,Avg_Num CSCH74_1_1,2007,61,256... (4 Replies)
Discussion started by: Vavad
4 Replies

8. UNIX for Dummies Questions & Answers

split header row into one column

So, I have a massive file with thousands of columns I want a list of the headers in one column in another file. So I need to strip off the top line (can use head-1) But how can I convert from this format: A B C D E F G to A B C D E F G (6 Replies)
Discussion started by: polly_falconer
6 Replies

9. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

10. Shell Programming and Scripting

Split large file and add header and footer to each file

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (1 Reply)
Discussion started by: ashish4422
1 Replies
Login or Register to Ask a Question