Visit Our UNIX and Linux User Community


Split a single file into multiple files based on a value.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a single file into multiple files based on a value.
# 1  
Old 02-19-2016
Split a single file into multiple files based on a value.

Hi All,

I have the sales_data.csv file in the directory as below.

Code:
SDDCCR;  SOM   ; MD6546474777   ;05-JAN-16
ABC   ;  KIRAN ; CB789          ;04-JAN-16
ABC   ;  RAMANA; KS566767477747 ;06-JAN-16
ABC   ;  KAMESH; A33535335      ;04-JAN-16
SDDCCR;  DINESH; GD6674474747   ;08-JAN-16
SDDCCR;  SUJAN ; SA6666464664646;05-JAN-16
XYZ  ;   AMAR  ; AB123456       ;04-JAN-16
XYZ  ;   RAJ   ; CS78890        ;04-JAN-16
XYZ  ;   GANE  ; MD6546474777   ;08-JAN-16
XXX  ;   MANO  ; MD6546474777   ;04-JAN-16

I want to split the file as below based on last column value.

sales_data04-JAN-16.csv
Code:
ABC   ;  KIRAN ; CB789          ;
ABC   ;  KAMESH; A33535335      ;
XYZ  ;   AMAR  ; AB123456       ;
XYZ  ;   RAJ   ; CS78890        ;
XXX  ;   MANO  ; MD6546474777   ;

sales_data05-JAN-16.csv
Code:
SDDCCR;  SOM   ; MD6546474777   ;
SDDCCR;  SUJAN ; SA6666464664646;

sales_data06-JAN-16.csv
Code:
ABC   ;  RAMANA; KS566767477747 ;

sales_data08-JAN-16.csv
Code:
SDDCCR;  DINESH; GD6674474747   ;
XYZ  ;   GANE  ; MD6546474777   ;

I have tried the following script but its working.

Code:
#!/bin/ksh 
file_name=sales_data.csv
file_format=$4
awk ' NR==1 {HD = $1 OFS $2  OFS $3; next; next} {FN = DIR"/sales_data"  file_out 
                 if (!(Se[FN]++)) print HD > FN
                 print  $1, $2, $3 >  FN}' FS=' *; *' OFS=";" DIR="$SAL_DIR" file_out="$file_format" $SAL_DIR/$file_name
if [ $? -ne 0 ]
then
  echo "report generation failed"
  fin_anormale $0
fi

#End of File

Please help me.

Thanks.
Moderator's Comments:
Mod Comment Please use code tags


---------- Post updated at 07:04 PM ---------- Previous update was at 06:31 PM ----------

Sorry I forgot.Please help me.

---------- Post updated at 07:27 PM ---------- Previous update was at 07:04 PM ----------

Hi All,

Please help me ASAP.


Thanks.

Last edited by jim mcnamara; 02-19-2016 at 09:32 AM.. Reason: code tags
# 2  
Old 02-19-2016
ASAP is a highly deprecated term in these fora.

---------- Post updated at 16:24 ---------- Previous update was at 16:19 ----------

How about
Code:
awk -F";" '{print > "sales_data" $4 ".csv"}' file

This User Gave Thanks to RudiC For This Post:
# 3  
Old 02-19-2016
Quote:
Originally Posted by RudiC
ASAP is a highly deprecated term in these fora.

---------- Post updated at 16:24 ---------- Previous update was at 16:19 ----------

How about
Code:
awk -F";" '{print > "sales_data" $4 ".csv"}' file

$4 should not be included:

Code:
awk -F\; '{f= "sales_data" $4 ".csv"; $4=""; print > f}' OFS=\; file

# 4  
Old 02-19-2016
hi
Code:
awk 'BEGIN{FS=OFS=";"} FNR==NR{a[$4]=$4;next} $4 in a{print $1 OFS $2 OFS $3 ";" >> "sales_date"a[$4]".csv" }' file file

---------- Post updated at 03:01 PM ---------- Previous update was at 02:30 PM ----------

Quote:
Originally Posted by Franklin52
$4 should not be included:

Code:
awk -F\; '{f= "sales_data" $4 ".csv"; $4=""; print > f}' OFS=\; file

Or
Code:
awk -F";" '{print $1";"$2";"$3 > "sales_data" $4 ".csv"}' file

# 5  
Old 02-23-2016
Hi All,

Thanks a lot to all.

I forgot to mention one thing the sales_data.csv is having heading.
Code:
SCID     NAME     SCODE           CREATE_DATE
SDDCCR;  SOM   ; MD6546474777   ;05-JAN-16
ABC   ;  KIRAN ; CB789          ;04-JAN-16
ABC   ;  RAMANA; KS566767477747 ;06-JAN-16
ABC   ;  KAMESH; A33535335      ;04-JAN-16
SDDCCR;  DINESH; GD6674474747   ;08-JAN-16
SDDCCR;  SUJAN ; SA6666464664646;05-JAN-16
XYZ  ;   AMAR  ; AB123456       ;04-JAN-16
XYZ  ;   RAJ   ; CS78890        ;04-JAN-16
XYZ  ;   GANE  ; MD6546474777   ;08-JAN-16
XXX  ;   MANO  ; MD6546474777   ;04-JAN-16

I need heading for every split file as below.

sales_data04-JAN-16.csv

Code:
SCID     NAME     SCODE  
ABC   ;  KIRAN ; CB789          ;
ABC   ;  KAMESH; A33535335      ;
XYZ  ;   AMAR  ; AB123456       ;
XYZ  ;   RAJ   ; CS78890        ;
XXX  ;   MANO  ; MD6546474777   ;

sales_data05-JAN-16.csv

Code:
SCID     NAME     SCODE        
SDDCCR;  SOM   ; MD6546474777   ;
SDDCCR;  SUJAN ; SA6666464664646;

sales_data06-JAN-16.csv

Code:
SCID     NAME     SCODE    
ABC   ;  RAMANA; KS566767477747 ;

sales_data08-JAN-16.csv
Code:
SCID     NAME     SCODE     
SDDCCR;  DINESH; GD6674474747   ;
XYZ  ;   GANE  ; MD6546474777   ;

My below script is working fine apart from ganerating one more saparate file for heading
and not giving heading in all split files.
Code:
#!/bin/ksh 
file_name=sales_data.csv
awk ' NR==4 {HD = $1 OFS $2  OFS $3; next; next} {FN = DIR"/sales_data_"$4".csv"  file_out 
                 if (!(Se[FN]++)) print HD > FN
                 print  $1, $2, $3 >  FN}' FS=' *; *' OFS=";" DIR="$SAL_DIR" file_out $SAL_DIR/$file_name
if [ $? -ne 0 ]
then
  echo "report generation failed"
  fin_anormale $0
fi

Please help me.

Thanks.

Previous Thread | Next Thread
Test Your Knowledge in Computers #933
Difficulty: Medium
Microcontrollers operating within embedded systems (such as the Raspberry Pi, Arduino, and other similar systems) always have internal hardware to keep track of time.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split file into multiple files based on empty lines

I am using below code to split files based on blank lines but it does not work. awk 'BEGIN{i=0}{RS="";}{x="F"++i;}{print > x;}' Your help would be highly appreciated find attachment of sample.txt file (2 Replies)
Discussion started by: imranrasheedamu
2 Replies

2. Shell Programming and Scripting

Split a big file into multiple files based on first four characters

I have a requirement to split a huge file to smaller text files based on first four characters which look like ABCD 1234 DFGH RREX : : : : : 0000 Each of these records are OF EQUAL bytes with a different internal layout based on the above first digit identifier.. Any help to start... (5 Replies)
Discussion started by: etldev
5 Replies

3. Shell Programming and Scripting

awk script to split file into multiple files based on many columns

So I have a space delimited file that I'd like to split into multiple files based on multiple column values. This is what my data looks like 1bc9A02 1 10 1000 FTDLNLVQALRQFLWSFRLPGEAQKIDRMMEAFAQRYCQCNNGVFQSTDTCYVLSFAIIMLNTSLHNPNVKDKPTVERFIAMNRGINDGGDLPEELLRNLYESIKNEPFKIPELEHHHHHH 1ku1A02 1 10... (9 Replies)
Discussion started by: viored
9 Replies

4. Shell Programming and Scripting

Split single file into multiple files using pattern matching

I have one single shown below and I need to break each ST|850 & SE to separate file using unix script. Below example should create 3 files. We can use ST & SE to filter as these field names will remain same. Please advice with the unix code. ST|850 BEG|PO|1234 LIN|1|23 SE|4 ST|850... (3 Replies)
Discussion started by: prasadm
3 Replies

5. Shell Programming and Scripting

Split a file into multiple files based on field value

Hi, I've one requirement. I have to split one comma delimited file into multiple files based on one of the column values. How can I achieve this Unix Here is the sample data. In this case I have split the files based on date column(c4) Input file c1,c2,c3,c4,c5... (1 Reply)
Discussion started by: manasvi24
1 Replies

6. Shell Programming and Scripting

split XML file into multiple files based on pattern

Hello, I am using awk to split a file into multiple files using command: nawk '{ if ( $1 == "<process" ) { n=split($2, arr, "\""); file=arr } print > file }' processes.xml <process name="Process1.process"> ... (3 Replies)
Discussion started by: chiru_h
3 Replies

7. Shell Programming and Scripting

Split a file into multiple files based on the input pattern

I have a file with lines something like. ...... 123_start ...... ....... 123_end .... ..... 456_start ...... ..... 456_end .... ..... 789_start .... .... 789_end (6 Replies)
Discussion started by: abinash
6 Replies

8. Shell Programming and Scripting

Split the single file lines into multiple files

Let's assume that I have a file name called ĎA' and it has 100 lines in it and would like to split these 100 lines into 4 files as specified bellow. INPUT: Input file name A 1 2 3 4 5 6 7 8 9 ........100 Output: 4 output files (x,y,z,w) File x should contains (Skip 4 lines)... (15 Replies)
Discussion started by: subbarao25
15 Replies

9. Shell Programming and Scripting

Split single file into multiple files based on the number in the column

Dear All, I would like to split a file of the following format into multiple files based on the number in the 6th column (numbers 1, 2, 3...): ATOM 1 N GLY A 1 -3.198 27.537 -5.958 1.00 0.00 N ATOM 2 CA GLY A 1 -2.199 28.399 -6.617 1.00 0.00 ... (3 Replies)
Discussion started by: tomasl
3 Replies

10. UNIX for Dummies Questions & Answers

split a single sql file into multiple files

Hi,I have a single sql file containing many create table ddl's.Example: CREATE TABLE sec_afs ( rpt_per_typ_c char(1) NOT NULL, rpt_per_typ_t varchar(20) NULL, LOCK ALLPAGES go EXEC sp_primarykey 'sec_afs', rpt_per_typ_c go GRANT SELECT ON sec_afs TO developer_read_only... (5 Replies)
Discussion started by: smarter_aries
5 Replies

Featured Tech Videos