Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Split a file into two different file using shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split a file into two different file using shell script
# 1  
Old 03-02-2016
Code Split a file into two different file using shell script
could you please provide solution to split file and load into different files in shell script.

Code:
cat test.txt
"01"|""emp_name"":ram|""emp_sal"":600000|""emp_value"":""CREDITCARD""|""1410"":[[100
0.0]]
"02"|""emp_name"":syam|""emp_sal"":500000|""emp_value"":""voucher""|""1610"":[[1000.
0]]|""1510"":[[10.0]]
"03"|""emp_sal"":400|""emp_value"":""cr""|""1910"":[[50.0]]|""1860"":[[60.0]]
"04"|""emp_name"":karan|""emp_value"":""cr""|""1810"":[[50.0]]|""1530"":[[60.0]]
"05"|""emp_name"":kumar|""emp_sal"":20000||""2010"":[[50.0]]|""1530"":[[60.0]]

the output file should be
as given below
Code:
cat ouput1.txt
IDN|emp_name|emp_sal|emp_value
01|ram|600000| CREDITCARD
02|syam|500000|Voucher
03|XXXX|400|cr
04|Karan|XXXX|cr
05|Kumar|2000|XXXX
$
 
cat ouput2.txt
IDN|cust_com|cust_value
01|1410|1000.0
02|1610|1000.00
02|1510|10.0
03|1910|50.0
03|1860|60.0
04|1810|50.0
04|1530|60.0
05|2010|50.0
05|1530|60.0

Last edited by vbe; 03-02-2016 at 09:34 AM.. Reason: replaced fancy fonts + code tags
# 2  
Old 03-02-2016
Any attempts/thoughts/ideas from your side? And, please use code tags as required by forum rules.
# 3  
Old 03-02-2016
Unclear what rules
From your example, it is not clear what the rules of the "split" would be. Please describe what you are doing.
# 4  
Old 03-02-2016
try:
Code:
awk '
{gsub("\"","");}
NR==1 {
   l="IDN|";
   for (i=2; i<=4; i++) {
      t=$i;
      sub(":.*", "", t);
      l=l t ((i==4) ? "" : "|");
   }
   print l > "outout1.txt";
   print "IDN|cust_com|cust_value" > "outout2.txt";
}
{
   enf=esf=evf=vf=0;
   for (i=2; i<=NF; i++) {
     if ($i ~ /emp_name/) enf=i;
     if ($i ~ /emp_sal/) esf=i;
     if ($i ~ /emp_value/) evf=i;
     if (! vf && $i) if ($i !~ /[a-zA-Z]/) vf=i;
   }
   en=($enf ~ /emp_name/) ? $enf : "XXXX";
   sub(".*:", "", en);
   es=($esf ~ /emp_sal/) ? $esf : "XXXX";
   sub(".*:", "", es);
   ev=($evf ~ /emp_value/) ? $evf : "XXXX";
   sub(".*:", "", ev);
   print $1,en,es,ev > "outout1.txt";
   c=$1 "|";
   for (i=vf; i<=NF; i++) {
      gsub("[[]", "", $i);
      gsub("[]]", "", $i);
      s=t=$i;
      sub(":.*", "", t);
      sub(".*:", "", s);
      print $1,t,s > "outout2.txt";
   }
}
' FS="|" OFS="|" text.txt

This User Gave Thanks to rdrtx1 For This Post:
shabeena
# 5  
Old 03-03-2016
Hi RDRTX,

Thanks for your reply Smilie

its working , but there is some discrepancy in the output of the first record. the first record in the file1 should be "01|ram|600000| CREDITCARD " and in outfile2 should be "01|1410|1000.0" but the script you are provided printing the first record improperly. please find the following result of your script.

PFB output.

Code:
cat outout1.txt
IDN|emp_sal|emp_value|1410
me:ram|XXXX|600000|CREDITCARD
02|syam|500000|voucher
03|XXXX|400|cr
04|karan|XXXX|cr
05|kumar|20000|XXXX

Code:
cat outout2.txt
IDN|cust_com|cust_value
me:ram|1410|1000.0
02|1610|1000.0
02|1510|10.0
03|1910|50.0
03|1860|60.0
04|1810|50.0
04|1530|60.0
05|2010|50.0
05|1530|60.0

Last edited by shabeena; 03-03-2016 at 05:09 AM..
# 6  
Old 03-03-2016
I thought the post of the input file was off. To fix try:
Code:
awk '{printf $0 (($0 ~ /[]] *$/) ? "\n" : "")}' text.txt |awk '
{gsub("\"","");}
NR==1 {
   l="IDN|";
   for (i=2; i<=4; i++) {
      t=$i;
      sub(":.*", "", t);
      l=l t ((i==4) ? "" : "|");
   }
   print l > "outout1.txt";
   print "IDN|cust_com|cust_value" > "outout2.txt";
}
{
   enf=esf=evf=vf=0;
   for (i=2; i<=NF; i++) {
     if ($i ~ /emp_name/) enf=i;
     if ($i ~ /emp_sal/) esf=i;
     if ($i ~ /emp_value/) evf=i;
     if (! vf && $i) if ($i !~ /[a-zA-Z]/) vf=i;
   }
   en=($enf ~ /emp_name/) ? $enf : "XXXX";
   sub(".*:", "", en);
   es=($esf ~ /emp_sal/) ? $esf : "XXXX";
   sub(".*:", "", es);
   ev=($evf ~ /emp_value/) ? $evf : "XXXX";
   sub(".*:", "", ev);
   print $1,en,es,ev > "outout1.txt";
   c=$1 "|";
   for (i=vf; i<=NF; i++) {
      gsub("[[]", "", $i);
      gsub("[]]", "", $i);
      s=t=$i;
      sub(":.*", "", t);
      sub(".*:", "", s);
      print $1,t,s > "outout2.txt";
   }
}
' FS="|" OFS="|"

This User Gave Thanks to rdrtx1 For This Post:
shabeena
# 7  
Old 03-04-2016
Hi Rdrtx,

Still iam facing same issue. First record getting incorrect value for the both output.PFB error

Output1:
Code:
IDN|emp_sal|emp_value[/B]|1410
me:ram|XXXX|600000|CREDITCARD

Output2:
Code:
IDN|cust_com|cust_value
me:ram|1410|1000.0


---------- Post updated at 12:41 AM ---------- Previous update was at 12:41 AM ----------

Hi Rdrtx,

Thank you very much. its working fine Smilie

Appreciate your all efforts
Last edited by Don Cragun; 03-04-2016 at 11:21 PM.. Reason: Change FONT, SIZE, and B tags to CODE and ICODE tags. Remove lots of FONT and SIZE tags.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Shell script to Split matrix file with delimiter into multiple files

I have a large semicolon delimited file with thousands of columns and many thousands of line. It looks like: ID1;ID2;ID3;ID4;A_1;B_1;C_1;A_2;B_2;C_2;A_3;B_3;C_3 AA;ax;ay;az;01;02;03;04;05;06;07;08;09 BB;bx;by;bz;03;05;33;44;15;26;27;08;09 I want to split this table in to multiple files: ... (1 Reply)
Discussion started by: trymega
1 Replies

2. Shell Programming and Scripting

Shell script (sh file) logic to compare contents of one file with another file and output to file

Shell script logic Hi I have 2 input files like with file 1 content as (file1) "BRGTEST-242" a.txt "BRGTEST-240" a.txt "BRGTEST-219" e.txt File 2 contents as fle(2) "BRGTEST-244" a.txt "BRGTEST-244" b.txt "BRGTEST-231" c.txt "BRGTEST-231" d.txt "BRGTEST-221" e.txt I want to get... (22 Replies)
Discussion started by: pottic
22 Replies

3. Shell Programming and Scripting

Help with Shell Scrip in Masking particular columns in .csv file or .txt file using shell script

Hello Unix Shell Script Experts, I have a script that would mask the columns in .csv file or .txt file. First the script will untar the .zip files from Archive folder and processes into work folder and finally pushes the masked .csv files into Feed folder. Two parameters are passed ... (5 Replies)
Discussion started by: Mahesh G
5 Replies

4. Shell Programming and Scripting

Shell code for split/merge the file with certain number of columns

i have a file with some number of colums and each row have different number of fields. now my target is supposed to be, each row should have same number of columns. example: src file: 111,S3mobile,Samsu ng 112,Lu mia,Nok ia 113,brav ia,Sonyerichson tgt file: 111,S3mobile,Samsung... (8 Replies)
Discussion started by: abhilash_nakka
8 Replies

5. Shell Programming and Scripting

Split file based on file size in Korn script

I need to split a file if it is over 2GB in size (or any size), preferably split on the lines. I have figured out how to get the file size using awk, and I can split the file based on the number of lines (which I got with wc -l) but I can't figure out how to connect them together in the script. ... (6 Replies)
Discussion started by: ssemple2000
6 Replies

6. Shell Programming and Scripting

KSH script for split a txt file

I have a problem which I would like to solve by using UNIX power and inspired minds around world. Here is the problem I have a text file and it has data as follows 1X.....................1234567890123456789T1234598765XT1 (header) 1Z01............(sub HEADER) P100001............ Q1........... (4 Replies)
Discussion started by: ask.chowhan
4 Replies

7. Shell Programming and Scripting

[Solved] Script to split a file into two

Hi i have a file like a 12 b 13 c 14 d 15 I want to split it based on a blank line like in first file I should have a 12 b 13 and in the second file I have c 14 d 15 How can i do this? Any help will be greatly appreciated (5 Replies)
Discussion started by: prarat
5 Replies

8. Shell Programming and Scripting

how to get split output of a file, using perl script

Hi, I have file: data.log.1 ### s1 main.build.3495 main.build.199 main.build.3408 ###s2 main.build.3495 main.build.3408 main.build.199 I want to read this file and store in two arrays in Perl. I have following command, which is working fine on command prompt. perl -n -e... (1 Reply)
Discussion started by: ashvini
1 Replies

9. Shell Programming and Scripting

Shell script to split XML file

Hi, I'm experiencing difficulty in loading an XML file to an Oracle destination table.I keep running into a memory problem due to the large size of the file. I want to split the XML file into several smaller files based on the keyword(s)/tags : '' and '' and would like to use a Unix shell... (2 Replies)
Discussion started by: bayflash27
2 Replies

10. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies
Login or Register to Ask a Question