Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Script to generate csv file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to generate csv file
# 1  
Old 04-12-2016
Script to generate csv file
Dears,

I am new in shell world and I need your help in this, I have to create a report based on the output file generated by another program. I want to write a shell script for this.
The output file generated every 15 minutes but i can’t open it until the end of day so the script will get the file as an input the file will be as follows :
Code:
PEr;ST100000;ET101500,s1;Si1010;xw129;tr54;vyy87;ANf876;Gk67;kl888
PEr;ST100000;ET101500,s1;Si1020;xw129;tr54;vyy87;ANf876;Gk67;kl888
PEr;ST100000;ET101500,s1;Si1030;xw129;tr54;vyy87;ANf876;Gk67;kl888
PEr;ST100000;ET101500,s1;Si1040;xw129;tr54;vyy87;ANf876;Gk67;kl888
PEr;ET101500;ET103000,s1;Si1010;xw134;tr54;vyy87;ANf845;Gk63;kl882
PEr;ET101500;ET103000,s1;Si1020;xw169;tr34;vyy87;ANf576;Gk62;kl883
PEr;ET101500;ET103000,s1;Si1030;xw179;tr24;vyy87;ANf676;Gk62;kl884
PEr;ET101500;ET103000,s1;Si1040;xw189;tr94;vyy87;ANf764;Gk63;kl885
PEr;ET103000;ET104500,s1;Si1010;xw109;tr84;vyy87;ANf976;Gk67;kl888
PEr;ET103000;ET104500,s1;Si1020;xw129;tr74;vyy87;ANf776;Gk67;kl888
PEr;ET103000;ET104500,s1;Si1030;xw139;tr64;vyy87;ANf876;Gk67;kl888
PEr;ET103000;ET104500,s1;Si1040;xw125;tr54;vyy87;ANf876;Gk67;kl888

the expected result
Code:
Time:101500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;129;54;87;876;67;888
1020;129;54;87;876;67;888
1030;129;54;87;876;67;888
1040;129;54;87;876;67;888

Time:103000
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;134;54;87;845;63;l882
1020;169;34;87;576;62;l883
1030;179;24;87;676;62;l884
1040;189;94;87;764;63;l885

Last edited by vgersh99; 04-12-2016 at 12:37 PM.. Reason: code tags, please!
# 2  
Old 04-12-2016
Try
Code:
awk '
                {gsub (/[^0-9;]/, _)
                 $3 = substr ($3, 1, length ($3) - 1)
                }
$3 != LAST      {print "Time: " $3
                 print "Item_Nu;xw;tr;vyy;ANf;Gk;kl"
                 LAST = $3
                }
                {print $4, $5, $6, $7, $8, $9, $10
                }
' FS=";" OFS=";" file
Time: 101500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;129;54;87;876;67;888
1020;129;54;87;876;67;888
1030;129;54;87;876;67;888
1040;129;54;87;876;67;888
Time: 103000
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;134;54;87;845;63;882
1020;169;34;87;576;62;883
1030;179;24;87;676;62;884
1040;189;94;87;764;63;885
Time: 104500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;109;84;87;976;67;888
1020;129;74;87;776;67;888
1030;139;64;87;876;67;888
1040;125;54;87;876;67;888

This User Gave Thanks to RudiC For This Post:
abdul2020
# 3  
Old 04-12-2016
Hello abdul2020,

If you are not worried about the sequence of the time values then following may help you in same too.
Code:
awk -F"[;|,]" '{gsub(/[[:alpha:]]/,X,$0);Q=$3;gsub(/.*s1|^;/,X,$0);A[Q]=A[Q]?A[Q] ORS $0:$0;}END{for(i in A){print "Time" i ORS "Item_Nu;xw;tr;vyy;ANf;Gk;kl" ORS A[i]}}'  Input_file

Output will be as follows.
Code:
Time101500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
100000;101500,1;1010;129;54;87;876;67;888
100000;101500,1;1020;129;54;87;876;67;888
100000;101500,1;1030;129;54;87;876;67;888
100000;101500,1;1040;129;54;87;876;67;888
Time104500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
103000;104500,1;1010;109;84;87;976;67;888
103000;104500,1;1020;129;74;87;776;67;888
103000;104500,1;1030;139;64;87;876;67;888
103000;104500,1;1040;125;54;87;876;67;888
Time103000
Item_Nu;xw;tr;vyy;ANf;Gk;kl
101500;103000,1;1010;134;54;87;845;63;882
101500;103000,1;1020;169;34;87;576;62;883
101500;103000,1;1030;179;24;87;676;62;884
101500;103000,1;1040;189;94;87;764;63;885

EDIT: Adding a non one-liner form for solution on same.
Code:
awk -F"[;|,]" '{
                gsub(/[[:alpha:]]/,X,$0);
                Q=$3;
                gsub(/.*s1|^;/,X,$0);
                A[Q]=A[Q]?A[Q] ORS $0:$0;
               }
                END{
                        for(i in A){
                                        print "Time" i ORS "Item_Nu;xw;tr;vyy;ANf;Gk;kl" ORS A[i]
                                   }
               }
              '   Input_file

Thanks,
R. Singh
Last edited by RavinderSingh13; 04-12-2016 at 03:12 PM.. Reason: Added a non-one liner form of solution now.
This User Gave Thanks to RavinderSingh13 For This Post:
abdul2020
# 4  
Old 04-13-2016
Quote:
Originally Posted by RudiC
Try
Code:
awk '
                {gsub (/[^0-9;]/, _)
                 $3 = substr ($3, 1, length ($3) - 1)
                }
$3 != LAST      {print "Time: " $3
                 print "Item_Nu;xw;tr;vyy;ANf;Gk;kl"
                 LAST = $3
                }
                {print $4, $5, $6, $7, $8, $9, $10
                }
' FS=";" OFS=";" file
Time: 101500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;129;54;87;876;67;888
1020;129;54;87;876;67;888
1030;129;54;87;876;67;888
1040;129;54;87;876;67;888
Time: 103000
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;134;54;87;845;63;882
1020;169;34;87;576;62;883
1030;179;24;87;676;62;884
1040;189;94;87;764;63;885
Time: 104500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
1010;109;84;87;976;67;888
1020;129;74;87;776;67;888
1030;139;64;87;876;67;888
1040;125;54;87;876;67;888

Thank you so much for help.Once I tried it in the relay file I failed to edit it because I'm still learning . here sample of the real logs, also it removed the comma in the last five tags LMc ,LMv,LMg,RMv and RMg.

Code:
PEr;n534385;i123;Si4300;s1;PSt123000;PSp124500;EOdN;WAs0.05;RWa0.9852;WBb5.05;WBa5.1;WVq5.0596;BWs5.3088;BWb4.9765;TVs14650389.4344;TVb10501152.6287;BAv1088536.2633;BAu5551534.9429;NBa19.0832;AAv14650389.4344;AAu77776460.6215;ABa1047.6402;BBv4555746.6461;BBu23006520.5626;NBb215.3979;ABv10501152.6287;ABu52259658.6045;ABb587.6587;BMv2822141.4547;BMu14279027.7528;NBm117.2406;AMv12575771.0315;AMu65018059.613;ABm817.6495;OCt100;OCb100;OCa100;BAb76.0958;BAa-28.3254;CIb43.387;CIa7.4302;LMc5000,25000,50000,100000,250000,500000,750000,1000000,2000000,3000000;LMv99,99,99,99,99,99,99,99,99,99;LMg100,100,100,100,100,100,100,100,100,100;RMc5000,25000,50000,100000,250000,500000,750000,1000000,2000000,3000000;RMv99,99,99,99,99,99,99,99,99,99;RMg100,100,100,100,100,100,100,100,100,100;


Moderator's Comments:
Mod Comment Please use code tags as required by forum rules!



---------- Post updated at 05:51 AM ---------- Previous update was at 05:50 AM ----------

Quote:
Originally Posted by RavinderSingh13
Hello abdul2020,

If you are not worried about the sequence of the time values then following may help you in same too.
Code:
awk -F"[;|,]" '{gsub(/[[:alpha:]]/,X,$0);Q=$3;gsub(/.*s1|^;/,X,$0);A[Q]=A[Q]?A[Q] ORS $0:$0;}END{for(i in A){print "Time" i ORS "Item_Nu;xw;tr;vyy;ANf;Gk;kl" ORS A[i]}}'  Input_file

Output will be as follows.
Code:
Time101500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
100000;101500,1;1010;129;54;87;876;67;888
100000;101500,1;1020;129;54;87;876;67;888
100000;101500,1;1030;129;54;87;876;67;888
100000;101500,1;1040;129;54;87;876;67;888
Time104500
Item_Nu;xw;tr;vyy;ANf;Gk;kl
103000;104500,1;1010;109;84;87;976;67;888
103000;104500,1;1020;129;74;87;776;67;888
103000;104500,1;1030;139;64;87;876;67;888
103000;104500,1;1040;125;54;87;876;67;888
Time103000
Item_Nu;xw;tr;vyy;ANf;Gk;kl
101500;103000,1;1010;134;54;87;845;63;882
101500;103000,1;1020;169;34;87;576;62;883
101500;103000,1;1030;179;24;87;676;62;884
101500;103000,1;1040;189;94;87;764;63;885

EDIT: Adding a non one-liner form for solution on same.
Code:
awk -F"[;|,]" '{
                gsub(/[[:alpha:]]/,X,$0);
                Q=$3;
                gsub(/.*s1|^;/,X,$0);
                A[Q]=A[Q]?A[Q] ORS $0:$0;
               }
                END{
                        for(i in A){
                                        print "Time" i ORS "Item_Nu;xw;tr;vyy;ANf;Gk;kl" ORS A[i]
                                   }
               }
              '   Input_file

Thanks,
R. Singh
Thanks for help, the time sequence is important
Last edited by RudiC; 04-13-2016 at 09:28 AM.. Reason: Added code tags (again!)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generate .csv/ xls file report

There can be thousand of .ksh in a specific directory where sql files are called from ksh. Requirement is to loop through all the files content and generate a report like below: Jobname Type type sqlname gemd1970 sql daily tran01 gemw1971 sql weekly ... (6 Replies)
Discussion started by: vedanta
6 Replies

2. Shell Programming and Scripting

Script to generate .csv file

Dears,I need your help in this, I have to create a report based on the output file generated by another program. I want to write a shell script for this. The output file generated every 15 minutes but i can’t open it until the end of day so the script will get the file as an input the file will be... (8 Replies)
Discussion started by: abdul2020
8 Replies

3. Shell Programming and Scripting

BASH script to parse XML and generate CSV

Hi All, Hope all you are doing good! Need your help. I have an XML file which needs to be converted CSV file. I am not an expert of awk/sed so your help is highly appreciated!! XML file looks like this: <l:event dateTime="2013-03-13 07:15:54.713" layerName="OSB" processName="ABC"... (2 Replies)
Discussion started by: bhaskar_m
2 Replies

4. Shell Programming and Scripting

Script to generate csv file

Hello; I need to generate a csv file that contains a list of all the files in a particular server (from the root directory ie: \) that have a permission stamp of 777. I would like to create the csv so that it contains the following: server name, file name, full path name where file exists,... (17 Replies)
Discussion started by: gvolpini
17 Replies

5. Shell Programming and Scripting

Read a CSV file and generate SQL output

Friends, This is what I need: I will pass a CSV file as an input, and I want my shell to be reading that CSV file, and based on the parameters it should generate SQLs and write those SQL in a different file in the same location. I'm new to Shell scripting. I'm currently working on a... (25 Replies)
Discussion started by: Ram.Math
25 Replies

6. Shell Programming and Scripting

to read a CSV file and generate SQL output

Friends, This is what I need: I will pass a CSV file as an input, and I want my shell to be reading that CSV file, and based on the parameters it should generate SQLs and write those SQL in a different file in the same location. I'm new to Shell scripting. I'm currently working on a... (1 Reply)
Discussion started by: Ram.Math
1 Replies

7. Shell Programming and Scripting

Need to generate .csv file

I have a csv file with the following data Please find the attachment - zip ... (6 Replies)
Discussion started by: vaas
6 Replies

8. Shell Programming and Scripting

need help in Parsing a CSV file and generate a new output file

Hi Scripting Gurus, I am trying to parse a csv file and generate a new output file. The input file will be a variable length in turns of rows and columns. output file will have 8 columns. we have three columns from the header for each set. just to give little bit more clarification each row... (15 Replies)
Discussion started by: vkr
15 Replies

9. UNIX for Dummies Questions & Answers

generate CSV file using AWK script

Hi guys I have a text report that consists of text in some parts and data in some parts. e.g Report for changes in cashflows No changes were found Report for changes in Bills deal_num deal_date trader maturity log_creator DF_234 20-5-2008 tman 20-5-2009 tman... (2 Replies)
Discussion started by: magikminox
2 Replies

10. Shell Programming and Scripting

Generate csv file

I have a file which has some thousand records in the following format File: input.txt -> <option value="14333">VISWANADH VELAMURI</option> <option value="17020">VISWANADHA RAMA KRISHNA</option> I want to generate a csv file from the above file as follows File: output.txt -> ... (4 Replies)
Discussion started by: rahulrathod
4 Replies
Login or Register to Ask a Question