Sponsored Content
Top Forums Shell Programming and Scripting need help in Parsing a CSV file and generate a new output file Post 302218823 by aigles on Sunday 27th of July 2008 05:50:41 AM
Old 07-27-2008
Try and adapt the following script :
Code:
awk '

BEGIN {
  OFS = FS = ",";
  Columns_by_set = 4;
  Infos_by_set   = 3; # Output columns count from header for each set
  print "Date","index_id","currency","index_pro",
               "level","net","total","mcap";
}

$1 == "Header" {
   Sets_by_line   = int((NF -1) / Columns_by_set);
   Fields_by_line = NF;

   for (set=1; set<=Sets_by_line; set++) {
      infos = "";
      field = 2 + (set-1)*Columns_by_set
      for (col=1; col<=Infos_by_set; col++) {
         infos = infos $(field+col-1) (col<Infos_by_set ? OFS : "");
      }
      Header_infos[set] = infos;
   }

   next;
}

$1 == "Date" {
   next;
}

Sets_by_line {
   if (NF != Fields_by_line) {
      printf("Invalid datas line %d: Fields count is %d, must be %d\n",
             NR, NF, Fields_by_line);
      next;
   }

   for (set=1; set<=Sets_by_line; set++) {
      datas = $1 OFS Header_infos[set];
      field = 2 + (set-1)*Columns_by_set
      for (col=1; col<=Infos_by_set; col++) {
         datas = datas OFS $(field+col-1);
      }
      print datas
   }
}
' inputfile

Inputfile:
Code:
Header,4567,USD,BMI,,4568,USD,BMI,
Date,LEVEL,NET,TOTAL,MCAP,LEVEL,NET,TOTAL,MCAP
22-Dec-97,27.06492573,,20.9289574,,26.17032566,,20.60702136,
23-Dec-97,27.11627283,,20.96866346,,26.50090757,,20.86732796,
24-Dec-97,27.17448579,,21.01367878,,26.53347955,,20.89299111,
25-Dec-97,27.17854775,,21.02091981,,26.66484879,,20.99775394,
26-Dec-97,27.13228338,,20.98513719,,26.55463469,,20.91096371,
Header,9876,EUR,BMI,
Date,LEVEL,NET,TOTAL,MCAP
22-Dec-97,27.06492573,,20.9289574,
23-Dec-97,27.11627283,,20.96866346,
24-Dec-97,27.17448579,,21.01367878,
25-Dec-97,27.17854775,,21.02091981,
26-Dec-97,27.13228338,,20.98513719,

Output:
Code:
Date,index_id,currency,index_pro,level,net,total,mcap
22-Dec-97,4567,USD,BMI,27.06492573,,20.9289574
22-Dec-97,4568,USD,BMI,26.17032566,,20.60702136
23-Dec-97,4567,USD,BMI,27.11627283,,20.96866346
23-Dec-97,4568,USD,BMI,26.50090757,,20.86732796
24-Dec-97,4567,USD,BMI,27.17448579,,21.01367878
24-Dec-97,4568,USD,BMI,26.53347955,,20.89299111
25-Dec-97,4567,USD,BMI,27.17854775,,21.02091981
25-Dec-97,4568,USD,BMI,26.66484879,,20.99775394
26-Dec-97,4567,USD,BMI,27.13228338,,20.98513719
26-Dec-97,4568,USD,BMI,26.55463469,,20.91096371
22-Dec-97,9876,EUR,BMI,27.06492573,,20.9289574
23-Dec-97,9876,EUR,BMI,27.11627283,,20.96866346
24-Dec-97,9876,EUR,BMI,27.17448579,,21.01367878
25-Dec-97,9876,EUR,BMI,27.17854775,,21.02091981
26-Dec-97,9876,EUR,BMI,27.13228338,,20.98513719

Jean-Pierre.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generate csv file

I have a file which has some thousand records in the following format File: input.txt -> <option value="14333">VISWANADH VELAMURI</option> <option value="17020">VISWANADHA RAMA KRISHNA</option> I want to generate a csv file from the above file as follows File: output.txt -> ... (4 Replies)
Discussion started by: rahulrathod
4 Replies

2. Shell Programming and Scripting

2 problems: Mailing CSV file / parsing CSV for display

I have been trying to find a good solution for this seemingly simple task for 2 days, and I'm giving up and posting a thread. I hope someone can help me out! I'm on HPUX, using sqlplus, mailx, awk, have some other tools available, but can't install stuff that isn't already in place (without a... (6 Replies)
Discussion started by: soldstatic
6 Replies

3. Shell Programming and Scripting

Need to generate .csv file

I have a csv file with the following data Please find the attachment - zip ... (6 Replies)
Discussion started by: vaas
6 Replies

4. Shell Programming and Scripting

to read a CSV file and generate SQL output

Friends, This is what I need: I will pass a CSV file as an input, and I want my shell to be reading that CSV file, and based on the parameters it should generate SQLs and write those SQL in a different file in the same location. I'm new to Shell scripting. I'm currently working on a... (1 Reply)
Discussion started by: Ram.Math
1 Replies

5. Shell Programming and Scripting

Read a CSV file and generate SQL output

Friends, This is what I need: I will pass a CSV file as an input, and I want my shell to be reading that CSV file, and based on the parameters it should generate SQLs and write those SQL in a different file in the same location. I'm new to Shell scripting. I'm currently working on a... (25 Replies)
Discussion started by: Ram.Math
25 Replies

6. Shell Programming and Scripting

Script to generate csv file

Hello; I need to generate a csv file that contains a list of all the files in a particular server (from the root directory ie: \) that have a permission stamp of 777. I would like to create the csv so that it contains the following: server name, file name, full path name where file exists,... (17 Replies)
Discussion started by: gvolpini
17 Replies

7. Shell Programming and Scripting

Save output of updated csv file as csv file itself, part 2

Hi, I have another problem. I want to sort another csv file by the first field. result.csv SourceFile,Airspeed,GPSLatitude,GPSLongitude,Temperature,Pressure,Altitude,Roll,Pitch,Yaw /home/intannf/foto5/2015_0313_090651_219.JPG,0.,-7.77223,110.37310,30.75,996.46,148.75,180.94,182.00,63.92 ... (2 Replies)
Discussion started by: refrain
2 Replies

8. Shell Programming and Scripting

Generate .csv file a text file

Hi guys, I have a text file that states: cat prod.hosts 11.29.130.14 host1 host1.test.com web17 11.29.130.15 host2 host2.test.com web18 11.29.130.16 host3 host3.test.com web19 I want a .csv file that states: ... (2 Replies)
Discussion started by: Junaid Subhani
2 Replies

9. Shell Programming and Scripting

Script to generate csv file

Dears, I am new in shell world and I need your help in this, I have to create a report based on the output file generated by another program. I want to write a shell script for this. The output file generated every 15 minutes but i can’t open it until the end of day so the script will get the... (3 Replies)
Discussion started by: abdul2020
3 Replies

10. Shell Programming and Scripting

Script to generate .csv file

Dears,I need your help in this, I have to create a report based on the output file generated by another program. I want to write a shell script for this. The output file generated every 15 minutes but i can’t open it until the end of day so the script will get the file as an input the file will be... (8 Replies)
Discussion started by: abdul2020
8 Replies
Net::DNS::Header(3)					User Contributed Perl Documentation				       Net::DNS::Header(3)

NAME
Net::DNS::Header - DNS packet header class SYNOPSIS
"use Net::DNS::Header;" DESCRIPTION
A "Net::DNS::Header" object represents the header portion of a DNS packet. METHODS
new $header = Net::DNS::Header->new; $header = Net::DNS::Header->new($data); Without an argument, "new" creates a header object appropriate for making a DNS query. If "new" is passed a reference to a scalar containing DNS packet data, it creates a header object from that data. Returns undef if unable to create a header object (e.g., if the data is incomplete). print $header->print; Dumps the header data to the standard output. string print $header->string; Returns a string representation of the header object. id print "query id = ", $header->id, " "; $header->id(1234); Gets or sets the query identification number. qr print "query response flag = ", $header->qr, " "; $header->qr(0); Gets or sets the query response flag. opcode print "query opcode = ", $header->opcode, " "; $header->opcode("UPDATE"); Gets or sets the query opcode (the purpose of the query). aa print "answer is ", $header->aa ? "" : "non-", "authoritative "; $header->aa(0); Gets or sets the authoritative answer flag. tc print "packet is ", $header->tc ? "" : "not ", "truncated "; $header->tc(0); Gets or sets the truncated packet flag. rd print "recursion was ", $header->rd ? "" : "not ", "desired "; $header->rd(0); Gets or sets the recursion desired flag. cd print "checking was ", $header->cd ? "not" : "", "desired "; $header->cd(0); Gets or sets the checking disabled flag. ra print "recursion is ", $header->ra ? "" : "not ", "available "; $header->ra(0); Gets or sets the recursion available flag. rcode print "query response code = ", $header->rcode, " "; $header->rcode("SERVFAIL"); Gets or sets the query response code (the status of the query). qdcount, zocount print "# of question records: ", $header->qdcount, " "; $header->qdcount(2); Gets or sets the number of records in the question section of the packet. In dynamic update packets, this field is known as "zocount" and refers to the number of RRs in the zone section. ancount, prcount print "# of answer records: ", $header->ancount, " "; $header->ancount(5); Gets or sets the number of records in the answer section of the packet. In dynamic update packets, this field is known as "prcount" and refers to the number of RRs in the prerequisite section. nscount, upcount print "# of authority records: ", $header->nscount, " "; $header->nscount(2); Gets or sets the number of records in the authority section of the packet. In dynamic update packets, this field is known as "upcount" and refers to the number of RRs in the update section. arcount, adcount print "# of additional records: ", $header->arcount, " "; $header->arcount(3); Gets or sets the number of records in the additional section of the packet. In dynamic update packets, this field is known as "adcount". data $hdata = $header->data; Returns the header data in binary format, appropriate for use in a DNS query packet. COPYRIGHT
Copyright (c) 1997-2000 Michael Fuhr. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
perl(1), Net::DNS, Net::DNS::Resolver, Net::DNS::Packet, Net::DNS::Update, Net::DNS::Question, Net::DNS::RR, RFC 1035 Section 4.1.1 perl v5.8.0 2002-10-12 Net::DNS::Header(3)
All times are GMT -4. The time now is 09:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy