Sponsored Content
Top Forums Shell Programming and Scripting Need a script to parse data and output to csv Post 303006464 by sidnow on Thursday 2nd of November 2017 11:55:52 AM
Old 11-02-2017
Need a script to parse data and output to csv

I am not too savvy with arrays and am assuming that what I am looking for needs arrays. This is my requirement.



So I have the raw data that gets updated to a log as shown below

Code:
[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = Newark
    Age = 20
    Name= John
}

[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = NYC
    Age = 22
    Name = Mary
}


There are blocks of data in the sense, a block starts from '[' and closes at '}'. So from the above there are 2 blocks.

The fields are what appears between '{' and '}'. These fields are not fixed, in the sense, i will run the script and supply the fields, sometimes I will supply say 3 fields - Address, Age and Name and other times more or less.

Can a script be made to parse this data and output the data into csv?

So the output for the above would look as

Code:
DateTime                  Address   Age     Name
20171102 10:00:00  Newark    20        John
20171102 10:00:00  NYC         22        Mary


If i had say another field supplied say Subject, then the output would be


Code:
[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = Newark
    Age = 20
    Name = John
    Subject = Math
}

[20171102 10:00:00] StudentInfo:
FullInfo = {
    Address = NYC
    Age = 22
    Name = Mary
    Subject = Literature
}

I would the output in csv as

Code:
DateTime                  Address   Age     Name    Subject
20171102 10:00:00  Newark    20        John     Math
20171102 10:00:00  NYC         22        Mary     Literature


Please can some one advise. The farthest I have got is doing awk and parsing the data, but I dont know how to call it by block and the align it in columns
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unix Script to parse a CSV

I am writing a unix script that will parse a CSV and edit the values. My CSV looks like this 0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,2,0,0,0,0,0,0 10,11,7,0,4,12,2,3,7,0,11,3,12,4,0,5,5,4,5,0,8,6,12,0,9,3,3,0,2,7,8... (16 Replies)
Discussion started by: RJ17
16 Replies

2. Shell Programming and Scripting

parse csv file, sha1 hash and output

I have a file, not really a csv, but containing delineated data just the same. Lets call that file "raw_data.txt". It contains data in the format of company name:fein number like this: first company name:123456789 second company name:987654321 what i need to do is read this file, apply... (11 Replies)
Discussion started by: FreddyG
11 Replies

3. Shell Programming and Scripting

select data from oracle table and save the output as csv file

Hi I need to execute a select statement in a solaris environment with oracle database. The select statement returns number of rows of data. I need the data to be inserted into a CSV file with proper format. For that we normally use "You have to select all your columns as one big string,... (2 Replies)
Discussion started by: rdhanek
2 Replies

4. Shell Programming and Scripting

Extract and parse XML data (statistic value) to csv

Hi All, I need to parse some statistic data from the "measInfo" -eg. 25250000 (as highlighted) and return the result into line by line, and erasing all other unnecessary info/tag. Thought of starting with grep "measInfoID="25250000" but this only returns 1 line. How do I get all the output... (8 Replies)
Discussion started by: jackma
8 Replies

5. UNIX for Dummies Questions & Answers

Help to parse csv file with shell script

Hello ! I am very aware that this is not the first time this question is asked here, because I have already read a lot of previous answers, but none of them worked, so... As said in the title, I want to read a csv file with a bash script. Here is a sample of the file: ... (4 Replies)
Discussion started by: Grhyll
4 Replies

6. Shell Programming and Scripting

BASH script to parse XML and generate CSV

Hi All, Hope all you are doing good! Need your help. I have an XML file which needs to be converted CSV file. I am not an expert of awk/sed so your help is highly appreciated!! XML file looks like this: <l:event dateTime="2013-03-13 07:15:54.713" layerName="OSB" processName="ABC"... (2 Replies)
Discussion started by: bhaskar_m
2 Replies

7. Shell Programming and Scripting

Output large volume of data to CSV file

I have a program that output the ownership and permission on each directory and file on the server to a csv file. I am getting error message when I run the program. The program is not outputting to the csv file. Error: the file access permissions do not allow the specified action cannot... (2 Replies)
Discussion started by: dellanicholson
2 Replies

8. Shell Programming and Scripting

How to parse this file using awk and output in CSV format?

My source file looks like this: Cust-Number = "101" Cust-Name="Joe" Cust-Town="London" Cust-hobby="tennis" Cust-purchase="200" Cust-Number = "102" Cust-Name="Mary" Cust-Town="Newyork" Cust-hobby="reading" Cust-purchase="125" Now I want to parse this file (leaving out hobby) and... (10 Replies)
Discussion started by: Balav
10 Replies

9. Shell Programming and Scripting

Copy data to CSV file from txt output

Hi Everyone , Below is output from a TL1 , I want just the NE Name: and beside that the Temperature and the voltages in a csv file , Is this possible? > act-user:AB1S2a:ArshadFO:493::**********; AB1S2a 2016-02-07 10:13:24 M 493 COMPLD "ArshadFO:2016-02-07 10-04-55,0" ;... (11 Replies)
Discussion started by: adgjmpt
11 Replies

10. Programming

Python script to run multiple command and append data in output csv file

Experts, I am writing a script and able to write only small piece of code and not able to collect logic to complete this task. In input file have to look for name like like this (BGL_HSR_901_1AG_A_CR9KTR10) before sh iss neors. Record this (BGL_HSR_901_1AG_A_CR9KTR10) in csv file Now have to... (0 Replies)
Discussion started by: as7951
0 Replies
Address(3pm)						User Contributed Perl Documentation					      Address(3pm)

NAME
Palm::Address - Handler for Palm AddressBook databases SYNOPSIS
use Palm::Address; DESCRIPTION
The Address PDB handler is a helper class for the Palm::PDB package. It parses AddressBook databases. AppInfo block The AppInfo block begins with standard category support. See Palm::StdAppInfo for details. Other fields include: $pdb->{appinfo}{lastUniqueID} $pdb->{appinfo}{dirtyFields} I don't know what these are. $pdb->{appinfo}{fieldLabels}{name} $pdb->{appinfo}{fieldLabels}{firstName} $pdb->{appinfo}{fieldLabels}{company} $pdb->{appinfo}{fieldLabels}{phone1} $pdb->{appinfo}{fieldLabels}{phone2} $pdb->{appinfo}{fieldLabels}{phone3} $pdb->{appinfo}{fieldLabels}{phone4} $pdb->{appinfo}{fieldLabels}{phone5} $pdb->{appinfo}{fieldLabels}{phone6} $pdb->{appinfo}{fieldLabels}{phone7} $pdb->{appinfo}{fieldLabels}{phone8} $pdb->{appinfo}{fieldLabels}{address} $pdb->{appinfo}{fieldLabels}{city} $pdb->{appinfo}{fieldLabels}{state} $pdb->{appinfo}{fieldLabels}{zipCode} $pdb->{appinfo}{fieldLabels}{country} $pdb->{appinfo}{fieldLabels}{title} $pdb->{appinfo}{fieldLabels}{custom1} $pdb->{appinfo}{fieldLabels}{custom2} $pdb->{appinfo}{fieldLabels}{custom3} $pdb->{appinfo}{fieldLabels}{custom4} $pdb->{appinfo}{fieldLabels}{note} These are the names of the various fields in the address record. $pdb->{appinfo}{country} An integer: the code for the country for which these labels were designed. The country name is available as $Palm::Address::countries[$pdb->{appinfo}{country}]; $pdb->{appinfo}{misc} An integer. The least-significant bit is a flag that indicates whether the database should be sorted by company. The other bits are reserved. Sort block $pdb->{sort} This is a scalar, the raw data of the sort block. Records $record = $pdb->{records}[N]; $record->{fields}{name} $record->{fields}{firstName} $record->{fields}{company} $record->{fields}{phone1} $record->{fields}{phone2} $record->{fields}{phone3} $record->{fields}{phone4} $record->{fields}{phone5} $record->{fields}{address} $record->{fields}{city} $record->{fields}{state} $record->{fields}{zipCode} $record->{fields}{country} $record->{fields}{title} $record->{fields}{custom1} $record->{fields}{custom2} $record->{fields}{custom3} $record->{fields}{custom4} $record->{fields}{note} These are scalars, the values of the various address book fields. $record->{phoneLabel}{phone1} $record->{phoneLabel}{phone2} $record->{phoneLabel}{phone3} $record->{phoneLabel}{phone4} $record->{phoneLabel}{phone5} Most fields in an AddressBook record are straightforward: the "name" field always gives the person's last name. The "phoneN" fields, on the other hand, can mean different things in different records. There are five such fields in each record, each of which can take on one of eight different values: "Work", "Home", "Fax", "Other", "E-mail", "Main", "Pager" and "Mobile". The $record->{phoneLabel}{phone*} fields are integers. Each one is an index into @Palm::Address::phoneLabels, and indicates which particular type of phone number each of the $record->{phone*} fields represents. $record->{phoneLabel}{display} Like the phone* fields above, this is an index into @Palm::Address::phoneLabels. It indicates which of the phone* fields to display in the list view. $record->{phoneLabel}{reserved} I don't know what this is. METHODS
new $pdb = new Palm::Address; Create a new PDB, initialized with the various Palm::Address fields and an empty record list. Use this method if you're creating an Address PDB from scratch. new_Record $record = $pdb->new_Record; Creates a new Address record, with blank values for all of the fields. The AppInfo block will contain only an "Unfiled" category, with ID 0. "new_Record" does not add the new record to $pdb. For that, you want "$pdb->append_Record". SOURCE CONTROL
The source is in Github: http://github.com/briandfoy/p5-Palm/tree/master AUTHOR
Alessandro Zummo, "<a.zummo@towertech.it>" Currently maintained by brian d foy, "<bdfoy@cpan.org>" SEE ALSO
Palm::PDB(3) Palm::StdAppInfo(3) BUGS
The new() method initializes the AppInfo block with English labels and "United States" as the country. perl v5.10.1 2010-02-23 Address(3pm)
All times are GMT -4. The time now is 06:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy