03-25-2009
Question about Awk for Data Parsing
Hi,
I am doing some data parsing for some economics research. I was recently exposed to shell script and am brand new to awk. I have a large csv file (like 10G) and I would like to make it a lot smaller with awk, but it is a bit tricky for me and I haven't been able to get it yet. I would like to keep the first four columns, in then in the newly created csv file have the fifth column contain whatever "cell" (separated by commas) in the original file contains the string "}".
Can anyone provide some help?
Thank you very much.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I am trying to use a line of output in an XML file as input in another new XML file for processing purposes via a shell script. Since I am a newbie though, I'm not sure how to do this since the data is different everytime. I am using this technique with static data right now:
echo -n "Running... (5 Replies)
Discussion started by: corwin43
5 Replies
2. Shell Programming and Scripting
Hi I need to parse the following data using shell script
Table
-----
stage4n_abc 48
stage4o_abcd 4
adashpg_abc_HeartBeat 1
stage4l_asc 168
Can anyone gimme the solution.
I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies
3. Shell Programming and Scripting
Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies
4. Shell Programming and Scripting
Hi friends,
I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function.
INPUT(single parameter as complete string)
A;BCF;DFG;FD
... (3 Replies)
Discussion started by: sumesh.1988
3 Replies
5. Shell Programming and Scripting
I have below string
Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies
6. Shell Programming and Scripting
Hi,
I do have a data file which is divided into compartments by ---------. I would like to extract (parse) some of the data and numbers either using awk or sed
The file has the format:
CATGC
Best GO enrichment:
Genes/ORF that have the motifs (genes are sorted by max(pa+pd+po)):
... (6 Replies)
Discussion started by: Lucky Ali
6 Replies
7. Shell Programming and Scripting
Experts ,
Below is the data:
--- Physical volumes ---
PV Name /dev/dsk/c1t2d0
VG Name /dev/vg00
PV Status available
Allocatable yes
VGDA 2
Cur LV 8
PE Size (Mbytes) 8
Total PE 4350
Free PE 2036
Allocated PE 2314
Stale PE 0
IO Timeout (Seconds) default
--- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies
8. Shell Programming and Scripting
Heya,
I'm currently working on a script so I can see which cron jobs, if any, on a system are executing less frequently than 15 minutes (1 - 14 minutes). This is the only data I'm interested in. So far I have the following:
#!/bin/bash
IFS=$'\n';for line in `ls -f /var/spool/cron/*`;
do... (5 Replies)
Discussion started by: Elvirnith
5 Replies
9. Shell Programming and Scripting
Hi folks
I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file.
It all works - apart from the xml and config file format in the new file
with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies
10. Shell Programming and Scripting
Input File
Name of the session: filesrv_quo
snap Logical Units UID: 60:06:01:60:01:7B:25:00:C8:86:B0:CA:5B:A2:E0:11
Name of the session: verspn2_at_176_0218
snap Logical Units UID:
Name of the session: DRT-ny-iadsql1-c_ny-iadsql2-c
snap Logical Units UID: ... (4 Replies)
Discussion started by: greycells
4 Replies
IGAWK(1) Utility Commands IGAWK(1)
NAME
igawk - gawk with include files
SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ...
igawk [ all gawk options ] [ -- ] program-text file ...
DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).
AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like
@include getopt.awk
in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.
OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports.
EXAMPLES
cat << EOF > test.awk
@include getopt.awk
BEGIN {
while (getopt(ARGC, ARGV, "am:q") != -1)
...
}
EOF
igawk -f test.awk
SEE ALSO
gawk(1)
Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.
AUTHOR
Arnold Robbins (arnold@skeeve.com).
Free Software Foundation Nov 3 1999 IGAWK(1)