03-25-2009
Question about Awk for Data Parsing
Hi,
I am doing some data parsing for some economics research. I was recently exposed to shell script and am brand new to awk. I have a large csv file (like 10G) and I would like to make it a lot smaller with awk, but it is a bit tricky for me and I haven't been able to get it yet. I would like to keep the first four columns, in then in the newly created csv file have the fifth column contain whatever "cell" (separated by commas) in the original file contains the string "}".
Can anyone provide some help?
Thank you very much.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I am trying to use a line of output in an XML file as input in another new XML file for processing purposes via a shell script. Since I am a newbie though, I'm not sure how to do this since the data is different everytime. I am using this technique with static data right now:
echo -n "Running... (5 Replies)
Discussion started by: corwin43
5 Replies
2. Shell Programming and Scripting
Hi I need to parse the following data using shell script
Table
-----
stage4n_abc 48
stage4o_abcd 4
adashpg_abc_HeartBeat 1
stage4l_asc 168
Can anyone gimme the solution.
I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies
3. Shell Programming and Scripting
Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies
4. Shell Programming and Scripting
Hi friends,
I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function.
INPUT(single parameter as complete string)
A;BCF;DFG;FD
... (3 Replies)
Discussion started by: sumesh.1988
3 Replies
5. Shell Programming and Scripting
I have below string
Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies
6. Shell Programming and Scripting
Hi,
I do have a data file which is divided into compartments by ---------. I would like to extract (parse) some of the data and numbers either using awk or sed
The file has the format:
CATGC
Best GO enrichment:
Genes/ORF that have the motifs (genes are sorted by max(pa+pd+po)):
... (6 Replies)
Discussion started by: Lucky Ali
6 Replies
7. Shell Programming and Scripting
Experts ,
Below is the data:
--- Physical volumes ---
PV Name /dev/dsk/c1t2d0
VG Name /dev/vg00
PV Status available
Allocatable yes
VGDA 2
Cur LV 8
PE Size (Mbytes) 8
Total PE 4350
Free PE 2036
Allocated PE 2314
Stale PE 0
IO Timeout (Seconds) default
--- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies
8. Shell Programming and Scripting
Heya,
I'm currently working on a script so I can see which cron jobs, if any, on a system are executing less frequently than 15 minutes (1 - 14 minutes). This is the only data I'm interested in. So far I have the following:
#!/bin/bash
IFS=$'\n';for line in `ls -f /var/spool/cron/*`;
do... (5 Replies)
Discussion started by: Elvirnith
5 Replies
9. Shell Programming and Scripting
Hi folks
I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file.
It all works - apart from the xml and config file format in the new file
with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies
10. Shell Programming and Scripting
Input File
Name of the session: filesrv_quo
snap Logical Units UID: 60:06:01:60:01:7B:25:00:C8:86:B0:CA:5B:A2:E0:11
Name of the session: verspn2_at_176_0218
snap Logical Units UID:
Name of the session: DRT-ny-iadsql1-c_ny-iadsql2-c
snap Logical Units UID: ... (4 Replies)
Discussion started by: greycells
4 Replies
RBASH(1) General Commands Manual RBASH(1)
NAME
rbash - restricted bash, see bash(1)
RESTRICTED SHELL
If bash is started with the name rbash, or the -r option is supplied at invocation, the shell becomes restricted. A restricted shell is
used to set up an environment more controlled than the standard shell. It behaves identically to bash with the exception that the follow-
ing are disallowed or not performed:
o changing directories with cd
o setting or unsetting the values of SHELL, PATH, ENV, or BASH_ENV
o specifying command names containing /
o specifying a file name containing a / as an argument to the . builtin command
o specifying a filename containing a slash as an argument to the -p option to the hash builtin command
o importing function definitions from the shell environment at startup
o parsing the value of SHELLOPTS from the shell environment at startup
o redirecting output using the >, >|, <>, >&, &>, and >> redirection operators
o using the exec builtin command to replace the shell with another command
o adding or deleting builtin commands with the -f and -d options to the enable builtin command
o using the enable builtin command to enable disabled shell builtins
o specifying the -p option to the command builtin command
o turning off restricted mode with set +r or set +o restricted.
These restrictions are enforced after any startup files are read.
When a command that is found to be a shell script is executed, rbash turns off any restrictions in the shell spawned to execute the script.
SEE ALSO
bash(1)
GNU Bash-4.0 2004 Apr 20 RBASH(1)