03-12-2009
Reformatting Data in AWK
Dear AWK Users,
I have a data set that is so large (Gigabytes) that it cannot be opened in the vi editor in its entirety. But I can manipulate the entire thing in AWK. It is formatted in a regular manner such that it has the variable descriptions or listings preceeding the variables. The latter then follow in discrete batches.
datatype 1 datatype 2 datatype 3 datatype 4 datatype 5
...
...
datatype N
x1_1 x2_1 x3_1 x4_1 x5_1
...
...
xN_1
x1_2 x2_2 x3_2 x4_2 x5_2
...
...
xN_2
x1_N x2_N x3_N x4_N x5_N
...
...
xN_N
I don't need every variable so I would like to extract specific variables depending on need from the dataset and print them out it the following format.
datatype 1 datatype 2 dataype 4
x1_1 x2_1 x4_1
x1_2 x2_2 x4_2
...
...
x1_N x2_N x4_N
Any help would be appreciated.
Regards,
sda_rr
10 More Discussions You Might Find Interesting
1. Filesystems, Disks and Memory
i am trying to reformat a floppy i am using solaris 9 when i run this:
rmformat -F quick /vol/dev/aliases/floppy0
it tells me that it cannot perform the operation on a mounted device.
how do i unmount the device and format the floppy? (1 Reply)
Discussion started by: rmuhammad
1 Replies
2. Shell Programming and Scripting
I hopefully have a simple request - I need to process multiple files reformatting the output based on tags at the beginning of each line. So the data for the new 3 lines of the output file are in the HDR line and then the details are in the DTL tagged lines.
for ifile in $indir
do
echo... (1 Reply)
Discussion started by: jason_v_brown
1 Replies
3. Shell Programming and Scripting
I have some xml files that cannot be read using a standard parser, or I am using the wrong parser. The issues seems to be spaces in some of the tags.
Here is a sample,<UgUn 2 >
<Un>
-0.426753
</Un>
</UgUn>The parser isn't able to find the number 2, so that information is lost, etc. It seems... (16 Replies)
Discussion started by: LMHmedchem
16 Replies
4. UNIX for Dummies Questions & Answers
I have a file with temperature measurements:
Loc1,20090102,71.55
Loc1,20090103,71.65
Loc1,20090104,71.55
Loc1,20090105,71.54
Loc1,20090106,71.54
However, to load this into a database I would like to reformat the dates (column 2) from the yyyymmdd format to the yyyy-mm-dd format. I have... (2 Replies)
Discussion started by: figaro
2 Replies
5. UNIX for Dummies Questions & Answers
I have been reformatting dates from a data file to make them mysql compliant.
31-10-2011 Loc1
1-11-2011 Loc2
The first can be captured by this:
sed -i '' -e "s#\(..\)-\(..\)-\(....\)#\3-\2-\1#" data.txt
and leads to:
2011-10-31 Loc1
The second line is captured as follows:
sed -i... (2 Replies)
Discussion started by: figaro
2 Replies
6. Shell Programming and Scripting
Hi,
Some assistance with respect to the following problem will be very helpful.
I want to reformat my dataset in the following manner for subsequent analysis.
I have first column values (which repeat for each value of 2nd column) which are names, the second column specifies position ad the... (1 Reply)
Discussion started by: newbie83
1 Replies
7. Shell Programming and Scripting
Experts ,
Below is the data:
--- Physical volumes ---
PV Name /dev/dsk/c1t2d0
VG Name /dev/vg00
PV Status available
Allocatable yes
VGDA 2
Cur LV 8
PE Size (Mbytes) 8
Total PE 4350
Free PE 2036
Allocated PE 2314
Stale PE 0
IO Timeout (Seconds) default
--- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies
8. Shell Programming and Scripting
I need help reformatting an input file with spaces in the time field (4th field). I want the field to look like “hh:mm” with appropriate embedded zeros, but instead it has “h :m “ if the hour and/or minute are single character.
I'm pretty new to scripting and this is beyond me. Any help would... (4 Replies)
Discussion started by: lisep
4 Replies
9. Shell Programming and Scripting
Hi!
I have a pretty complex job - at least for me!
i have two csv-files with meassurement-data:
fileA
...... (2 Replies)
Discussion started by: IMPe
2 Replies
10. Shell Programming and Scripting
Hello UNIX experts,
I'm stumped finding a method to reformat a column. Input file is a two column tab-delimited file. Essentially, for every term that appears in column 2, I would like to summarize whether that term appears for every entry in column 1. In other words, make a header for each term... (2 Replies)
Discussion started by: torchij
2 Replies
LEARN ABOUT ULTRIX
ucblinks
ucblinks(1B) SunOS/BSD Compatibility Package Commands ucblinks(1B)
NAME
ucblinks - adds /dev entries to give SunOS 4.x compatible names to SunOS 5.x devices
SYNOPSIS
/usr/ucb/ucblinks [-e rulebase] [-r rootdir]
DESCRIPTION
ucblinks creates symbolic links under the /dev directory for devices whose SunOS 5.x names differ from their SunOS 4.x names. Where possi-
ble, these symbolic links point to the device's SunOS 5.x name rather than to the actual /devices entry.
ucblinks does not remove unneeded compatibility links; these must be removed by hand.
ucblinks should be called each time the system is reconfiguration-booted, after any new SunOS 5.x links that are needed have been created,
since the reconfiguration may have resulted in more compatibility names being needed.
In releases prior to SunOS 5.4, ucblinks used a nawk rule-base to construct the SunOS 4.x compatible names. ucblinks no longer uses nawk
for the default operation, although nawk rule-bases can still be specifed with the -e option. The nawk rule-base equivalent to the SunOS
5.4 default operation can be found in /usr/ucblib/ucblinks.awk.
OPTIONS
-e rulebase Specify rulebase as the file containing nawk(1) pattern-action statements.
-r rootdir Specify rootdir as the directory under which dev and devices will be found, rather than the standard root directory /.
FILES
/usr/ucblib/ucblinks.awk sample rule-base for compatibility links
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWscpu |
+-----------------------------+-----------------------------+
SEE ALSO
devlinks(1M), disks(1M), ports(1M), tapes(1M), attributes(5)
SunOS 5.10 13 Apr 1994 ucblinks(1B)