Reformatting Data in AWK


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Reformatting Data in AWK
# 8  
Old 03-12-2009
Did you try with nawk or /usr/xpg4/bin/awk as I suggested?
# 9  
Old 03-12-2009
I tried both.

Here is the output at the prompt and the file as I modified it follows:

vis3: .../tmp/u31/TRANS % nawk -f arrange.awk
nawk: syntax error at source line 1
context is
awk >>> ' <<<
nawk: bailing out at source line 1
vis3: .../tmp/u31/TRANS % /usr/xpg4/bin/awk -f arrange.awk
/usr/xpg4/bin/awk: file "arrange.awk": line 1: syntax error Context is:
>>> awk ' <<<
vis3: .../tmp/u31/TRANS %


CODE LISTING

awk '{ print _ }
$1 ~ /^[0-9]*$/ && $NF ~ /^[0-9]*$/ {
print _; _ = ""
}
{ _ = _ ? _ FS $0 : $0 }' test.data
# 10  
Old 03-12-2009
If you want to put the code into a file/script, you should remove some parts.
arrange.awk should contain the following code:

Code:
END { print _ }
$1 ~ /^[0-9]*$/ && $NF ~ /^[0-9]*$/ {
  print _; _ = ""
  }
{ _ = _ ? _ FS $0 : $0 }

Or just run the code on the command line ...
# 11  
Old 03-12-2009
Radoulov,

Thanks for the tip. It works with NAWK but the result is not what I wanted.

I think we have a misunderstanding somewhere and it's probably from my end.

The data example that I have has a bunch of a parameters names listed in a sequential order. It then has the actual data that correspond to those parameters occuring in the same order for various tests that were carried out.

So the data reads sequentially as follow:

Parameter Names
Data Packet 1
Date Packet 2
Data Packet 3
.
.
.
Data Packet N

I need for the data to be out put in a tabular format with the parameter names at the top of each column.

If you have time, please see the earlier part of the thread.

Thanks for your help.

Regards,

sda_rr
# 12  
Old 03-12-2009
I think the code already produces the output that you describe, except for the field separator (a single space instead of a tab).
Anyway, try running the following code and then open the file with MS Excel or OO Calc (converting text to columns with tab separators).
The records are too long to be correctly displayed on a terminal.

Code:
nawk 'END { print _ }
$1 ~ /^[0-9]*$/ && $NF ~ /^[0-9]*$/ {
  print _; _ = ""
  }
{ $1 = $1; _ = _ ? _ OFS $0 : $0 }
' OFS='\t'  infile > outfile.csv

# 13  
Old 03-12-2009
Try this:

Code:
nawk '{printf "%s%s", NR%12==1?$0: $2, NR%12==0?"\n":" "}' file

This is the output I get:

Code:
$ cat file
SET LOCATION ANGLE1 ANGLE2 TYPE
D2 probe101 probe201 probe301 probe401
probe501 probe102 probe202 probe302 probe402
probe502 probe103 probe203 probe303 probe403
probe503 probe104 probe204 probe304 probe404
probe504 probe105 probe205 probe305 probe405
probe505 probe106 probe206 probe306 probe406
probe506 probe107 probe207 probe307 probe407
probe507 probe108 probe208 probe308 probe408
probe508 probe109 probe209 probe309 probe409
probe509 probe110 probe210 probe310 probe410
probe510 cprobe101 cprobe201 cprobe301 cprobe401
1 50 5 0 1
16 1.00E+05 1.50E+05 1.00E+05 1.50E+05
1.13E+05 1.13E+05 1.13E+05 7.50E+04 1.13E+05
8.44E+04 8.44E+04 8.44E+04 5.63E+04 8.44E+04
6.33E+04 6.33E+04 6.33E+04 4.22E+04 6.33E+04
4.75E+04 4.75E+04 4.75E+04 3.16E+04 4.75E+04
3.56E+04 3.56E+04 3.56E+04 2.37E+04 3.56E+04
2.67E+04 2.67E+04 2.67E+04 1.78E+04 2.67E+04
2.00E+04 2.00E+04 2.00E+04 1.33E+04 2.00E+04
1.50E+04 1.50E+04 1.50E+04 1.00E+04 1.50E+04
1.13E+04 1.13E+04 1.13E+04 7.51E+03 1.13E+04
8.45E+03 8.45E+03 8.45E+03 5.63E+03 8.45E+03
2 100 10 0 2
1.18E+05 1.18E+05 1.18E+05 7.88E+04 1.18E+05
8.86E+04 8.86E+04 8.86E+04 5.91E+04 8.86E+04
6.64E+04 6.64E+04 6.64E+04 4.43E+04 6.64E+04
4.98E+04 4.98E+04 4.98E+04 3.32E+04 4.98E+04
3.74E+04 3.74E+04 3.74E+04 2.49E+04 3.74E+04
2.80E+04 2.80E+04 2.80E+04 1.87E+04 2.80E+04
2.10E+04 2.10E+04 2.10E+04 1.40E+04 2.10E+04
1.58E+04 1.58E+04 1.58E+04 1.05E+04 1.58E+04
1.18E+04 1.18E+04 1.18E+04 7.88E+03 1.18E+04
8.87E+03 8.87E+03 8.87E+03 5.91E+03 8.87E+03
2.10E+00 1.05E+02 1.05E+01 0.00E+00 2.10E+00
6 300 30 0 6
1.20E+05 1.20E+05 1.20E+05 8.03E+04 1.20E+05
9.04E+04 9.04E+04 9.04E+04 6.02E+04 9.04E+04
6.78E+04 6.78E+04 6.78E+04 4.52E+04 6.78E+04
5.08E+04 5.08E+04 5.08E+04 3.39E+04 5.08E+04
3.81E+04 3.81E+04 3.81E+04 2.54E+04 3.81E+04
2.86E+04 2.86E+04 2.86E+04 1.91E+04 2.86E+04
2.14E+04 2.14E+04 2.14E+04 1.43E+04 2.14E+04
1.61E+04 1.61E+04 1.61E+04 1.07E+04 1.61E+04
1.21E+04 1.21E+04 1.21E+04 8.04E+03 1.21E+04
9.05E+03 9.05E+03 9.05E+03 6.03E+03 9.05E+03
2.14E+00 1.07E+02 1.07E+01 0.00E+00 2.14E+00
$
$
$ nawk '{printf "%s%s", NR%12==1?$0: $2, NR%12==0?"\n":" "}' file
SET LOCATION ANGLE1 ANGLE2 TYPE probe101 probe102 probe103 probe104 probe105 probe106 probe107 probe108 probe109 probe110 cprobe101
1 50 5 0 1 1.00E+05 1.13E+05 8.44E+04 6.33E+04 4.75E+04 3.56E+04 2.67E+04 2.00E+04 1.50E+04 1.13E+04 8.45E+03
2 100 10 0 2 1.18E+05 8.86E+04 6.64E+04 4.98E+04 3.74E+04 2.80E+04 2.10E+04 1.58E+04 1.18E+04 8.87E+03 1.05E+02
6 300 30 0 6 1.20E+05 9.04E+04 6.78E+04 5.08E+04 3.81E+04 2.86E+04 2.14E+04 1.61E+04 1.21E+04 9.05E+03 1.07E+02
$
$

Regards
# 14  
Old 03-12-2009
For fixed number of fields (60 in your example), you can use something like this:

Code:
nawk > output.csv '{ 
  $1 = $1
  nf += NF
  _ = _ ? _ OFS $0 : $0 
  }
nf == 60 { 
  print _ RS 
  _ = nf = 0 
  }' ORS= OFS='\t' infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help reformatting column

Hello UNIX experts, I'm stumped finding a method to reformat a column. Input file is a two column tab-delimited file. Essentially, for every term that appears in column 2, I would like to summarize whether that term appears for every entry in column 1. In other words, make a header for each term... (2 Replies)
Discussion started by: torchij
2 Replies

2. Shell Programming and Scripting

awk --> math-operation in data-record and joining with second file data

Hi! I have a pretty complex job - at least for me! i have two csv-files with meassurement-data: fileA ...... (2 Replies)
Discussion started by: IMPe
2 Replies

3. Shell Programming and Scripting

awk help reformatting badly formatted time field

I need help reformatting an input file with spaces in the time field (4th field). I want the field to look like “hh:mm” with appropriate embedded zeros, but instead it has “h :m “ if the hour and/or minute are single character. I'm pretty new to scripting and this is beyond me. Any help would... (4 Replies)
Discussion started by: lisep
4 Replies

4. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

5. Shell Programming and Scripting

Reformatting data in matrix form

Hi, Some assistance with respect to the following problem will be very helpful. I want to reformat my dataset in the following manner for subsequent analysis. I have first column values (which repeat for each value of 2nd column) which are names, the second column specifies position ad the... (1 Reply)
Discussion started by: newbie83
1 Replies

6. UNIX for Dummies Questions & Answers

Date reformatting

I have been reformatting dates from a data file to make them mysql compliant. 31-10-2011 Loc1 1-11-2011 Loc2 The first can be captured by this: sed -i '' -e "s#\(..\)-\(..\)-\(....\)#\3-\2-\1#" data.txt and leads to: 2011-10-31 Loc1 The second line is captured as follows: sed -i... (2 Replies)
Discussion started by: figaro
2 Replies

7. UNIX for Dummies Questions & Answers

Date reformatting

I have a file with temperature measurements: Loc1,20090102,71.55 Loc1,20090103,71.65 Loc1,20090104,71.55 Loc1,20090105,71.54 Loc1,20090106,71.54 However, to load this into a database I would like to reformat the dates (column 2) from the yyyymmdd format to the yyyy-mm-dd format. I have... (2 Replies)
Discussion started by: figaro
2 Replies

8. Shell Programming and Scripting

reformatting xml file, sed or awk I think (possibly perl)

I have some xml files that cannot be read using a standard parser, or I am using the wrong parser. The issues seems to be spaces in some of the tags. Here is a sample,<UgUn 2 > <Un> -0.426753 </Un> </UgUn>The parser isn't able to find the number 2, so that information is lost, etc. It seems... (16 Replies)
Discussion started by: LMHmedchem
16 Replies

9. Shell Programming and Scripting

awk multiple file reformatting

I hopefully have a simple request - I need to process multiple files reformatting the output based on tags at the beginning of each line. So the data for the new 3 lines of the output file are in the HDR line and then the details are in the DTL tagged lines. for ifile in $indir do echo... (1 Reply)
Discussion started by: jason_v_brown
1 Replies

10. Filesystems, Disks and Memory

reformatting a floppy!

i am trying to reformat a floppy i am using solaris 9 when i run this: rmformat -F quick /vol/dev/aliases/floppy0 it tells me that it cannot perform the operation on a mounted device. how do i unmount the device and format the floppy? (1 Reply)
Discussion started by: rmuhammad
1 Replies
Login or Register to Ask a Question