awk - rearange data


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk - rearange data
# 1  
Old 02-17-2010
awk - rearange data

Dear All,

once again I am encountering a problem with awk.

The file looks like this:

Code:
BroadLeaves 43.6 clc2006 37.6 Conifers 8.3 edge100 5.1 dem30sec 3.9 aspect 1.5 slope 0
dem30sec 58.3 Conifers 28.5 clc2006 7.3 edge100 3 slope 2.4 BroadLeaves 0.4 aspect 0.1
....
...
..

My desired output would be a file with 7 columns for each category one with the corresponding value

Code:
aspect BroadLeaves clc2006 Conifers dem30sec edge100 slope
1.5 43.6 37.6 8.3 5.1 3.9 5.1 0 
0.1 0.4 28.5 28.5 58.3 3 2.4
...
..
..

Is there anyway to do this efficiently? The only way I can think of at the moment is a series of if statements.

Any guidance is greatly appreciated, thanks in advance!
# 2  
Old 02-17-2010
Cuold you please provide more example input data (a bigger sample)?
# 3  
Old 02-17-2010
Hi,

thanks for your reply, the sample stays more or less the same, only the varaible change in their position.

Code:
BroadLeaves 43.6 clc2006 37.6 Conifers 8.3 edge100 5.1 dem30sec 3.9 aspect 1.5 slope 0
dem30sec 58.3 Conifers 28.5 clc2006 7.3 edge100 3 slope 2.4 BroadLeaves 0.4 aspect 0.1
dem30sec 50.2 Conifers 43.2 clc2006 3 edge100 2.1 slope 0.8 aspect 0.5 BroadLeaves 0.3
dem30sec 47.4 Conifers 42 clc2006 5.2 edge100 2.8 slope 1.3 BroadLeaves 0.7 aspect 0.5
Conifers 44.8 dem30sec 36.8 clc2006 8.6 edge100 7.2 BroadLeaves 1.2 slope 0.9 aspect 0.4
dem30sec 43.5 Conifers 37.8 edge100 12.1 clc2006 2 BroadLeaves 2 aspect 1.6 slope 1.1
Conifers 43.2 dem30sec 40.6 BroadLeaves 7.1 edge100 3.5 clc2006 3 slope 2.1 aspect 0.5
dem30sec 46.6 Conifers 40.8 edge100 5.2 aspect 3.2 clc2006 2.9 BroadLeaves 1 slope 0.3
dem30sec 52.5 Conifers 36.7 edge100 5.5 clc2006 2.2 aspect 1.5 BroadLeaves 1 slope 0.6
dem30sec 49 Conifers 33.8 edge100 5.8 slope 4.5 clc2006 3 BroadLeaves 2.5 aspect 1.3
dem30sec 47.1 Conifers 40.9 clc2006 4.8 slope 4.1 edge100 1.4 BroadLeaves 1.2 aspect 0.5
dem30sec 44.7 Conifers 38.6 clc2006 7 edge100 6.4 slope 1.6 BroadLeaves 1.4 aspect 0.2
Conifers 28.2 dem30sec 16.7 edge100 15.9 clc2006 14.2 BroadLeaves 12.6 aspect 6.7 slope 5.8
dem30sec 46.5 Conifers 42.6 clc2006 4.8 edge100 4.4 BroadLeaves 0.8 aspect 0.5 slope 0.4
dem30sec 53.8 Conifers 35 edge100 4.1 clc2006 4 aspect 1.9 slope 0.8 BroadLeaves 0.4
Conifers 40.7 dem30sec 39.6 edge100 7.6 clc2006 6.4 BroadLeaves 2.5 aspect 2 slope 1.3
dem30sec 44.7 Conifers 43 clc2006 5.2 edge100 3.4 slope 1.6 aspect 1.3 BroadLeaves 0.8
Conifers 51 dem30sec 36.8 edge100 5.1 clc2006 3.5 BroadLeaves 1.4 aspect 1.1 slope 0.9
dem30sec 47.3 Conifers 42.8 edge100 5.3 clc2006 2 slope 1.8 BroadLeaves 0.4 aspect 0.4
dem30sec 45.8 Conifers 37.3 edge100 7.2 clc2006 6.1 aspect 1.6 slope 1.3 BroadLeaves 0.6
dem30sec 52.4 Conifers 37.7 clc2006 4.5 edge100 3.1 BroadLeaves 1.1 slope 0.8 aspect 0.5
dem30sec 43.8 Conifers 42.5 edge100 7.6 clc2006 3.6 slope 1.7 BroadLeaves 0.5 aspect 0.4
dem30sec 43.8 Conifers 43.7 clc2006 5.9 edge100 4.3 aspect 1.2 BroadLeaves 0.6 slope 0.6
Conifers 49.2 dem30sec 27.6 edge100 6.9 aspect 6.9 BroadLeaves 4 clc2006 3.5 slope 1.9
dem30sec 56.5 Conifers 35.6 edge100 3.9 clc2006 2.3 BroadLeaves 0.9 slope 0.5 aspect 0.2
dem30sec 51.4 Conifers 37.9 edge100 5.1 clc2006 3.6 BroadLeaves 1.1 slope 0.8 aspect 0.1

thanks
# 4  
Old 02-17-2010
well not sure about awk, but it would be very easy with perl.
are you interested in knowing the soln?
# 5  
Old 02-17-2010
That would be great. I havent worked with Perl so fare but I am familiar with other programming languages.

Thanks J
# 6  
Old 02-17-2010
Create a file with some name, say converter.pl with content like,

Code:
#!/usr/bin/perl -w
use strict;

my @keyList = qw (edge100 BroadLeaves Conifers clc2006 slope aspect dem30sec);

print join (",", @keyList) . "\n";
while (my $line = <>) {
        my %arr = split (/\s/, $line);
        my @vals;
        foreach my $key (@keyList) {
                push (@vals, $arr{$key});
        }
        print join (",", @vals) . "\n";
}

run it like
Code:
cat  data| ./converter.pl

# 7  
Old 02-17-2010
Great, thanks a lot it works perfectly and does exactly what I was looking for!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk --> math-operation in data-record and joining with second file data

Hi! I have a pretty complex job - at least for me! i have two csv-files with meassurement-data: fileA ...... (2 Replies)
Discussion started by: IMPe
2 Replies

2. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

3. Shell Programming and Scripting

Help to get correct data using awk

I have this input.|user1 |10.10.10.10 |23|046|1726 (212) |0 |user2 |10.10.10.11 |23|046|43 (17) |0 |test |10.10.10.12 |23|046|45 (10) |0 |test1 |10.10.10.13 |23|046|89 (32) |0 I need to get the data for a user like thisuser1 1726 user2 43 test 45 test1 89... (11 Replies)
Discussion started by: Jotne
11 Replies

4. Shell Programming and Scripting

extract data with awk

i have a following output file PF Release 2.4 on SERVICE at Mon Feb 6 18:41:02 2012 ---------------------------------------- ---------------- |pPF |SEP |CAPS |CALLS |OPEN | |-------------------------------------------------------------| | 0 ---... (1 Reply)
Discussion started by: gauravah
1 Replies

5. Shell Programming and Scripting

Rearange fields within a csv

Hello with sed and tr I have converted my "quite big" cronlist into the below csv file Here is a sample: 05,15,15,25,35,45,55;*;*;*;*;/SCRIPT1;/SCRIPT1.log 10;6,16;*;*;*;/SCRIPT2;/SCRIPT2.log 30;*;*;*;*;/SCRIPT3;/SCRIPT3.cfg;/SCRIPT3.log 50;8;*;*;*;/SCRIPT4;-1;/SCRIPT4.log ... (6 Replies)
Discussion started by: drbiloukos
6 Replies

6. Shell Programming and Scripting

AWK help. how to compare a variable with a data array in AWK?

Hi all, i have a data array as follows. array=ertfgj2345 array=456ttygkd . . . array=errdjt3235 so number or elements in the array can varies depending on how big the data input is. now i have a variable, and it is $1 (there are $2, $3 and so on, i am only interested in $1). ... (9 Replies)
Discussion started by: usustarr
9 Replies

7. Shell Programming and Scripting

Awk: Data Insert

Hi guys, I'm having some difficulties in insert some details to the following contents. I need to insert "TEST" under MIR & a value "25" to the next line. So far, I am able to insert "TEST" by using awk to capture MIR as the identifier. However, I am having some difficulties in inserting "25"... (3 Replies)
Discussion started by: nantheless
3 Replies

8. Shell Programming and Scripting

Reformatting Data in AWK

Dear AWK Users, I have a data set that is so large (Gigabytes) that it cannot be opened in the vi editor in its entirety. But I can manipulate the entire thing in AWK. It is formatted in a regular manner such that it has the variable descriptions or listings preceeding the variables. The latter... (13 Replies)
Discussion started by: sda_rr
13 Replies

9. Shell Programming and Scripting

help reformat data with awk

I am trying to write an awk program to reformat a data table and convert the date to julian time. I have all the individual steps working, but I am having some issues joing them into one program. Can anyone help me out? Here is my code so far: # This is an awk program to convert the dates from... (4 Replies)
Discussion started by: climbak
4 Replies

10. UNIX for Dummies Questions & Answers

Extrat data using AWK

How can I extract data using AWK I have data as follow 01--------- 02----- 03----- 03--- I want to extract data to the file 01.dat,02.dat and 03.dat the field value should be the part of file name OutFile="xx" awk -F"|" '{OutFile = $1; print OutFile}' x1 > $OutFile echo $OutFile this... (1 Reply)
Discussion started by: kadamr
1 Replies
Login or Register to Ask a Question