Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Rearranging into new columns (awk?)

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Rearranging into new columns (awk?)
# 1  
Old 07-26-2012
Rearranging into new columns (awk?)
Hi experts,

I've used several solutions from this forum to delete nonsense and rearrange data in the project file I'm working on. I'm hoping you guys can give me some tips on further rearranging the data (I've seen a few solutions by searching, but one specific item has me stumped, which is only using the date-time fields once).

I have a text file that looks like this:

Code:
2MTEMP
2012071512
73.28
2MTEMP
2012071513
75.65
MSLP
2012071512
1014.60
MSLP
2012071513
1015.11
2MDEW
2012071512
65.52
2MDEW
2012071513
67.33

I'd like to rearrange it into columns to look like:
Code:
TIME           2MTEMP      MSLP     2MDEW  
2012071512     73.28      1014.60   65.52
2012071513     75.65      1015.11   67.33

Each field contains its own separate date/time field (eg 2012071512) but I only need it to strip out the valid times once and then use only the field name (2MTEMP,MSLP,etc) and the value following to line up as given above.

I'd really appreciate your help!
Last edited by coryvp; 07-26-2012 at 01:45 PM..
# 2  
Old 07-26-2012
Hi, try:

Code:
awk '
  {
    h=$1
    getline d
    getline v
    Date[d]
    Header[h]
    Value[h,d] = v
  }

  END {
    s="TIME\t"
    for (h in Header) {
      s = s "\t" h
    }
    print s
    for (d in Date) {
      s = d
      for (h in Header) {
        s = s "\t" Value[h,d]
      }
      print s
    }
  }
' infile


Code:
TIME		MSLP	2MDEW	2MTEMP
2012071512	1014.60	65.52	73.28
2012071513	1015.11	67.33	75.65

This User Gave Thanks to Scrutinizer For This Post:
coryvp
# 3  
Old 07-28-2012
Thanks for your help!! I have one question; I noticed when this operated on a longer dataset, it confused some of the date-time groups (for instance, it inserted 2012071618 after 2012071612, while skipping those in between--the last two digits of each group are the UTC hour, by the way) rather than going in chronological order. Is there something I could add to tell it the specific date-time format so it will keep them in order? Thanks again!
# 4  
Old 07-28-2012
To preserve the correct order:

If you are using gawk:
Code:
WHINY_USERS=1 gawk ' ....

Or you could pipe everything through sort -n and add the header later.
Otherwise a change would be required in the script...
# 5  
Old 07-28-2012
Quote:
WHINY_USERS=1
--- lovely!
# 6  
Old 07-29-2012
Quote:
Originally Posted by coryvp
Thanks for your help!! I have one question; I noticed when this operated on a longer dataset, it confused some of the date-time groups (for instance, it inserted 2012071618 after 2012071612, while skipping those in between--the last two digits of each group are the UTC hour, by the way) rather than going in chronological order. Is there something I could add to tell it the specific date-time format so it will keep them in order? Thanks again!
if there are only these 3 columns of values: 2MTEMP, MSLP, 2MDEM, try these:
Code:
list="2MTEMP
2012071512
73.28
2MTEMP
2012071513
75.65
MSLP
2012071512
1014.60
MSLP
2012071513
1015.11
2MDEW
2012071512
65.52
2MDEW
2012071513
67.33"
echo "${list}"|awk '{
  header=$1;
  getline datetime
  getline value
  date_list[datetime]
  value_list[datetime,header]=value
}
END{
    for (t in date_list)
        print t"\t"value_list[t,"MSLP"]"\t"value_list[t,"2MDEW"]"\t"value_list[t,"2MTEMP"]
}'|sort

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Rearranging the column

I have a text file like this, I would like to rearrange the first column (Name) according to the third column(percentage)in descending order. I mean methionine with the highest percentage should be the first one to appear under the name column. But I also want to exclude the headers from this... (2 Replies)
Discussion started by: cathum
2 Replies

2. Shell Programming and Scripting

Rearranging Data Set

Hello everybody, I've got the following problem: The data set I have is an ASCII file containing a header over 4 lines and the actual data comprised of dezimal numbers in a 1000x1000 grid (1000 lines and 1000 columns). Since I want to plot the data in GMT I need to convert it into the... (3 Replies)
Discussion started by: Evilknievel
3 Replies

3. Shell Programming and Scripting

Help with rearranging file with script

Hi Guys I normally do thins with a Windows program but I am trying to rearrange a filename based on delimiters in Ubuntu. Example v017 __ Detective Academy Q #133 Murder in the Village Of Suspension Bridges &&& Part 9.cbz = Detective Academy Q v017 #133 Murder in the Village Of Suspension... (9 Replies)
Discussion started by: itschrisonline
9 Replies

4. Shell Programming and Scripting

Help with splitting and rearranging a field in awk

Please help! I figured I would take this one to the experts. I'm working with a field that contains contents such as: LastName FirstName For example: Smith John I'm trying to take this field and split it so that it is two separate fields (first name and last name). I then need to print... (5 Replies)
Discussion started by: IX3R0XI
5 Replies

5. Shell Programming and Scripting

Counting and rearranging the rows

Hi, I have a file that I re-arranged using awk and unix commands to produce a file that looks like this: JOE JOE JOE JOE JOE BOB BOB HI HI HI I want to count how many of the same rows there are and print it on the second column while only maintaining the original name once. The... (5 Replies)
Discussion started by: phil_heath
5 Replies

6. Shell Programming and Scripting

Doubt with rearranging file through awk

Filename1.xml NO 2012-11-16 02:00:27 20121115/pathname/ asia Filename1.rec YES 2012-11-16 01:20:24 20121115/pathname asia FIleName2.xml YES 2012-11-16 01:20:25 20121115/pathaname asia if the file content are... (6 Replies)
Discussion started by: manas_ranjan
6 Replies

7. Shell Programming and Scripting

Rearranging

Hello, I spent all day trying to write a script and cannot find the solution :( I have plenty files looking like this: several hundred lines precede the following interesting Bla xxx: Blub = -7537.37687 Blub = -100.644746 Blub = -3247.61954 . . . Blub = 1324.82567 Blub =... (2 Replies)
Discussion started by: tempestas
2 Replies

8. UNIX for Dummies Questions & Answers

Rearranging whole columns

Hello all, I have a text file that is arranged: name 3 7 2 9 5 jim a d e g k max d g u x g rob f w v k o This is just an example as my real file has >1000 individuals and >64,000 columns. I need to rearrange the file so that the columns appear in numerical order so that name... (3 Replies)
Discussion started by: doobedoo
3 Replies

9. Shell Programming and Scripting

Rearranging columns

Hi, I have an input file as follows : input.txt abcdTXXqwe axdfSYYrew dasgTXXqwt gtfsTYYwer gadfSXXerw gwerSYYTXX Now I have to get four output files. output1.txt should have the first four cloumns, Where the rows containing 5th column as T and 6th-7th columns as XX output2.txt... (5 Replies)
Discussion started by: sudhamacs
5 Replies

10. Shell Programming and Scripting

Rearranging fields from a pipe

I have a large file that I am pulling only certain fields out of but my output I would like to rearrange the field order via a pipe. I have been looking through the site and man pages and have come to a loss. I am running on HP cut -c33-38,44-46,62-65,91-98 <file> | grep -e <value> >... (4 Replies)
Discussion started by: bthomas
4 Replies
Login or Register to Ask a Question