Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Using awk to rearrange data according to date

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using awk to rearrange data according to date
# 1  
Old 08-11-2011
Using awk to rearrange data according to date
Hi,

I have a large data frame as shown below, where data is separated into years.

Code:
10 May 2011 Created: 10 May 11 15:05 GMT Scale: SIO-2005 and others
GC-MD, Cape Grim, Tasmania, Lat.: 40.68S, Lon.: 144.69E, Alt: 94m above sea level
You can use the following format in Fortran to read data in different columns, (F10.5, 2I4,I6, 2I4,I6,1X,10(F10.3,a1))
    scale:  --  --    --  --  --    --     SIO-05     SIO-05     SIO-05     SIO-05     SIO-98     SIO-05 Tohoku-Uni     SIO-98    CSIRO94    CSIRO94
     unit:  --  --    --  --  --    --        ppt        ppt        ppt        ppt        ppb        ppt        ppb        ppt        ppb        ppb
      time  DD  MM  YYYY  hh min ABSDA     CFC-11     CFC-12    CH3CCl3       CCl4        N2O    CFC-113        CH4      CHCl3         CO         H2
2010.00004   1   1  2010   0  23 11628    239.925    533.582      8.181     86.203    322.195     75.690   1736.198      4.644     35.280    529.335 
2010.00012   1   1  2010   1   3 11628    239.436    533.013      8.224     86.178    321.900     75.496   1735.882      4.892     34.441    527.677 
2010.00020   1   1  2010   1  43 11628    239.884    533.140      7.948     85.851    322.215     75.451   1736.369      4.557     34.916    531.719 
2010.00027   1   1  2010   2  23 11628    239.977    533.657      8.383     86.410    322.339     75.445   1736.773      4.598     34.633    527.731 
2010.00035   1   1  2010   3   3 11628    239.687    533.042      8.032     86.418    322.198     75.405   1735.862      4.534     32.125    527.888 
2010.00042   1   1  2010   3  43 11628    239.733    533.058      7.837     86.113    322.212     75.436   1736.022      4.831     33.056    527.984

This data is a combined file made of many individual files using the following code

Code:
find inputfile.dap -type f -name "ASA-ads*.dap" -exec cat {} > outputfile.dap \;

However, when the data is concatenated, the years are grouped together randomly i.e.

Code:
2006.99960  31  12  2006  20  30 10531    245.851    539.703     13.245     90.006    320.299     77.632   1726.949      3.763 
2006.99968  31  12  2006  21  10 10531    246.049    539.310     13.765     89.212    319.926     77.785   1727.129      3.962 
2006.99975  31  12  2006  21  50 10531    245.749    539.796     13.780     90.154    319.921     77.487   1727.728      4.416 
2006.99983  31  12  2006  22  30 10531    246.021    540.320     13.269     90.244    320.180     77.321   1727.638      3.481 
2006.99990  31  12  2006  23  10 10531      0.000      0.000      0.000      0.000      0.000      0.000      0.000      0.000 
2002.00317   2   1  2002   3  47  8707    257.111    545.207     34.268     95.579    317.494     81.554   1756.145      5.212 
2002.00340   2   1  2002   5  47  8707    257.308    544.846     33.534     95.665    317.179     81.664   1758.244      5.483 
2002.00416   2   1  2002  12  27  8707      0.000      0.000      0.000      0.000      0.000      0.000   1749.232      0.000 
2002.00462   2   1  2002  16  27  8707    256.935    544.529     33.267     95.426    317.447     81.879   1748.079      5.164 
2002.00469   2   1  2002  17   7  8707    257.089    544.416     33.801     95.295    317.342     81.812   1747.684      5.099 
2002.00507   2   1  2002  20  27  8707    256.728    544.895     33.264     94.751    317.383     81.572   1749.352      5.107

Is there a way to sort the data chronologically?

Thanks
# 2  
Old 08-11-2011
Unless I am missing something, isn't the output you pasted already in order?

Looking at the first column:

2006.99960
2006.99968
2006.99975
2006.99983
2006.99990
2002.00317
2002.00340
2002.00416
2002.00462
2002.00469
2002.00507
# 3  
Old 08-11-2011
If just "sort" doesn't do what you want try to pipe it to
Code:
sort -n -k4,4 -k3,3 -k2,2

You can add -r for reverse sorting.
# 4  
Old 08-11-2011
hey dude2cool, I meant sort it from the earliest date first to the most recent date. There's 10 years worth of data, the years themselves are together but are grouped randomly.

Hi yazu, I tried piping through sort before and your code, but it doesn't seem to be working.

Thanks guys
# 5  
Old 08-11-2011
how about :

sort -n -k1, sorts on column one only.
This User Gave Thanks to dude2cool For This Post:
gd9629
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to calculate date and show data

data: hostcomment { host_name=myhost01 entry_type=1 comment_id=1 source=0 persistent=1 entry_time=1415723753 expires=0 expire_time=0 author=hpsm comment_data=IM0837437472 } program { modified_host_attributes=1 modified_service_attributes=1 enable_notifications=1... (20 Replies)
Discussion started by: SkySmart
20 Replies

2. Shell Programming and Scripting

Use awk to count and rearrange entries

How can I use awk to count the occurrence of field 2 and rearrange the output like below: Input: OA1 FM AA OA0 FM CC ON0 FM CC FN1 FN BB OY1 FN BB OY2 FN CC OY3 FN CC YT0 FM AA KW1 FN CC KW3 FM BB YT4 FM AA FN2 FT BB OA3 FT AA ON7 FM BB (14 Replies)
Discussion started by: aydj
14 Replies

3. Shell Programming and Scripting

Rearrange Lines with awk

I need to rearrange the lines in the input file in the example below: Input: LG1 R500 A-170 F1:81 F1:22 F2:32 F1:71 LG1 R700 A-203 F2:17 E2:18 LG1 R700 B-224 E1:9 LG2 R500 C-235 E2:9 F2:17 Output: LG1 R500 A-170 F1:81 LG1 R500 A-170 F1:22 LG1 R500 A-170 F2:32 LG1 R500 A-170... (2 Replies)
Discussion started by: aydj
2 Replies

4. Shell Programming and Scripting

Using awk to rearrange fields

Hi, I am required to arrange columns of a file i.e make the 15th column into the 1st column. I am doing awk 'begin {fs=ofs=","} {print $15,$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14}' ad.data>ad.csv the problem is that column 15 gets to column 1 but it is not comma separated with the... (10 Replies)
Discussion started by: seddoubt
10 Replies

5. Shell Programming and Scripting

how to rearrange a matrix with awk

Hi, every one. I have two files ,one is in matrix like this, one is a list with the same data as the matrix. AB AE AC AD AA AF SA 3 4 5 6 4 6 SC 5 7 2 8 4 3 SD 4 6 5 3 8 3 SE 45 ... (5 Replies)
Discussion started by: xshang
5 Replies

6. Shell Programming and Scripting

script to rearrange data.

Hello. I have data in the following format (the spaces at the beginning of lines are included): 1 2 2 0.39621 0.00000 1 2 2 0.00000+-0.0000 * 1 2 ... (5 Replies)
Discussion started by: andersgs
5 Replies

7. UNIX for Dummies Questions & Answers

Rearrange columns and rows with awk

Hello, I have the following problem I have two columns with numbers arranged as follows: x1 y1 x2 y2 .... .... x250 y250 Now I need them arranged as follows: "string a" x1 y1 x1 y2 "string b" "string a" x1 y2 x2 y2 (3 Replies)
Discussion started by: Tom46
3 Replies

8. Shell Programming and Scripting

awk sed cut? to rearrange random number of fields into 3 fields

I'm working on formatting some attendance data to meet a vendors requirements to upload to their system. With some help on the forums here, I have the data close. But they've since changed what they want. The vendor wants me to submit three fields to them. Field 1 is the studentid field,... (4 Replies)
Discussion started by: axo959
4 Replies

9. Shell Programming and Scripting

Rearrange data from 2 files

Dear All, I have the below files A file contains 1473 1649 1670 1758 1767 1784 B file contains 1242 1246 1264 1268 1284 (3 Replies)
Discussion started by: yahyaaa
3 Replies

10. Shell Programming and Scripting

Need help in AWK;Search String and rearrange columns

Hi AWK Experts, file1.txt contains: 29b11b820ddcc:-|OHad.perWrk|spn_id=AH111|spn_ordtyp=MY_REQ|msg_typ=ah.ntf.out|spn_ordid=928176|spn_nid=3|msg_strt=1175615334703|msg_que=oput|diff=371|17:48:55,074|17:48:55,084|10 file2.txt contains:... (2 Replies)
Discussion started by: spring_buck
2 Replies
Login or Register to Ask a Question