Transpose multiple rows (with a mix of space and enter) to a single column


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Transpose multiple rows (with a mix of space and enter) to a single column
# 1  
Old 07-25-2013
Transpose multiple rows (with a mix of space and enter) to a single column

How to change the uploaded weekly file data to the following format?
Code:
New Well_Id,Old Well_Id,District,Thana,Date,Data,R.L,WellType,Lati.,Longi.
BAG001,PT006,BARGUNA,AMTALI,1/2/1978,1.81,2.29,Piezometer,220825,901430
BAG001,PT006,BARGUNA,AMTALI,1/9/1978,1.87,2.29,Piezometer,220825,901430
BAG001,PT006,BARGUNA,AMTALI,1/16/1978,1.98,2.29,Piezometer,220825,901430
BAG001,PT006,BARGUNA,AMTALI,1/23/1978,2.03,2.29,Piezometer,220825,901430
BAG001,PT006,BARGUNA,AMTALI,1/30/1978,2.03,2.29,Piezometer,220825,901430

Input file:
Code:
                      GROUNDWATER CIRCLE-2,EDPCELL,BWDB
                  Depth to Watertable from Measuring Point
                     Measuring Frequency Monday 0600 hrs
               End Year 88.88 Missing Data 99.99 Unit in metre
New Well_Id :  BAG001       Old Well_Id :  PT006      
District    :  BARGUNA            Thana :  AMTALI             
No. of Change  1
Well Type   Date      Village            Lati.   Longi.   R.L    P.H   Depth 
Piezometer  11.12.77  Amtali             220825  901430    2.29  0.46   8.53
Piezometer  30.12.85  Amtali             220825  901430    2.29  0.46  23.47
------------------------------------------------------------------------------
Yr. 1978  St. Date Jan 02
  1.81  1.87  1.98  2.03  2.03  2.11  2.16  2.14  2.16  2.24  2.29  2.39  2.34
  2.24  2.26  2.14  2.21  2.24  2.14  1.88  1.68  1.47  1.42  1.53  1.50  1.20
  1.20  1.17  1.12  1.14  1.14  1.17  1.12  1.14  0.99  1.17  1.42  1.45  1.40
  1.32  1.42  1.45  1.47  1.53  1.50  1.53  1.47  1.53  1.63  1.60 99.99 99.99
 88.88
Yr. 1979  St. Date Jan 01
  1.70  1.73  1.98  1.96  1.98  2.01  2.01  1.93  1.88  1.85  1.85  1.83  1.91
  1.85  1.83  1.81  1.78  1.73  1.68  1.73  1.73  1.63  1.65  1.60  1.68  1.63
  1.60  1.55  1.50  1.58  1.53  1.32  1.22  1.14  1.14  1.22  1.45  1.42  1.45
  1.40  1.42  1.45  1.53  1.58  1.53  1.73  1.63  1.65  1.70  1.75  1.63  1.75
  1.75 88.88


Last edited by Scott; 07-25-2013 at 06:42 PM.. Reason: Please use code tags
# 2  
Old 07-25-2013
Here is an awk program that you can start working on.

Note that it will be a bit slow because the code is calling GNU date to perform date arithmetic:
Code:
awk '
        BEGIN {
                print "New Well_Id,Old Well_Id,District,Thana,Date,Data,R.L,WellType,Lati.,Longi."
        }
        /Well_Id/ {
                nwid = $4
                owid = $8
        }
        /District/ {
                dist = $3
                than = $6
        }
        /Well Type/ {
                getline
                wtype = $1
                lati = $4
                logi = $5
                rl = $6
        }
        /^Yr/ {
                DT = sprintf ( "%d-%s-%d", $NF, $(NF-1), $2 )
                cmd = "date -d\""DT"\" +%m/%d/%Y"
                cmd | getline DT
                close ( cmd )
        }
        /^[ ]*[0-9]+/ {
                for ( i = 1; i <= NF; i++ )
                {
                        print nwid, owid, dist, than, DT, $i, rl, wtype, lati, logi
                        cmd = "date -d\""DT" +7 days\" +%m/%d/%Y"
                        cmd | getline DT
                        close ( cmd )
                }
        }
' OFS=, BAG001.txt

This User Gave Thanks to Yoda For This Post:
# 3  
Old 07-26-2013
Speed up date by using an associative array
Code:
BEGIN {
.
.
.
        MONTH["Jan"]=1
        MONTH["Feb"]=2
        MONTH["Mar"]=3
        MONTH["Apr"]=4
        MONTH["May"]=5
        MONTH["Jun"]=6
        MONTH["Jul"]=7
        MONTH["Aug"]=8
        MONTH["Sep"]=9
        MONTH["Oct"]=10
        MONTH["Nov"]=11
        MONTH["Dec"]=12
        }

Then when you parse for date just do...
Code:
/^Yr./          {
                year=$2
                month=MONTH[$5]
                day=int($6)
                }

I am not sure the sample output file is really consistent...it's not clear how to parse the numeric data. Next time show a complete output file.
# 4  
Old 07-26-2013
@ Yoda, Thanks for your quick reply.

But there are some small errors that still exists. For example,
1. some extra enters (I highlighted them in the uploaded file)
2. It counted 88.88 against some dates which I dont want. (highlighted with color red in the uploaded file)

@ Yoda and others,

I uploaded a sheet of xls, for having the exact format I am looking for.

I have approximately 1265 txt files in a same folder which I need to sort similarly. Can you also suggest me a loop for this?

I really appreciate your help. Looking forward to hear you soon.

Last edited by sara.nowreen; 07-27-2013 at 06:12 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to transpose pieces of data in a column to multiple rows?

Hello Everyone, I am very new to the world of regular expressions. I am trying to use grep/sed for the following: Input file is something like this and there are multiple such files: abc 1 2 3 4 5 ***END*** abc 6 7 8 9 ***END*** abc 10 (2 Replies)
Discussion started by: shellnewuser
2 Replies

2. Programming

To transpose rows to column in hadoop

Hi, i am having an HDFS file which is comma seperated, i need to transpose from rows to column only the header columns text.csv cnt,name,place 1,hi,nz 2,hello,aus I need cnt, name, place while using below command in hadoop getting the error hadoop fs -fmt -1 text.csv (0 Replies)
Discussion started by: rohit_shinez
0 Replies

3. Shell Programming and Scripting

Converting Single Column into Multiple rows

Hi .. anyone can you help me ? i need to convert text below into multiple columns interface; GigabitEthernet0/0/0/0 description; TRUNK_PE-D2-JT2-VPN_Gi0/0/0/0_TO_ME4-A-JKT-JT_4/1/1_1G mtu 9212 negotiation auto interface; GigabitEthernet0/0/0/0.11 description; tes encapsulation;... (1 Reply)
Discussion started by: mad3linux
1 Replies

4. Shell Programming and Scripting

Convert single column into multiple rows

Convert Single column to multiple rows file a.txt contains data like below Server=abc Run=1 Tables=10 Sessions=16 Time=380 Jobs=5 Server=abc Run=2 Tables=15 Sessions=16 Time=400 Jobs=5 Server=abc Run=3 Tables=20 Sessions=16 Time=450 (5 Replies)
Discussion started by: sol_nov
5 Replies

5. Shell Programming and Scripting

Converting Single Column into Multiple rows, but with strings to specific tab column

Dear fellows, I need your help. I'm trying to write a script to convert a single column into multiple rows. But it need to recognize the beginning of the string and set it to its specific Column number. Each Line (loop) begins with digit (RANGE). At this moment it's kind of working, but it... (6 Replies)
Discussion started by: AK47
6 Replies

6. UNIX for Dummies Questions & Answers

[SOLVED] splitting a single column(with spaces) into multiple rows

Hi All, My requisite is to split a single column of phonemes seperated by spaces into multiple rows. my input file is: a dh u th a qn ch A v U r k my o/p should be like: adhu a dh u (3 Replies)
Discussion started by: girlofgenuine
3 Replies

7. Shell Programming and Scripting

How to merge multiple rows into single row if first column matches ?

Hi, Can anyone suggest quick way to get desired output? Sample input file content: A 12 9 A -0.3 2.3 B 1.0 -4 C 34 1000 C -111 900 C 99 0.09 Output required: A 12 9 -0.3 2.3 B 1.0 -4 C 34 1000 -111 900 99 0.09 Thanks (3 Replies)
Discussion started by: cbm_000
3 Replies

8. Shell Programming and Scripting

awk transpose rows to column

Need to transpose in awk rows to column like this: input: A1,6,5,4 3,2,1, A2,8,7,9,10,11,12,13,14 A3,1,2,3,5,7,8,9 A4,9,4,8,1,5,3, output: A1,1 A1,2 A1,4 ... A2,7 A2,8 ... A3,1 A3,2 ... A4,1 A4,3 (5 Replies)
Discussion started by: sdf
5 Replies

9. Shell Programming and Scripting

Single column into multiple rows

Hi all, I need to convert this file having just one column into two column file current file: a 15 b 21 c 34 d 48 e 10 wanted: a 15 b 21 c 34 (15 Replies)
Discussion started by: prachiagra
15 Replies

10. Shell Programming and Scripting

Converting Single Column into Multiple rows

i have single column which is starting with same string(many number of rows) i have to convert each into a single row.how can i do that? laknar std mes 23 55 laknar isd phone no address amount 99 I have to convert above like below. laknar|std|mes|23|55 laknar|isd|phone... (3 Replies)
Discussion started by: laknar
3 Replies
Login or Register to Ask a Question