Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Linux - Pivot Rows to Columns

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Linux - Pivot Rows to Columns
# 8  
Old 01-25-2016
Quote:
Originally Posted by RudiC
Be assured that it is working on a well-formed text file. Did you examine your input file is correctly structured?

You can set the field separator by either the -F, option in front of the "script" parameter or the variable definition FS=","
Hi Rudi,

Thanks for the above.

Unfortunately I am not able to get your command to work. Below is my data file:

Code:
ABC|00000001|15-Dec-15|13400
ABC|00000001|31-Jan-16|13490
ABC|00000001|29-Feb-16|13500
ABC|00000001|31-May-16|40200
ABC|00000001|31-Aug-16|42000
ABC|00000002|15-Dec-15|13400
ABC|00000002|31-Jan-16|13490
ABC|00000002|29-Feb-16|13500
ABC|00000002|31-May-16|40200
ABC|00000002|31-Aug-16|42000

And below is the code I am running:

Code:
 

 awk -F "|" '
NR == 1         {print "Col1\tCol2\tCol3\tCol4\tCol5\tCol6\tCol7\tCol8\tCol9\tCol10\tCol110\tCol12"
                 next
                }
                {IX = $1 FS $2
                }
IX != LAST      {printf "%s%s", SEP, $0
                 LAST = IX
                 SEP = ORS
                 next
                }
                {sub ($1 "[     ]*" $2 "[       ]*", "")
                 printf "\t%s", $0
                }
END             {print ""
                }
' RWPivotTest.csv

Below is the output result I am getting:

Code:
Col1 Col2 Col3 Col4 Col5 Col6 Col7 Col8 Col9 Col10 Col110 Col12
ABC|00000001|31-Jan-16|13,490 ABC|00000001|29-Feb-16|13,500 ABC|00000001|31-May-16|40,200 ABC|00000001|31-Aug-16|42,000
ABC|00000002|15-Dec-15|13,400 ABC|00000002|31-Jan-16|13,490 ABC|00000002|29-Feb-16|13,500 ABC|00000002|31-May-16|40,200 ABC|00000002|31-Aug-16|42,000

Are you able to assist further? Smilie
Last edited by Don Cragun; 01-25-2016 at 03:18 PM.. Reason: Add CODE tags for output.
# 9  
Old 01-25-2016
Quote:
Originally Posted by RichZR
.
.
.
Unfortunately I am not able to get your command to work.
.
.
.
OF COURSE NOT! You changed the structure of your data file in a way that makes my snippet fail:
- The header is gone
- the field separator is a different one (which on its own is already fatal)
- the FS is a meta char for regexes, so additional measures must be taken

Why don't you give correct, representative sample data from the start?

Try:
Code:
awk -F\| '
NR == 1         {print "Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12"
                }

                {IX = $1 FS $2
                }
IX != LAST      {printf "%s%s", SEP, $0
                 LAST = IX
                 SEP = ORS
                 next
                }
                {sub ($1 "\\" FS "*" $2 "\\" FS "*", "")
                 printf "%s%s", FS, $0
                }
END             {print ""
                }
'  file
Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12
ABC|00000001|15-Dec-15|13400|31-Jan-16|13490|29-Feb-16|13500|31-May-16|40200|31-Aug-16|42000
ABC|00000002|15-Dec-15|13400|31-Jan-16|13490|29-Feb-16|13500|31-May-16|40200|31-Aug-16|42000

Last edited by RudiC; 01-25-2016 at 05:06 PM.. Reason: typo
This User Gave Thanks to RudiC For This Post:
RichZR
# 10  
Old 01-25-2016
As long as the rows are consistent (if there were not, the header can not be 12)

Code:
perl -nle 'BEGIN{print "Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12"}
 /^(\w+\|\d+)(\|.*$)/ and $id{$1} .= $2; if($. % 5 == 0){print "$1$id{$1}"; %id = ()}' richzr.file

Code:
Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|Col9|Col10|Col11|Col12
ABC|00000001|15-Dec-15|13400|31-Jan-16|13490|29-Feb-16|13500|31-May-16|40200|31-Aug-16|42000
ABC|00000002|15-Dec-15|13400|31-Jan-16|13490|29-Feb-16|13500|31-May-16|40200|31-Aug-16|42000

This User Gave Thanks to Aia For This Post:
RichZR
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rows to columns

Hi, I have a text file with records as below Service Contract: Account Type: Client Number: Group Number: Account Currency: I want to print 2nd,3rd and 5th as a separate column, like -> Account Type: ,Client Number: ,Account Currency: How can I do that? (1 Reply)
Discussion started by: dsid
1 Replies

2. Shell Programming and Scripting

Pivot Rows to Columns, with field separator

Hi All, I have a requirement to flatten data out, based on the value in COL_2. Our file is pipe delimited, however COL_2 contains a comma separated string, which we would like to pivot out from one row into multiple rows. Please see my example input data below: Input Data ... (4 Replies)
Discussion started by: RichZR
4 Replies

3. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

4. Shell Programming and Scripting

Columns to rows

Hi, I have a big file, with thousands of rows, and I want to put every 7 rows in a line. Input file: str1, val2, val3 str2, val4, val5 str3, val22, val33 str4, val44, val55 str5, val6, val7 str6, val77, val88 str7, val99, val00 str1, som2, som3 str2, som4, som5 str3, som22, som33 ... (11 Replies)
Discussion started by: apenkov
11 Replies

5. Shell Programming and Scripting

Evaluate 2 columns, add sum IF two columns match on two rows

Hi all, I know this sounds suspiciously like a homework course; but, it is not. My goal is to take a file, and match my "ID" column to the "Date" column, if those conditions are true, add the total number of minutes worked and place it in this file, while not printing the original rows that I... (6 Replies)
Discussion started by: mtucker6784
6 Replies

6. Shell Programming and Scripting

Deleting all the fields(columns) from a .csv file if all rows in that columns are blanks

Hi Friends, I have come across some files where some of the columns don not have data. Key, Data1,Data2,Data3,Data4,Data5 A,5,6,,10,, A,3,4,,3,, B,1,,4,5,, B,2,,3,4,, If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies

7. Shell Programming and Scripting

Rows to Columns

Hi Guru's, I have a requirement where i need to convert rows to column based on a key column. Input: Account_id|Trip_Org|Trip_Dest|City|Hotel_Nm 123|DFW|CHI|Dallas|Hyatt 123|LAS|LPA|Vegas|Hyatt Palace Output:... (6 Replies)
Discussion started by: rakesh5300
6 Replies

8. Windows & DOS: Issues & Discussions

Columns to Rows

I want to create a script with gawk. I have the following file with 2 columns: A 1 A 2 A 3 B 1 B 2 B 3 C 1 C 2 D 1 D 2 D 3 D 4 and i want to convert to: (1 Reply)
Discussion started by: sameeribraimo
1 Replies

9. Shell Programming and Scripting

Rows into columns?

I have a file thats space delimited that looks something like this: Joe Smith jsmith 43234 bill1;bill2;read;read2;schedule Andy Summers asummers 11232 bill1;read Beth McConnel bmconnel 43443 read;read2;schedule;bill Susan Fowler sfowler 09332 bill1;read;schedule I need to... (8 Replies)
Discussion started by: regexnub
8 Replies

10. Shell Programming and Scripting

# of rows and columns

Hi, Does anyone know the command to know the # of rows and columns for a file? thanks (3 Replies)
Discussion started by: kylle345
3 Replies
Login or Register to Ask a Question