Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Awk/sed script for transposing any number of rows with header row

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk/sed script for transposing any number of rows with header row
# 1  
Old 04-23-2013
Question Awk/sed script for transposing any number of rows with header row
Greetings!


I have been trying to find out a way to take a CSV file with a large number of rows, and a very large number of columns (in the thousands) and convert the rows to a single column of data, where the first row is a header representing the attribute name and the subsequent series of rows contains the value.

For instance:

In my CSV:
Code:
Row1: (header) Letter, Weight, Color, Cost
Row2: A, 20, Blue, 5
Row3: DD, 200, Orange, 100
...  (and so forth)


I am trying to get the output to be:
Code:
Letter,A
Weight,20
Color,Blue
Cost,5

Letter,DD
Weight,200
Color,Orange
Cost,100

I found this awk code is useful:

Code:
BEGIN {FS=OFS=","}

{
for (i=1;i<=NF;i++)
{
 arr[NR,i]=$i;
 if( big <=  NF)
  big=NF;
 }
}

END {
  for(i=1;i<=big;i++)
   {
    for(j=1;j<=NR;j++)


    printf("%s%s",arr[j,i], (j==NR ? "" : OFS));
    #printf("%s%s",arr[j,i], (j==NR ? "" : OFS));
    print "";
}
}

but this combines the values:
Code:
Letter,A,DD
Weight,20,200
Color,Blue,Orange
Cost,5,100

So in the end, i want to keep them completely separate. I am sure the answer is simple, but I am new to awk/sed and am having some difficulties figuring out what the trouble is. Anybody that can help me out, I would learn tremendously from the example. Thank you in advance!!
Last edited by Franklin52; 04-24-2013 at 04:13 AM.. Reason: Please use code tags
# 2  
Old 04-24-2013
An awk program:
Code:
awk -F, '
        NR == 1 {
                        gsub ( " ", X )
                        split ( $0, H, "," )
        }
        NR > 1 {
                        for ( i = 1; i <= NF; i++ )
                                print H[i], $i
                        printf "\n"
        }
' OFS=, file.csv

# 3  
Old 04-24-2013
Thank you Yoda! This worked like a charm. There was something I forgot to mention that you'll probably know how to address... what if one of the values in the header is something like "PLANET" and I want to tack on whatever that planet is to the front of every line. So for instance:


Row1: (header) Letter, Weight, Color, Cost, PLANET
Row2: A, 20, Blue, 5, MARS
Row3: DD, 200, Orange, 100, REMULAK
... (and so forth)

We now get:

MARS, Letter,A
MARS, Weight,20
MARS, Color,Blue
MARS, Cost,5

REMULAK, Letter,DD
REMULAK, Weight,200
REMULAK, Color,Orange
REMULAK, Cost,100


Thank you Master... Smilie
# 4  
Old 04-24-2013
Code:
awk -F, '
        NR == 1 {
                        gsub ( " ", X )
                        split ( $0, H, "," )
        }
        NR > 1 {
                        for ( i = 1; i <= (NF-1); i++ )
                                print $NF, H[i], $i
                        printf "\n"
        }
' OFS=, file.csv

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reseting row count every given number of rows

I have a file with 48 rows. I am counting 6 rows and adding 6 to that number and repeating the operation, and then output the value in column 1. For the second column, I would like to get sort of a binary output (1s and 2s) every 3rd row. This is what I have: awk '{print ++src +... (1 Reply)
Discussion started by: Xterra
1 Replies

2. Shell Programming and Scripting

awk to skip header row and add string to field

The awk below does put in VUS in the 9th field but I can not seem to skip the header then add the VUS. I tried to incorporate NR >=2 and NR > 1 with no luck. Thank you :). input Chr Start End Ref Alt Func.refGene PopFreqMax CLINSIG Classification chr1 43395635 ... (5 Replies)
Discussion started by: cmccabe
5 Replies

3. Shell Programming and Scripting

At text to field 1 of header row using awk

I am just trying to insert the word "Index" using awk. The below is close but seems to add the word at the end and I can not get the syntax correct to add from the beginning. Thank you :). awk -F'\t' -v OFS='\t' '{ $-1=$-1 OFS "Index"}$1=$1' file current output Chr Start End ... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

Match in awk skipping header rows

I am trying to match $1-$7 between the two files and if a match is found then the contents of $8 in file2 and copied over. The awk I tried is below. There is also a header row in file2 that has the Chr Start End Ref Alt that does not need to be searched. Thank you :). awk awk... (3 Replies)
Discussion started by: cmccabe
3 Replies

5. UNIX for Dummies Questions & Answers

append column and row header to a file in awk script.

Hi! Is there a way to append column and row header to a file in awk script. For example if I have Jane F 39 manager Carlos M 40 system administrator Sam F 20 programmer and I want it to be # name gend age occup 1 Jane F 39 manager 2 Carlos M ... (4 Replies)
Discussion started by: FUTURE_EINSTEIN
4 Replies

6. Shell Programming and Scripting

Combining multiple rows in single row based on certain condition using awk or sed

Hi, I'm using AIX(ksh shell). > cat temp.txt "a","b",0 "c",bc",0 "a1","b1",0 "cc","cb",1 "cc","b2",1 "bb","bc",2 I want the output as: "a","b","c","bc","a1","b1" "cc","cb","cc","b2" "bb","bc" I want to combine multiple lines into single line where third column is same. Is... (1 Reply)
Discussion started by: samuelray
1 Replies

7. Shell Programming and Scripting

awk, string as record separator, transposing rows into columns

I'm working on a different stage of a project that someone helped me address elsewhere in these threads. The .docs I'm cycling through look roughly like this: 1 of 26 DOCUMENTS Copyright 2010 The Age Company Limited All Rights Reserved The Age (Melbourne, Australia) November 27, 2010... (9 Replies)
Discussion started by: spindoctor
9 Replies

8. Shell Programming and Scripting

how to add the number of row and count number of rows

Hi experts a have a very large file and I need to add two columns: the first one numbering the incidence of records and the another with the total count The input file: 21 2341 A 21 2341 A 21 2341 A 21 2341 C 21 2341 C 21 2341 C 21 2341 C 21 4567 A 21 4567 A 21 4567 C ... (6 Replies)
Discussion started by: juelillo
6 Replies

9. Shell Programming and Scripting

shell script(Preferably awk or sed) to print selected number of columns from each row

Hi Experts, The question may look very silly by seeing the title, but please have a look at it clearly. I have a text file where the first 5 columns in each row were supposed to be attributes of a sample(like sample name, number, status etc) and the next 25 columns are parameters on which... (3 Replies)
Discussion started by: ks_reddy
3 Replies

10. Shell Programming and Scripting

SED: delete matching row and 4 next rows?

Hi, Tried to look for solution, and found something similar but could not adapt the solution for my needs.. I'm trying to match a pattern (in this case "ProcessType")in a logfile, then delete that line and the 4 following lines. The logfile looks as follows: ProcessType: PROCESS_A... (5 Replies)
Discussion started by: Juha
5 Replies
Login or Register to Ask a Question