converting rows into columns


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting converting rows into columns
# 15  
Old 01-04-2011
Quote:
Originally Posted by Chubler_XL
The supplied code only works with GNU awk ...
Really?
Code:
$ mawk '$1=$1' RS= infile685
PC_1 wf_test1 Test
PC_2 wf_test2 Test
PC_3 wf_test3 Test

Quote:
If RS is null, then records are separated by sequences consisting of a <newline> plus one or more blank lines, leading or trailing blank lines shall not result in empty records at the beginning or end of the input, and a <newline> shall always be a field separator, no matter what the value of FS is
http://pubs.opengroup.org/onlinepubs...ag_20_06_13_03
# 16  
Old 01-04-2011
Scrutinizer,

It's not the null RS that's the issue it's the stripping of \n from $0 when assigning fields, here is my results using awk on AIX:


Code:
$ awk '$1=$1' RS= infile

Code:
PC_1
wf_test1 Test PC_2 wf_test2 Test PC_3 wf_test3 Test


Additional debug:
Code:
$ awk '$1=$1{print "<"$0">"}' RS= infile

Code:
<PC_1 wf_test1 Test> <PC_2 wf_test2 Test> <PC_3 wf_test3 Test>



===Update===

Funny enough on AIX 5.3 this does work:
Code:
awk '$1=$1""' RS= infile


Last edited by Chubler_XL; 01-04-2011 at 07:03 PM..
# 17  
Old 01-04-2011
Then IMO that awk does not work according to the POSIX Specification because newline should turn into a field separator no matter what the value of FS, as per the previous quote in #15. and further:
Quote:
The symbol $0 shall refer to the entire record; setting any other field causes the re-evaluation of $0
awk

This accomplished by assigning $1 to $1. And by re-evaluating $0, the field separators are changed to OFS which is a single space.
Is there a nawk on AIX?

---------- Post updated at 00:18 ---------- Previous update was at 00:05 ----------

Quote:
Originally Posted by Chubler_XL
[..]

===Update===

Funny enough on AIX 5.3 this does work:
Code:
awk '$1=$1""' RS= infile

Hmm forcing into string context makes a difference with that AIX implementation. Perhaps that implementation has an 'optimization' that if you assign the same value is is wrongly assumed that nothing needs to be done and by adding the double quotes it is regarded as different? A bit fishy, no?
This User Gave Thanks to Scrutinizer For This Post:
# 18  
Old 01-04-2011
Yep, it looks like AIX awk is incorrectly optimising this particular assignment, there is a /usr/bin/nawk but it's a hard link to awk and has the same issue.

Interesting, but I doubt this is going to be the OPs problem, unless he is also on AIX 5.3.
This User Gave Thanks to Chubler_XL For This Post:
# 19  
Old 01-05-2011
When I was real serious about rotating I wrote this C tool:
Code:
colrot.c:

#include <stdio.h>
#include <string.h>

static char    buf[200000000];
static char    **bufs = NULL ;
static char    *cp, *cp2, *bcp ;
static char    *isep = "|\n" ;
static char    *osep = "\t" ;
static char    *bsep = "" ;
static int    rows = 1 ;
static int    cols ;
static int    col ;
static int    row ;
static int    bct = 0 ;

static int p_gets( char *b, size_t s )
{
    if ( !fgets( b, s, stdin ) )
    {
        if ( ferror( stdin ) )
        {
            perror( "stdin" );
            exit( 1 );
        }

        *b = NULL ;
        return 1 ;
    }

    return 0 ;
}

static void p_puts( char *b )
{
    if ( EOF == fputs( b, stdout ) )
    {
        if ( ferror( stdout ) )
        {
            perror( "stdout" );
            exit( 1 );
        }

        exit( 0 );
    }
}

main( int argc, char **argv )
{
    /* get $1 */
    for ( col = 1 ; col < argc ; col++ )
    {
        if ( !strcmp( argv[col], "-is" )
          && ++col < argc )
        {
            *isep = argv[col][0] ;
            continue ;
        }

        if ( !strcmp( argv[col], "-os" )
          && ++col < argc )
        {
            osep = argv[col] ;
            continue ;
        }

        if ( !strcmp( argv[col], "-bs" )
          && ++col < argc )
        {
            bsep = argv[col] ;
            continue ;
        }

        if ( ( rows = atoi( argv[col] ) ) > 0 )
        {
            continue ;
        }

        fputs(
"\n"
"Usage: colrot [ -is <i_sepc> ] [ -os <o_seps> ] [ -bs <b_seps> ] [ <lpb> ]\n"
"\n"
"Reads and saves a header line, and then reads <lpb> (default one) line\n"
"blocks, and writes the header and each block rotated, so each column is\n"
"on a line.  The character <i_sepc> (default pipe=|) defines input fields.\n"
"String <o_seps> (default a tab character) separates output fields.  A line\n"
"with string <b_seps> (default empty) separates output blocks.\n"
"\n"
            , stderr );
        exit( 1 );
    }

    /* get column name line */
    if ( p_gets( buf, sizeof( buf ) ) )
        exit( 0 );

    /* count columns */
    for ( cp = buf, cols = 0 ; *cp ; cols++ )
    {
        cp = strpbrk( cp, isep );
        cp++ ;
    }

    /* malloc a matrix of char pointers */
    if ( !( bufs = (char**)malloc( sizeof(char*) * cols * ( rows + 1 ))))
    {
        perror( "malloc()" );
        exit( 1 );
    }

    /* find and null terminate column names */
    for ( cp = buf, col = 0 ; col < cols ; col++ )
    {
        bufs[col] = cp ;
        cp = strpbrk( cp, isep );
        *cp = NULL ;
        cp++ ;
    }

    bcp = cp ;

    loop:

    /* read in N rows above header; rows + 1 total */
    for ( row = 1 ; row <= rows ; row++ )
    {
        /* ignore EOF until starting a new block */
        if ( p_gets( cp, sizeof(buf) - ( cp - buf ) ) )
        {
            if ( row == 1 ) /* nothing buffered */
            {
                exit( 0 );
            }
            /* shrink the last block and continue */
            /* we will read EOF again on row 1 of the next pass */
            rows = row - 1 ;
            break ;
        }

        /* locate and null terminate all columns, fake missing cols */
        for ( col = 0 ; col < cols ; col++ )
        {
            bufs[ col + ( row * cols ) ] = cp ;
            cp += strcspn( cp, isep );

            /* if sep found, null term field and step into next */
            switch ( *cp )
            {
            case 0:
                break ;
            case '\n':
                *cp = NULL ;
                break ;
            default:
                *cp = NULL ;
                cp++ ;
            }
        }

        /* write next line just past last field null */
        cp++ ;
    }

    /* go through matrix column and then reverse row,
        marking empty fields for supression */
    for ( col = 0 ; col < cols ; col++ )
    {
        for ( row = rows ; row > 0 ; row-- )
        {
            if ( *( bufs[ col + ( row * cols ) ] ) )
            {
                break ;
            }
    
            /* NULL pointer means nothing here or beyond */
            bufs[ col + ( row * cols ) ] = NULL ;
        }
    }

    if ( bct++ )
    {
        /* put a line between blocks */
        p_puts( bsep );
        p_puts( "\n" );
    }

    /* go through matrix column and then row, printing */
    for ( col = 0 ; col < cols ; col++ )
    {
        for ( row = 0 ; row <= rows ; row++ )
        {
            /* NULL pointer means nothing here or beyond */
            if ( !bufs[ col + ( row * cols ) ] )
            {
                break ;
            }

            if ( row )
            {
                p_puts( osep );
            }

            p_puts( bufs[ col + ( row * cols ) ] );
        }

        p_puts( "\n" );
    }

    /* start another block, reading just above header */
    cp = bcp ;
    goto loop ;
}

# 20  
Old 01-06-2011
Hi DGPickett, that looks like something nice to try with a future case of transposition, but despite the title of this thread, there is no converting columns into rows here, is there?
# 21  
Old 01-07-2011
Quote:
Originally Posted by Scrutinizer
Hi DGPickett, that looks like something nice to try with a future case of transposition, but despite the title of this thread, there is no converting columns into rows here, is there?
That was my first comment! One column is a trivial, edge case. This C solution is valid, general, robust and can even do this little task.

If the title brings the needy here, this bit of code had been very good to me. I particularly found the conversion of "many rows of many columns" into "a column of header and one or few columns of data" to be very useful for inspection of bulky matrices. If you know the lines per row, you can jump up and down to see the same column in different rows, or search on the column name. You can take the data out of Excel or SQL and flip it around before putting in on paper or into Excel. Since Excel used to have a 255 column & 4096 row limit, wider rows could be rotated for inspection.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Converting columns of text to rows, with blank lines

I've spent the past hour trying different things and googling for this solution and cannot find the answer. Found variations of this, but not this exact thing. I have the following text, which is the output from our mainframe. Each field is on a separate line, with a blank line between each... (7 Replies)
Discussion started by: lupin..the..3rd
7 Replies

2. Shell Programming and Scripting

Converting rows into columns

hi all I need a help ..I have a script that takes has command and its output is like below.. a b a v v a c I am assigning the above outputs to a variable .. <variable name> = <command output> problem here is when I echo the variable ..it gives me output like " a b... (3 Replies)
Discussion started by: shankarb
3 Replies

3. Shell Programming and Scripting

Compare 2 csv files by columns, then extract certain columns of matcing rows

Hi all, I'm pretty much a newbie to UNIX. I would appreciate any help with UNIX coding on comparing two large csv files (greater than 10 GB in size), and output a file with matching columns. I want to compare file1 and file2 by 'id' and 'chain' columns, then extract exact matching rows'... (5 Replies)
Discussion started by: bkane3
5 Replies

4. UNIX for Dummies Questions & Answers

Converting txt output to rows and columns and send report via mail.

Hi All, I would like to send below output in a tabular column ( xml or excel ) and send a mail. vinay unix anil sql vamsee java request to suggest a solution. (1 Reply)
Discussion started by: Girish19
1 Replies

5. UNIX for Dummies Questions & Answers

Converting Columns To Rows Sort Of

Hi I'm a UNIX awk and sed novice at best. I'm trying to creat a .csv file so it can be graphed in Excel. Tried various xargs, awk, sed and paste but just can't seem to get the data to line up. Not sure if this is beyond for a question in these forums. Any help would greatly be appreciated. Have... (4 Replies)
Discussion started by: jimmyf
4 Replies

6. Shell Programming and Scripting

Converting rows to columns in csv file

Hi, I have a requirement to convert rows into columns. data looks like: c1,c2,c3,.. r1,r2,r3,.. p1,p2,p3,.. and so on.. output shud be like this: c1,r1,p1,.. c2,r2,p2,.. c3,r3,p3,.. Thanks in advance, (12 Replies)
Discussion started by: Divya1987
12 Replies

7. Shell Programming and Scripting

Converting rows to columns using shell script

I have a script which converts rows to columns. file_name=$1 mailid=$2 #CREATE BACKUP OF ORIGINAL FILE #cp ${file_name}.xlsx ${file_name}_temp.xlsx #tr '\t' '|' < ${file_name}_temp.xlsx > ${file_name}_temp.csv #rm ${file_name}_temp.xlsx pivot_row=`head -1 ${file_name}` sed 1d... (3 Replies)
Discussion started by: andy538
3 Replies

8. UNIX for Dummies Questions & Answers

Converting columns into rows

Is there anyway to convert columns into raws using awk? (or any other command line):eek::eek::eek::eek::eek::eek::eek::eek::eek: (1 Reply)
Discussion started by: cosmologist
1 Replies

9. UNIX for Dummies Questions & Answers

Converting rows into multiple-rows

Hi every one; I have a file with 22 rows and 13 columns which includes floating numbers. I want to parse the file so that every five columns in the row would be a new record (row). For example, the first line in the old file should be converted into three lines with first two lines contain 5... (6 Replies)
Discussion started by: PHL
6 Replies

10. Shell Programming and Scripting

how to converting rows to columns, bash

I have in file these words: @fraza1 = rw @fraza2 = r @fraza3 = r @fraza4 = r @fraza5 = r @fraza1 = r @fraza6 = r @fraza7 = r @fraza2 = r @fraza8 = r @fraza9 = r ... I would like so that: ,rw,@fraza1 ,r,@fraza2 (2 Replies)
Discussion started by: patrykxes
2 Replies
Login or Register to Ask a Question