Grep from files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Grep from files
# 1  
Old 05-26-2013
Grep from files

I have file input

Code:
1_a> xxx . 11
Start              A B C D E F            G            H           I                 J
=================================================================================================================
Finish=1           0 0 3 3 3 0 (DISABLED) 0 (DISABLED) 1 (ENABLED) 1 (ENABLED) 1 (ENABLED)

2_a> xxx . 11
Start              C D E F            G            H           I                 J
=================================================================================================================
Finish=1           3 3 3 0 (DISABLED) 1 (DISABLED) 0 (ENABLED) 1 (ENABLED) 0 (ENABLED)

i expect the output :

Code:
1_a,A,B,C,D,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1 
2_a,C,D,E,F,G,H,I,J
2_a,3,3,3,0,1,0,1,0

# 2  
Old 05-26-2013
Assuming your data is in a file named t.data this is one way:

Code:
awk '
    />/ { split( $1, a, ">" ); sect=a[1]; next; }

    /Start/ {
        s = "";
        printf( "%s ", sect );

        for( i = 2; i <= NF; i++ )
        {
            printf( "%c%s", s, $(i) );
            s = ",";
        }
        printf( "\n" );
        next;
    }

    /Finish/ {
        s = "";
        printf( "%s ", sect );
        for( i = 2; i <= NF; i++ )
        {
            if( substr( $(i), 1, 1 ) != "(" )  #if not a disabled/enabled thing
            {
                printf( "%c%s", s, $(i) );
                s = ",";
            }
        }
        printf( "\n" );
    }
' t.data

This User Gave Thanks to agama For This Post:
# 3  
Old 05-26-2013
Another awk approach:
Code:
awk '
        />/ {
                gsub ( />.*/, X )
                h = $0
        }
        /Start/ {
                gsub ( "Start", X )
                $1 = $1
                s = h OFS $0
        }
        /Finish/ {
                gsub (/Finish=.|\(DISABLED\)|\(ENABLED\)/, X )
                $1 = $1
                t = h OFS $0
                print s RS t
        }
' OFS=, file

This User Gave Thanks to Yoda For This Post:
# 4  
Old 05-26-2013
thanks Mr Agama & Mr Yoda

i need another step on my data. After executing the script above, the result is

Code:
1_a,A,B,C,D,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1 
2_a,C,D,E,F,G,H,I,J
2_a,3,3,3,0,1,0,1,0
3_a,A,D,E,F,G,H,I,J
3_a,3,3,3,0,1,0,1,0

I want to process the result again to collect the data in particular order

expected output
Code:
x_a,A,B,C,D,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1
2_a,0,0,3,3,3,0,1,0,1,0
3_a,3,0,0,3,3,0,1,0,1,0

thanks
# 5  
Old 05-26-2013
Are your headings and data values always 1 character long?
# 6  
Old 05-26-2013
Headings are in different amount of character Mr Don

input
Code:
1_a,A,BX,C,DYY,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1 
2_a,C,DYY,E,F,G,H,I,J
2_a,3,3,3,0,1,0,1,0
3_a,A,DYY,E,F,G,H,I,J
3_a,3,3,3,0,1,0,1,0
4_a,J
4_a,1

output

Code:
x_a,A,BX,C,DYY,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1
2_a,0,0,3,3,3,0,1,0,1,0
3_a,3,0,0,3,3,0,1,0,1,0
4_a,0,0,0,0,0,0,0,0,0,1

# 7  
Old 05-26-2013
Have a go with either of the programmes below. If your input file is huge, you may run into issues because these load everything into memory. There are ways to work round those issues, but have a try with these before switching to a method that involves two passes over the data.

Code:
awk '
    />/ {
        split( $1, a, ">" );
        sect_order[sidx++] = a[1];
        next;
    }

    /Start/ {
        split( "", columns, "." );  # delete is not supported in all awk flavours; this deletes values in columns
        for( i = 2; i <= NF; i++ )
        {
            if( length( $(i) ) > maxl )     # for alignment if that is needed
                maxl = length( $(i) );

            if( !header_seen[$(i)]++ )
                header_order[hidx++] = $(i);

            columns[i] = $(i);
            s = ",";
        }
        next;
    }

    /Finish/ {
        j = 0;
        for( i = 2; i <= NF; i++ )
        {
            if( substr( $(i), 1, 1 ) != "(" )
            {
                values[sect,columns[j]] = $(i);
                j++
            }
        }

    }

    END {
        fmt = sprintf( ",%%%dd", maxl );            # for column alignment
        fmth = sprintf( ",%%%ds", maxl );

        printf( "x_a" );                            # print header record
        for( j = 0; j < hidx; j++ )
            printf( fmth, header_order[j] );
        printf( "\n" );

        for( i = 0; i < sidx; i++ )                 # print each section
        {
            sect = sect_order[i];
            printf( "%s", sect );
            for( j = 0; j < hidx; j++ )
                printf( fmt, values[sect,header_order[j]] );
            printf( "\n" );
        }
    }
' t.data

It will align columns padded with spaces. If you don't need that, then the code below is a bit more simple. In both cases, the order of the headers is the order that they are encountered in the input data. A sort could be added if needed.


Code:
awk '
    />/ {
        split( $1, a, ">" );
        sect_order[sidx++] = a[1];
        next;
    }

    /Start/ {
        split( "", columns, "." );  # delete is not supported in all awk flavours; this deletes values in columns
        for( i = 2; i <= NF; i++ )
        {
            if( !header_seen[$(i)]++ )
                header_order[hidx++] = $(i);

            columns[i] = $(i);
            s = ",";
        }
        next;
    }

    /Finish/ {
        j = 0;
        for( i = 2; i <= NF; i++ )
        {
            if( substr( $(i), 1, 1 ) != "(" )
            {
                values[sect,columns[j]] = $(i);
                j++
            }
        }

    }

    END {
        printf( "x_a" );                            # print header record
        for( j = 0; j < hidx; j++ )
            printf( ",%s", header_order[j] );
        printf( "\n" );

        for( i = 0; i < sidx; i++ )                 # print each section
        {
            sect = sect_order[i];
            printf( "%s", sect );
            for( j = 0; j < hidx; j++ )
                printf( ",%d", values[sect,header_order[j]] );
            printf( "\n" );
        }
    }
' t.data


Last edited by agama; 05-26-2013 at 09:48 PM.. Reason: Corrected bug
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk and grep using two files

I have two file one will have WWN and port details and another file will have allies name. I need to check what allies name was given to each wwn and create a consolidate report which will both port details, allies name. file one switchshow 133 1 21 188500 id N4 ... (4 Replies)
Discussion started by: ranjancom2000
4 Replies

2. Shell Programming and Scripting

Python - glob () - How to grep same files with different extension files

Hi I Have a directory and i have some files below abc.txt abc.gif gtee.txt ghod.pid umni.log unmi.tar How can use glob function to grep abc files , i have created a variable "text" and i assigned value as "abc", please suggest me how can we use glob.glob( ) to get the output as below... (2 Replies)
Discussion started by: kumar85shiv
2 Replies

3. Shell Programming and Scripting

grep 1000s of files with 1000s of grep values

Hi, I have around 200,000 files in a given directory. I need to cat each of these files and grep them for thousands of identifier values (or strings) in a given text file. The text file looks something like this: 1234 1243545 1234353 121324 etc with thousands of entries. Can... (3 Replies)
Discussion started by: mantis
3 Replies

4. Shell Programming and Scripting

grep all files in a directory

I am not sure if i am doing this correctly since it returns quickly. i need to grep for a keyword in all files in a directory grep keyword /mydirectory is that correct? I just want to know which files have a keyword in it. i am using korn shell in solaris 5.1. There does not appear to... (12 Replies)
Discussion started by: guessingo
12 Replies

5. Shell Programming and Scripting

grep for certain files using a file as input to grep and then move

Hi All, I need to grep few files which has words like the below in the file name , which i want to put it in a file and and grep for the files which contain these names and move it to a new directory , full file name -C20091210.1000-20091210.1100_SMGBSC3:1000... (2 Replies)
Discussion started by: anita07
2 Replies

6. Shell Programming and Scripting

Need to grep for .log files

Hi, Have two files. 1. robin.log 2. robin_log When is grep i shuld get only robin.log file. Please help. I Tried ls | grep ".log" But its not working Thanks Robin (6 Replies)
Discussion started by: robinbannis
6 Replies

7. Shell Programming and Scripting

Grep Different Files Using a Loop?

I have a script to GREP for a text expression within certain files, the files being named file.11012008 thru file.11302008. 30 files in all, one for each day of the month. Instead of entering the following 3 lines of code 30 different times, I'm trying to find a way to loop the process: ... (6 Replies)
Discussion started by: foleyml
6 Replies

8. Shell Programming and Scripting

how to grep word from .gz files & .z files

Hi I have find some account numbers from .gz files. There thousands of .gz files, also i am unable to grep .gz / .z files. I want to find file names & and those lines from list. Could you pls tell me / give me any syntax for finding word from ,gz files using grep? Thanks in advance (8 Replies)
Discussion started by: udaya_subbu
8 Replies

9. Shell Programming and Scripting

grep from .z files

There are archived files with .Z in the server. I want to grep a code within those files. How can I do this? Please help. Koho (4 Replies)
Discussion started by: koho
4 Replies

10. Shell Programming and Scripting

Script to grep several files

Hi I am totally new to Unix and I am in need of a script that can grep several files at once. the script needed: change directory grep for keywords "error" "fail" "warning" in different files with different names, same extention but specific file names. if any of the keywords is found to show... (4 Replies)
Discussion started by: sqloyd
4 Replies
Login or Register to Ask a Question