Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Grep from files

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Grep from files
# 8  
Old 05-26-2013
Hi agama,
When I tried your scripts, the headers came out fine, but all of the data fields were zero:
Code:
x_a,  A, BX,  C,DYY,  E,  F,  G,  H,  I,  J
1_a,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0
2_a,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0
3_a,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0
    and
x_a,A,BX,C,DYY,E,F,G,H,I,J
1_a,0,0,0,0,0,0,0,0,0,0
2_a,0,0,0,0,0,0,0,0,0,0
3_a,0,0,0,0,0,0,0,0,0,0

When I tried an input file containing:
Code:
1_a> xxx . 11
Start              A BX C DYY E F            G            H           I                 J 
=================================================================================================================
Finish=1           1 2 3 4 5 6 (DISABLED) 7 (DISABLED) 8 (ENABLED) 9 (ENABLED) 10 (ENABLED)     
        
2_a> xxx . 11
Start              C DYY E F            G            H           I                 J
================================================================================================================= 
Finish=1           203 204 205 206 (DISABLED) 207 (DISABLED) 208 (ENABLED) 209 (ENABLED) 210 (ENABLED)

3_a> xxx . 11 
Start              A DYY E F            G            H           I                 J                    
=================================================================================================================
Finish=1           301 304 305 306 (DISABLED) 307 (DISABLED) 308 (ENABLED) 309 (ENABLED) 310 (ENABLED)

and used the following awk script:
Code:
awk '/>/ {   fn[++fnc] = substr($0, 1, index($0, ">") - 1)
        next       }
/Start/ {
        for(i = 2; i <= NF; i++) {
                if(!($i in hl)) {
                        h[++hc] = $i
                        hl[$i] = hc
                }
                ch[++chc] = $i
         }
        chc = 0
        next
}
/Finish/ {
        for(i = 2; i <= NF; i++) {
                if($i ~ /[^0-9]/) continue
                d[fnc, ch[++dhc]] = $i
        }
        dhc = 0
        next
}
END {   # Print headers...
        printf("x_a")
        for(i = 1; i <= hc; i++) printf(",%s", h[i])
        printf("\n")
        # Print data...
        for(i = 1; i <= fnc; i++) {
                printf("%s", fn[i])
                for(j = 1; j <= hc; j++)
                        printf(",%d", d[i, h[j]])
                printf("\n")
        }
}' input

it produces the output:
Code:
x_a,A,BX,C,DYY,E,F,G,H,I,J
1_a,1,2,3,4,5,6,7,8,9,10
2_a,0,0,203,204,205,206,207,208,209,210
3_a,301,0,0,304,305,306,307,308,309,310

This was tested using awk on OS X. As always, if you want to use this script on a Solaris/SunOS system, use /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or nawk instead of awk.

PS According to the standards, the delete command can be used to delete an array element, but the behavior is unspecified if you try to use delete to remove an entire array in a single delete operation. My script reuses the array ch[] without deleting elements, but for a given section it will only use the elements defined on the corresponding Start line, so any extra elements are just ignored. (Of course this is assuming that the number of elements on the Start and Finish lines are the same.)
These 2 Users Gave Thanks to Don Cragun For This Post:
agama radius
# 9  
Old 05-26-2013
Quote:
Originally Posted by Don Cragun
Hi agama,
When I tried your scripts, the headers came out fine, but all of the data fields were zero:
ERK!!

I made a change, and tested it, but failed to see all zeros. Going blind I think.

I've fixed the bug by reverting to my original logic and it works now for me. THANKS!

Code:
awk '
    />/ {
        split( $1, a, ">" );
        sect=a[1];
        sect_order[sidx++] = sect;
        next;
    }

    /Start/ {
        split( "", columns, "." );  # delete is not supported in all awk flavours; this deletes values in columns
        for( i = 2; i <= NF; i++ )
        {
            if( !header_seen[$(i)]++ )
                header_order[hidx++] = $(i);

            columns[i] = $(i);
            s = ",";
        }
        next;
    }

    /Finish/ {
        j = 0;
        for( i = 2; i <= NF; i++ )
        {
            if( substr( $(i), 1, 1 ) != "(" )
            {
                values[sect,columns[j]] = $(i);
                j++
            }
        }

    }

    END {
        printf( "x_a" );                            # print header record
        for( j = 0; j < hidx; j++ )
            printf( ",%s", header_order[j] );
        printf( "\n" );

        for( i = 0; i < sidx; i++ )                 # print each section
        {
            sect = sect_order[i];
            printf( "%s", sect );
            for( j = 0; j < hidx; j++ )
                printf( ",%d", values[sect,header_order[j]] );
            printf( "\n" );
        }
    }
' t.data

OR

Code:
awk '
    />/ {
        split( $1, a, ">" );
        sect=a[1];
        sect_order[sidx++] = sect;
        next;
    }

    /Start/ {
        split( "", columns, "." );  # delete is not supported in all awk flavours; this deletes values in columns
        for( i = 2; i <= NF; i++ )
        {
            if( length( $(i) ) > maxl )     # for alignment if that is needed
                maxl = length( $(i) );

            if( !header_seen[$(i)]++ )
                header_order[hidx++] = $(i);

            columns[i] = $(i);
            s = ",";
        }
        next;
    }

    /Finish/ {
        j = 0;
        for( i = 2; i <= NF; i++ )
        {
            if( substr( $(i), 1, 1 ) != "(" )
            {
                values[sect,columns[j]] = $(i);
                j++
            }
        }

    }

    END {
        fmt = sprintf( ",%%%dd", maxl );            # for column alignment
        fmth = sprintf( ",%%%ds", maxl );

        printf( "x_a" );                            # print header record
        for( j = 0; j < hidx; j++ )
            printf( fmth, header_order[j] );
        printf( "\n" );

        for( i = 0; i < sidx; i++ )                 # print each section
        {
            sect = sect_order[i];
            printf( "%s", sect );
            for( j = 0; j < hidx; j++ )
                printf( fmt, values[sect,header_order[j]] );
            printf( "\n" );
        }
    }
' t.data

This User Gave Thanks to agama For This Post:
radius
# 10  
Old 05-26-2013
when i run the script above for input
Code:
1_a,A,BX,C,DYY,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1 
2_a,C,DYY,E,F,G,H,I,J
2_a,3,3,3,0,1,0,1,0
3_a,A,DYY,E,F,G,H,I,J
3_a,3,3,3,0,1,0,1,0
4_a,J
4_a,1

the result
Code:
x_a

cmiiw
# 11  
Old 05-26-2013
Quote:
Originally Posted by radius
when i run the script above for input
Code:
1_a,A,BX,C,DYY,E,F,G,H,I,J
1_a,0,0,3,3,3,0,0,1,1,1 
2_a,C,DYY,E,F,G,H,I,J
2_a,3,3,3,0,1,0,1,0
3_a,A,DYY,E,F,G,H,I,J
3_a,3,3,3,0,1,0,1,0
4_a,J
4_a,1

The scripts posted earlier in the thread work on the original data as input, not what you are giving it. Did you try with the original data? I ran the original data through the two I posted and the one that Don posted and all yield what I expected to be the desired output.
# 12  
Old 05-27-2013
actually, this is my file input

Code:
ABC179W_Down,TM,startBaseCut40WMachineDown,startBaseCut60WMachineDown,licenseStateNum40WMachineDown,licenseStateNum60WMachineDown
ABC179W_Down,MachineBFunction=1,3,0,1,0
BCD187W_Lack_2_Time,TM,startBaseCut100WMachineDown,startBaseCut120WMachineDown,startBaseCut40WMachineDown,startBaseCut60WMachineDown,startBaseCut80WMachineDown,licenseStateNum100WMachineDown,licenseStateNum120WMachineDown,licenseStateNum40WMachineDown,licenseStateNum60WMachineDown,licenseStateNum80WMachineDown
BCD187W_Lack_2_Time,MachineBFunction=1,0,0,3,0,0,0,0,1,0,0
ADB650W_AXE_Aussie,TM,startBaseCut40WMachineDown,startBaseCut60WMachineDown,licenseStateNum40WMachineDown,licenseStateNum60WMachineDown
ADB650W_AXE_Aussie,MachineBFunction=1,0,0,0,1
GHS052W_Cambod_Rush,TM,startBaseCut100WMachineDown,startBaseCut120WMachineDown,startBaseCut40WMachineDown,startBaseCut60WMachineDown,startBaseCut80WMachineDown,licenseStateNum100WMachineDown,licenseStateNum120WMachineDown,licenseStateNum40WMachineDown,licenseStateNum60WMachineDown,licenseStateNum80WMachineDown
GHS052W_Cambod_Rush,MachineBFunction=1,0,0,3,0,0,0,0,1,0,0

sorry for simplifying my input file before, i thought it will be aligned with my data as long as the formation of input data is equal..so sorry

i expect to have

Code:
name,TM,startBaseCut100WMachineDown,startBaseCut120WMachineDown,startBaseCut40WMachineDown,startBaseCut60WMachineDown,startBaseCut80WMachineDown,licenseStateNum100WMachineDown,licenseStateNum120WMachineDown,licenseStateNum40WMachineDown,licenseStateNum60WMachineDown,licenseStateNum80WMachineDown
ABC179W_Down,MachineBFunction=1,0,0,3,0,0,0,0,1,0,0
BCD187W_Lack_2_Time,MachineBFunction=1,0,0,3,0,0,0,0,1,0,0
ADB650W_AXE_Aussie,MachineBFunction=1,0,0,0,0,0,0,0,0,1,0
GHS052W_Cambod_Rush,MachineBFunction=1,0,0,3,0,0,0,0,1,0,0

pls your help again sir
# 13  
Old 05-27-2013
This thread is continued in a thread titled Multi condition mapping
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk and grep using two files

I have two file one will have WWN and port details and another file will have allies name. I need to check what allies name was given to each wwn and create a consolidate report which will both port details, allies name. file one switchshow 133 1 21 188500 id N4 ... (4 Replies)
Discussion started by: ranjancom2000
4 Replies

2. Shell Programming and Scripting

Python - glob () - How to grep same files with different extension files

Hi I Have a directory and i have some files below abc.txt abc.gif gtee.txt ghod.pid umni.log unmi.tar How can use glob function to grep abc files , i have created a variable "text" and i assigned value as "abc", please suggest me how can we use glob.glob( ) to get the output as below... (2 Replies)
Discussion started by: kumar85shiv
2 Replies

3. Shell Programming and Scripting

grep 1000s of files with 1000s of grep values

Hi, I have around 200,000 files in a given directory. I need to cat each of these files and grep them for thousands of identifier values (or strings) in a given text file. The text file looks something like this: 1234 1243545 1234353 121324 etc with thousands of entries. Can... (3 Replies)
Discussion started by: mantis
3 Replies

4. Shell Programming and Scripting

grep all files in a directory

I am not sure if i am doing this correctly since it returns quickly. i need to grep for a keyword in all files in a directory grep keyword /mydirectory is that correct? I just want to know which files have a keyword in it. i am using korn shell in solaris 5.1. There does not appear to... (12 Replies)
Discussion started by: guessingo
12 Replies

5. Shell Programming and Scripting

grep for certain files using a file as input to grep and then move

Hi All, I need to grep few files which has words like the below in the file name , which i want to put it in a file and and grep for the files which contain these names and move it to a new directory , full file name -C20091210.1000-20091210.1100_SMGBSC3:1000... (2 Replies)
Discussion started by: anita07
2 Replies

6. Shell Programming and Scripting

Need to grep for .log files

Hi, Have two files. 1. robin.log 2. robin_log When is grep i shuld get only robin.log file. Please help. I Tried ls | grep ".log" But its not working Thanks Robin (6 Replies)
Discussion started by: robinbannis
6 Replies

7. Shell Programming and Scripting

Grep Different Files Using a Loop?

I have a script to GREP for a text expression within certain files, the files being named file.11012008 thru file.11302008. 30 files in all, one for each day of the month. Instead of entering the following 3 lines of code 30 different times, I'm trying to find a way to loop the process: ... (6 Replies)
Discussion started by: foleyml
6 Replies

8. Shell Programming and Scripting

how to grep word from .gz files & .z files

Hi I have find some account numbers from .gz files. There thousands of .gz files, also i am unable to grep .gz / .z files. I want to find file names & and those lines from list. Could you pls tell me / give me any syntax for finding word from ,gz files using grep? Thanks in advance (8 Replies)
Discussion started by: udaya_subbu
8 Replies

9. Shell Programming and Scripting

grep from .z files

There are archived files with .Z in the server. I want to grep a code within those files. How can I do this? Please help. Koho (4 Replies)
Discussion started by: koho
4 Replies

10. Shell Programming and Scripting

Script to grep several files

Hi I am totally new to Unix and I am in need of a script that can grep several files at once. the script needed: change directory grep for keywords "error" "fail" "warning" in different files with different names, same extention but specific file names. if any of the keywords is found to show... (4 Replies)
Discussion started by: sqloyd
4 Replies
Login or Register to Ask a Question