Assign zero to strings that don't appear in block, store result in AWK array


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Assign zero to strings that don't appear in block, store result in AWK array
# 8  
Old 10-24-2011
agama,

Many thanks, I'll try your new code right away and let you know asap.Smilie

I think if a group is missing for all groups blocks won't be a problem because it'll mean that only exist those groups that appear as unique Groups in the
file.

Best regards

---------- Post updated at 09:20 PM ---------- Previous update was at 08:21 PM ----------

Tested and works independently. Now I'll test it including it in my main awk script and following the structure of code as you suggested me. I'll let you know

Many thanks.

---------- Post updated at 11:00 PM ---------- Previous update was at 09:20 PM ----------

Hi again agama,

Is possible to process file1 first? how would be the structure?

I ask this because when awk code is processing file2 generates another array, but doing comparison with one of the arrays created when the code reads file1.

I've tried change the order as follow, but doesn't work (changes in red):
Code:
awk  '
    NR == FNR { 
    # ----------- blocks for processing file 1 ------------------------
    /^<x / {
        str = gensub(/(.+")([0-9]+)(">)(.+)(<\/.+)/, "\\2|\\4", "g")
        split( str, a, "|" );

        if( !seen[a[2]]++ )              # new group name, add it to the list
            list[++nlist] = a[2];

        agroup[group+0,a[2]] = a[1];   # changed to track across whole file

        # your original code
        B[gensub(/pattern/,"how","g")]      #Storing desired data in array B
        C[gensub(/pattern/,"how","g")]      # #Storing desired data in array C

        # small change to match D with A
        dgroup[group+0,a[2]] = gensub(/pattern/, "\\2|\\4", "g") # changed to track across whole file

        next;
    }

    /^<\/group>/ {
        group++;
        next;
    }

{
        # some other processing for file2
        if($0 == Arr1[d+1]) {Ln[d+1]=FNR;if(d<length(Arr1)-1){d++}} # Arr1 is created when processing file1
        next;
    }

    END {
        asort( list );
        for( g = 0; g < group; g++ )            # build A and D with groups seen
        {
            for( i = 1; i <= nlist; i++ )       
            {
                A[++aidx] = sprintf( "%d|%s", agroup[g,list[i]], list[i] );
                D[aidx] = dgroup[g,list[i]];
            }
        }

        # whatever end processing on A and D can be done here
        for( i = 1; i <= length( A ); i++ )         # my testing to ensure they align
            printf( "(%s) (%s)\n", A[i], D[i] );
    }
' file1 file2

Thanks again for your help

---------- Post updated 10-24-11 at 05:21 AM ---------- Previous update was 10-23-11 at 11:00 PM ----------

Hi agama again,

I've been able to adapt your code and suggestions into my main code. I saw that was much more complicated to
generate the array in the same awk code, then I generated an array and stored data in bash array. This bash array
is the input to main awk code.

At the beginnig I had some issues, but I was able to set the correct format of the array expected by the split() function.

The final code is as below:
Code:
oldIFS=$IFS # Default field separator in bash, IFS=" "
IFS=$'\n' # Changing temporaly to "|"
UnqGroups=( $( awk '/^<x /{print gensub(/(.+">)(.+)(<\/.+$)/,"\\2","g")}' file1 | sort -u | tr '\n' '|') )  #Unique groups
IFS=$oldIFS #Set it again to " ".

awk -v z="${UnqGroups[*]}" 'BEGIN {nlist=split(z,list,"|")-1}
    NR==FNR{    
        if($0 ~ /j v="/){
            B[gensub(/pattern/,"\\2","g")] 
            x[gensub(pattern/,"","g")];asorti(x,C) 
            
            # To generate Array A
            n=gensub(/.+="|".+$/,"","g")
            group[gensub(/pattern/,"\\2","g")]=n;
            next;
            
        }
        if($0 ~ /<\/group>/) {
        for( i = 1; i <= nlist; i++ )
            A[++w]=sprintf("%d|%s", group[list[i]], list[i] );
        delete group;   # clear for next go round
        next;} 
next}
{ 
    # code to work with file2
} 
END{ Print arrays info    }' file1 file2

Many thanks again both for all help and time.

Best regards
# 9  
Old 10-24-2011
Glad you were able to work it out!

Yes, it would have been possible to generate the array's before the processing of the second file.... You would have just needed to change END { to a function (something like function gen_array() {) and then invoke it once at the start of the second file:

Code:
FNR != NR {
    if( FNR == 1 )
          gen_array();   # only invoked on 1st record of each new file

    # rest of your code
}

Doubt you'll go back and make the changes (I wouldn't once I had something working), but figure I'd post it for some closure here.
# 10  
Old 10-25-2011
Hi again agama,

Even when I could adapt your code to the main awk code with success using a bash array, It would be great to know how would be the way you say using the function.

How would be the function, so I can test?

Regards.
# 11  
Old 10-25-2011
This is the way I'd do it:

Code:
awk  '
    function build(     g, i )
     {
        asort( list );
        for( g = 0; g < group; g++ )            # build A and D with groups seen
        {
            for( i = 1; i <= nlist; i++ )       # end of group, it is now safe to fill in D and A
            {
                A[++aidx] = sprintf( "%d|%s", agroup[g,list[i]], list[i] );
                D[aidx] = dgroup[g,list[i]];
            }
        }
    }

    NR != FNR {
        if( FNR == 1 )          # build the arrays when 1st record of second file is read
            build();

        # some other processing for file2
        next;
    }

    # ----------- blocks for processing file 1 ------------------------
    /^<x / {
        str = gensub(/(.+")([0-9]+)(">)(.+)(<\/.+)/, "\\2|\\4", "g")
        split( str, a, "|" );
        agroup[group+0,a[2]] = a[1];
        if( !seen[a[2]]++ )
            list[++nlist] = a[2];

        # your original code
        B[gensub(/pattern/,"how","g")]      #Storing desired data in array B
        C[gensub(/pattern/,"how","g")]      # #Storing desired data in array C

        # small change to match D with A
        dgroup[group+0,a[2]] = gensub(/pattern/, "\\2|\\4", "g") # #Storing desired data in array D

        next;
    }

    /^<\/group>/ {
        group++;
        next;
    }
' file1 file2

# 12  
Old 10-26-2011
agama, hello,

I must say, many thanks!!! it works just perfectSmilieSmilie.

It works faster than the other way because all is in awk and I dont need anymore at least 2 arrays that I was generating before.

I've learn many things with this thread, I'm some new to awk, but completely new with awk UDF functions. I have now better idea in how to use them.

At the end I'm not sure why if I use the structure of "NR != FNR" is not working for me, It worked great in this way:
Code:
awk  '
    function build(     g, i )
     {
        asort( list );
        for( g = 0; g < group; g++ )            # build A and D with groups seen
        {
            for( i = 1; i <= nlist; i++ )       # end of group, it is now safe to fill in D and A
            {
                A[++aidx] = sprintf( "%d|%s", agroup[g,list[i]], list[i] );
                D[aidx] = dgroup[g,list[i]];
            }
        }
    }

 NR == FNR 
 {  # ----------- blocks for processing file 1 ------------------------
   if($0 ~ /^<x /) {
        str = gensub(/(.+")([0-9]+)(">)(.+)(<\/.+)/, "\\2|\\4", "g")
        split( str, a, "|" );
        agroup[group+0,a[2]] = a[1];
        if( !seen[a[2]]++ )
            list[++nlist] = a[2];

        # your original code
        B[gensub(/pattern/,"how","g")]      #Storing desired data in array B

        next;
    }

   if( $0 ~ /^<\/group>/) {
        group++;
        next;
    }
}

  {
        if( FNR == 1 )          # build the arrays when 1st record of second file is read
            build();

        # some other processing for file2
        next;
    }
' file1 file2

Many thanks again for your great help, support and time.

Much appreciated.

Best regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Assign awk gsub result to a variable

Hello, I have searched but failed to find what exactly im looking for, I need to eliminate first "." in a output so i can use something like the following echo "./abc/20141127" | nawk '{gsub("^.","");print}' what i want is to use gsub result later on, how could i achieve it? Let say... (4 Replies)
Discussion started by: EAGL€
4 Replies

2. Shell Programming and Scripting

How to Assign an shell array to awk array?

Hello All, Can you please help me with the below. #!/bin/bash ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5 EXTRACT_DT:30-SEP-12 VER_NUM:1" ARR="No Differences In Stage Between HASH_TOTALS & HASH_TOTALS_COMP For UNINUM:0722075 PROVIDER:5... (14 Replies)
Discussion started by: Ariean
14 Replies

3. Shell Programming and Scripting

Store value in array with awk

Hi everybody I wanna store some values that r in a .txt file in some arrays for example I have: 32782 28 32783 02 32784 01 32785 29 32786 25 32787 25 32788 00 32789 25 32790 02 32791 29 32792 23 32793 01 32794 28 and I need to save the first... (4 Replies)
Discussion started by: Behrouzx77
4 Replies

4. Shell Programming and Scripting

create an array which can store the strings from the user input in shell script

I want to create an array which can store the strings from the user input in shell script . example :- I want to store the 5 fruits name in a single array which the user provides . (1 Reply)
Discussion started by: Pkast
1 Replies

5. Shell Programming and Scripting

Can -v option in awk be used to store an array of variables?

I want to pass an array of variables to be inserted by awk in the 2nd column of a file. Empl No. Employee Age 1000000 22 1100000 24 1200000 26 Now, I want to pass an array having three different ages which need to replace the... (7 Replies)
Discussion started by: Nishi_Licious
7 Replies

6. Shell Programming and Scripting

awk assign output of array to specific field-number

With this script i want to print the output to a specific field-number . Can anybody help? awk 'NR=FNR{split(FILENAME,fn,"_");nr=$2;f = $1} END{for (i=1;i<=f;i++) print i,$fn=nr}' input_5.csv input_6.csvinput_5.csv 4 135 5 185 6 85 11 30input_6.csv 1 90 3 58 4 135 7 60 8 55 10... (1 Reply)
Discussion started by: sdf
1 Replies

7. Shell Programming and Scripting

assign awk command result to a variable

#!/bin/sh # ## MYSTRING = `awk '/myApp.app/' /Users/$USER/Library/Preferences/loginwindow.plist` if then echo String not found defaults write /Users/$USER/Library/Preferences/loginwindow AutoLaunchedApplicationDictionary -dict-add -string Hide -bool YES -string Path -string... (9 Replies)
Discussion started by: dedmakar
9 Replies

8. Shell Programming and Scripting

Prase a file and store and result to an array

Dear all, I have a file having the following formats: ThreadFail=Web1=1234 ThreadFail=Web2=2345 ThreadFail=Web3=12 ConnectionFail=DB1=11 ConnectionFail=DB2=22 The number of lines will be different from every time . How can I parse the file and store the result to an a array inside... (6 Replies)
Discussion started by: youareapkman
6 Replies

9. Shell Programming and Scripting

assign awk array with printf

I am trying to assign a awk array for further processing later in the script. I can't seem to figure it out. If someone could look at this and help me, I would very much appreciate it. Thanks in Advance. for ( x = 1 ; x <= Var ; x++ ) { if ( x in varr ) { ... (2 Replies)
Discussion started by: timj123
2 Replies

10. Shell Programming and Scripting

How to store query multiple result in shell script variable(Array)

:) Suppose,I have one table A. Table A have one column. Table A have 10 rows. I want this 10 rows store into shell script variable. like #!/bin/ksh v_shell_var=Hi here in call oracle , through loop How can I store table A's 10 rows into v_shell_var (Shell Script Array). Regards, Div (4 Replies)
Discussion started by: div_Neev
4 Replies
Login or Register to Ask a Question