Awk - query about arrays


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Awk - query about arrays
# 1  
Old 04-21-2012
Awk - query about arrays

Hello again,

I have example config file with two arrays:
Code:
tab1[0]="name1 surname1"
tab1[1]="name2 surname2"
tab1[2]="name3 surname3"

tab2[0]="First"
tab2[1]="Second"

and csv file:
Code:
"aaaaa","surname1","name1","ddddd,eeeee","ffffff","ggggg","3","2012/02/22 12:25:21","2012/02/22 00:00:00","8","hhhh","First"
"aaaaa","bbbbbb","ccccc","ddddd,eeeee","ffffff","ggggg","3","2012/02/22 12:25:21","2012/02/22 00:00:00","8","hhhh","Second"
"aaaaa","surname2","name2","ddddd,eeeee","ffffff","ggggg","3","2012/02/22 12:25:21","2012/02/22 00:00:00","8","hhhh","iiii"

is it possible to pass those arrays to awk and print (redirect to another file) only those records where
1. fields 2 and 3 match to any element from tab1
AND
2. field 12 match to any element from tab2
?

Now I am planning my script but so far my only idea is to run two loops (pseudo-code):
Code:
> temp.file
for i in tab1.length; do
awk '$2 $3 == tab1[$i]' file >> temp.file
done
for i in tab2.length; do
awk '$12 == tab2[$i]' file >> temp.file
done

I will be gratful for any advise/ideas or links which can help me.

Kind regards,
# 2  
Old 04-21-2012
Is that the way your conf file has to be, or can you reorganize it as you please? That's not how I'd have organized it, it takes a bit more processing that way... Just simple files like
Code:
name1 surname1
name2 surname2
name3 surname3

would be much simpler for awk to handle.

But as is, if you slap a BEGIN { ... } around the data you have, awk can actually run it. Then you just need a few loops to split and reorganize it into something you can easily look up.

Code:
$ cat awkarr.sh

#!/bin/sh

# Create a file full of awk code to run
cat <<EOF > /tmp/$$
BEGIN {
        # Slurp the contents of your config file into this bit of awk
        `cat conffile`

        # Get arrays into form A["name"]=1 for easy lookup later
        for(X in tab1)
        {
                split(tab1[X], A);
                for(X in A)     T1[A[X]]++;
        }

        # Add a " since the extra quotes on the first and last record
        # aren't all eaten by FS
        for(X in tab2)  T2[tab2[X]"\""]++

        FS="\",\""
}

# Dollar only needs escaping because its in a here-document
T1[\$2] && T1[\$3] && T2[\$12]

EOF

awk -f /tmp/$$ data

rm -f /tmp/$$

$ ./awkarr.sh

"aaaaa","surname1","name1","ddddd,eeeee","ffffff","ggggg","3","2012/02/22 12:25:21","2012/02/22 00:00:00","8","hhhh","First"

$


Last edited by Corona688; 04-21-2012 at 06:05 PM..
This User Gave Thanks to Corona688 For This Post:
# 3  
Old 04-21-2012
Thank you for quick reply.

Config file is flexible - I am creating it in php.

Could you please tell me how (in your version of config file) I can recognize which fields belong to tab1 and which are for tab2?

Meantime I will analyze your code. Thank you for support Smilie

Kind regards,
Paweł
# 4  
Old 04-21-2012
I took a different approach that assumed the config file couldn't be changed.

Code:
awk -F "," '
    NR != FNR {
        gsub( "\"", "", $0 );
        x = $3 " " $2;
        if( x in tab1 && $13 in tab2 )
            print;
        next;
    }

    /^tab1/ {
        gsub( "\"", "", $0 );
        split( $0, a, "=" );
        tab1[a[2]] = 1;
        next;
    }

    /^tab2/ {
        gsub( "\"", "", $0 );
        split( $0, a, "=" );
        tab2[a[2]] = 1;
        next;
    }

'  config-file-name  data-file-name



Reads the config file and builds a map for tab1 and tab2, then reads the data file. I've made the BIG assumption that there aren't any embedded commas in the data. I also note that by your description, the field with "First" is field 13 not field 12.
This User Gave Thanks to agama For This Post:
# 5  
Old 04-21-2012
Quote:
Originally Posted by agama
I took a different approach that assumed the config file couldn't be changed.
My approach uses his original config file, too.
# 6  
Old 04-21-2012
@Corona688: your solutions works perfectly for me (with small modifications). Many thanks for help
@agama: Thank you for your code.

It's a pleasure to learn from you guys. Thanks ! Smilie

Kind regards,
Paweł
# 7  
Old 04-21-2012
Quote:
Originally Posted by Corona688
My approach uses his original config file, too.
Oops, had I looked closely I would have seen that. My brain fixated on your question about being able to change it.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Arrays in awk

Array A contains lines of numbers from files. Array B contains number of fields for each line of each file. I want to run on array A. To do that I need to know the number of fields for each file in array A (because each line doesn't have the same NF). The NF is in array B, problem is: I don't... (6 Replies)
Discussion started by: guitarist684
6 Replies

2. Shell Programming and Scripting

awk Arrays

So I'm back once again beating my head off a wall trying to figure out how to get this to work. My end goal is to take input such as what's below, which will be capture in real time with a tail -f from a file or piped output from another command: ... (5 Replies)
Discussion started by: ShadowBlade72
5 Replies

3. Shell Programming and Scripting

help in awk arrays!

Hi, buddies I am new to shell scripting and trying to solve a problem. I read about arrays in awk that they are quite powerful and are associative in nature. Awk Gurus Please help! I have a file: Id1 pp1 0t4 pp8 xy2 Id43 009y black Id6 red xy2 Id12 new pp1 black I have... (5 Replies)
Discussion started by: geli21
5 Replies

4. UNIX for Dummies Questions & Answers

awk arrays

Hi Can someone please explain the logic of awk arrays. I have been doing some reading but I dont understand this: #!/usr/bin/gawk -f { arr++; } end { for(i in arr) { print arr,i } } As I understand arr refs the arrays index, so while $2 is a string that cant... (2 Replies)
Discussion started by: chronics
2 Replies

5. UNIX for Dummies Questions & Answers

Help to know the various possible uses of awk arrays

Hi to all, I write this time to ask for different syntax or ways of arrays within awk and uses. Since I don't know how actually could work and the real potencial of awk arrays, I look for a proactive reply/help in order to get more information than it seems I'm trying to get. I think I... (2 Replies)
Discussion started by: cgkmal
2 Replies

6. Shell Programming and Scripting

awk arrays can do this better - but how?

Hi, I have spent the afternoon trawling Google, Unix.com and Unix in a Nutshell for information on how awk arrays work, and I'm not really getting too far. I ahve a batch of code that I am pretty sure can be better managed using awk, but I'm not sure how to use awk arrays to do what I'm... (1 Reply)
Discussion started by: littleIdiot
1 Replies

7. Shell Programming and Scripting

Perl and Net::LDAP, objects and arrays query

Hi I'm not a programmer but am muddling through as best I can. I am trying to set up a PostSearchHook for Radiator (RADIUS server), that carries out an LDAP lookup, and, based on the string returned ("staff" or "student") in the "businessCategory" attribute, will set the $role to be either 40... (3 Replies)
Discussion started by: mikie
3 Replies

8. Shell Programming and Scripting

Arrays in awk

Hi, I've written the following code to manipulate the first 40 lines of a data file into my desired order: #!/bin/awk -f { if (NR<=(4)){ a=a$0" "} else { if ((NR >= (5)) && (NR <= (13))) { b=b$0" " } else {if ((NR >= (14)) && (NR <= (25))){ c=c$0" "} ... (5 Replies)
Discussion started by: catwoman
5 Replies

9. Shell Programming and Scripting

awk arrays

Guys, OK so i have been trying figure this all all day, i guess its a pretty easy way to do it. Right, so i have to column of data which i have gotten from one huge piece of data. What i would like to do is to put both of these into one array using awk. Is this possible?? If so could... (1 Reply)
Discussion started by: imonthejazz
1 Replies

10. UNIX for Advanced & Expert Users

Two or more arrays in Awk

Hi All, I have been working on awk and arrays. I have this small script: cat maillog*|awk -F: '$2=="SMTP-Accept" && $5~/string/ {lastdate=substr($1,1,8); internaluser=$5; v++} END {for (j in v) {print lastdate, v, j}'| sort>> mail.list This gives me the number of mails users are getting. ... (1 Reply)
Discussion started by: nitin
1 Replies
Login or Register to Ask a Question