nawk array help


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers nawk array help
# 1  
Old 04-16-2008
nawk array help

This is a derivative of a question I asked earlier, but I can't figure out how to make all of the pieces work (mainly the array piece).

How do I get the following data set:
aaa|bbb|ccc|123|ddd...
abc|def|ghi|123|jkm...
aaa|bbb|ccc|456|ddd...
abc|def|ghi|456|jkm...
aaa|bbb|ccc|789|ddd...
abc|def|ghi|789|jkm...

to look like:
aaa|bbb|ccc|123|ddd...
abc|def|ghi|123|jkm...

Bascially I need to dedupe the file and I don't care which of the multiples I end up with. Also since the file layout can be multiple lenghts I was thinking about using $0 to just cover my bases.

My name says it all. Help!
# 2  
Old 04-16-2008
Oh, I almost forgot, the only field that is used in matching is the first field - the "aaa" or "abc".
# 3  
Old 04-16-2008
Try...
Code:
awk -F '|' '!a[$1]++' file1

# 4  
Old 04-16-2008
Quote:
Originally Posted by Ygor
Try...
Code:
awk -F '|' '!a[$1]++' file1

It works great! Any chance you could explain what you did?

Thanks.
# 5  
Old 04-16-2008
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash arrays: rebin/interpolate smaller array to large array

hello, i need a bit of help on how to do this effectively in bash without a lot of extra looping or massive switch/case i have a long array of M elements and a short array of N elements, so M > N always. M is not a multiple of N. for case 1, I want to stretch N to fit M arrayHuge H = (... (2 Replies)
Discussion started by: f77hack
2 Replies

2. Shell Programming and Scripting

Nawk Problem - nawk out of space in tostring on

Hi.. i am running nawk scripts on solaris system to get records of file1 not in file2 and find duplicate records in a while with the following scripts -compare nawk 'NR==FNR{a++;next;} !a {print"line"FNR $0}' file1 file2duplicate - nawk '{a++}END{for(i in a){if(a-1)print i,a}}' file1in the middle... (12 Replies)
Discussion started by: Abhiraj Singh
12 Replies

3. Shell Programming and Scripting

Bash 3.2 - Array / Regex - IF 3rd member in array ends in 5 digits then do somthing...

Trying to do some control flow parsing based on the index postion of an array member. Here is the pseudo code I am trying to write in (preferably in pure bash) where possible. I am thinking regex with do the trick, but need a little help. pesudo code if == ENDSINFIVEINTS ]]; then do... (4 Replies)
Discussion started by: briandanielz
4 Replies

4. Shell Programming and Scripting

PERL : Read an array and write to another array with intial string pattern checks

I have an array and two variables as below, I need to check if $datevar is present in $filename. If so, i need to replace $filename with the values in the array. I need the output inside an ARRAY How can this be done. Any help will be appreciated. Thanks in advance. (2 Replies)
Discussion started by: irudayaraj
2 Replies

5. Shell Programming and Scripting

Nesting - two nawk into one nawk

hi people; this is my two awk code: nawk '/cell+-/{r=(NF==8) ? $4FS$5FS$6 : NF==7 ? $4FS$5 : $4 ;c=split(r,rr);for (i=1;i<=c;i++){if(rr != "111111"){printf($3" %d ""\n",(i+3))}}printf("")}' /home/gc_sw/str.txt > /home/gc_sw/predwn.txt nawk -F'*' '{gsub(/ *$/,"")}$0=$1$($NF-2)'... (2 Replies)
Discussion started by: gc_sw
2 Replies

6. Shell Programming and Scripting

Multiple array problem in NAWK

I HAD these 2 files: file1 pictures.txt 5 ref2313 4 ref2345 3 ref5432 2 ref4244 1 dance.txt 6 ref2342 5 ref2352 4 ref0695 3 ref5738 2 ref4948 1 treehouse.txt 6 ref8573 5 ref3284 4 ref5838 3 ref4738 2 ref4573 1 file2 pictures.txt 1 3 dance.txt 2 4 treehouse.txt 3 5 what I... (1 Reply)
Discussion started by: linuxkid
1 Replies

7. Shell Programming and Scripting

NAWK array to store lines from huge file

Hi, I would like to clarify about the NAWK array to store multiple lines from huge file. The file is having an unique REF.NO, I wants to store the lines (it may be 100+ lines) till I found the new REF.NO. How can I apply NAWK - arrays for the above? Rgds, sharif. (1 Reply)
Discussion started by: sharif
1 Replies

8. Shell Programming and Scripting

PHP: Search Multi-Dimensional(nested) array and export values of currenly worked on array.

Hi All, I'm writing a nagios check that will see if our ldap servers are in sync... I got the status data into a nested array, I would like to search key of each array and if "OK" is NOT present, echo other key=>values in the current array to a variable so...eg...let take the single array... (1 Reply)
Discussion started by: zeekblack
1 Replies

9. Programming

Creating an array to hold posix thread ids: Only dynamic array works

I am facing a strange error while creating posix threads: Given below are two snippets of code, the first one works whereas the second one gives a garbage value in the output. Snippet 1 This works: -------------- int *threadids; threadids = (int *) malloc (num_threads * sizeof(int)); ... (4 Replies)
Discussion started by: kmehta
4 Replies

10. Shell Programming and Scripting

how to access values of awk/nawk variables outside the awk/nawk block?

i'm new to shell scripting and have a problem please help me in the script i have a nawk block which has a variable count nawk{ . . . count=count+1 print count } now i want to access the value of the count variable outside the awk block,like.. s=`expr count / m` (m is... (5 Replies)
Discussion started by: saniya
5 Replies
Login or Register to Ask a Question