Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Modify log files to get uniq data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Modify log files to get uniq data
# 8  
Old 08-25-2009
$$$$$
Last edited by asirohi; 09-08-2009 at 02:57 PM..
# 9  
Old 08-25-2009
Code:
nawk '
  { a[$1]=($1 in a)?a[$1] OFS $NF:$0}
  END {
    for(i in a)
      if(split(a[i],t, OFS)>2) print a[i]
  }' myFile

# 10  
Old 08-25-2009
$$$$$$
Last edited by asirohi; 09-08-2009 at 02:57 PM..
# 11  
Old 08-25-2009
Quote:
Originally Posted by asirohi
Can you please explain this what is happening in the above line of code?
Code:
nawk '
  { a[$1]=($1 in a)?a[$1] OFS $NF:$0}
  END {
    for(i in a)
      if(split(a[i],t, OFS)>2) print a[i]
  }' myFile

Thanks
Adsi
Do you how 'associative arrays' work in awk?
If not, try reading up starting with 'man nawk' first.
# 12  
Old 08-25-2009
Hello,

I can understant what associative array is. But if you could tell me what is happening here below, it would be great.

Code:
a[$1]=($1 in a)?a[$1] OFS $NF:$0}

Last edited by vgersh99; 08-26-2009 at 12:19 PM.. Reason: code tags, PLEASE!
# 13  
Old 08-26-2009
Quote:
Originally Posted by asirohi
Hello,

I can understant what associative array is. But if you could tell me what is happening here below, it would be great.

Code:
{a[$1]=($1 in a)?a[$1] OFS $NF:$0}

one can re-write the above as:
Code:
if ($1 in a)
   a[$1]=a[$1] OFS $NF
else
   a[$1]=$0

Is this clearER?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help in awk: running a loop with one column and segregate data 4 each uniq value in that field

Hi All, I have a file like this(having 2 column). Column 1: like a,b,c.... Column 2: having numbers. I want to segregate those numbers based on column 1. Example: file. a 5 b 9 b 620 a 710 b 230 a 330 b 1910 (4 Replies)
Discussion started by: Raza Ali
4 Replies

2. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies

3. UNIX for Dummies Questions & Answers

Modify Column Data using Shell Script

HI Guys, Input :- P081 wr1 12p0d5: 22.8 P081 wr1 12p2d18: 23.1 P149 wr1 1pxcud6/port_0_dev_7: 20.4 P149 wr1 1pxcud4/port_1_dev_10: 22.4 OutputP081 wr1 120 22.8 P081 wr1 122 23.1 P149 wr1 10 20.4 P149 wr1 11 22.4 In in First two line delete p and after d untill : In Last two line... (4 Replies)
Discussion started by: pareshkp
4 Replies

4. Shell Programming and Scripting

Filtering data using uniq and sed

Hello, Does anyone know an easy way to filter this type of file? I want to get everything that has score (column 2) 100.00 and get rid of duplicates (for example gi|332198263|gb|EGK18963.1| below), so I guess uniq can be used for this? gi|3379182634|gb|EGK18561.1| 100.00... (6 Replies)
Discussion started by: narachaid
6 Replies

5. Shell Programming and Scripting

How can I modify my script to include or substitute missing data?

Let me start off by saying I am a self taught sometimes scripter so what you will see below won't be pretty. I have created a script to parse through a file with a large amount of data and simply pull out what I need. In doing this I create several files and then paste them together in order to... (2 Replies)
Discussion started by: fsanchez
2 Replies

6. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt (6 Replies)
Discussion started by: Fahmida
6 Replies

7. Shell Programming and Scripting

Dynamically modify data in file?

Hiya fellas, I have a data file which contains these fields... EMPID EMPNAME GENDER DESIGNATION AGE SALARY this datafile looks like this.. Now, i'm making a script which will allow me to edit anything on execution. ie. First it will ask which EMPID is to be modified. and then it... (3 Replies)
Discussion started by: rushyang
3 Replies

8. Shell Programming and Scripting

Using uniq on log files

I have this log file which I need to count the number of repeated line and do some manipulation. test.log: June 3 03:33:38 test 1 June 3 10:31:22 test 2 June 3 10:32:22 test 2 June 3 10:33:22 test 3 June 3 10:33:22 test 3 June 3 10:34:22 test 4 June 3 10:35:22 test 5 ... (4 Replies)
Discussion started by: jazzaddict
4 Replies

9. Shell Programming and Scripting

Help needed with Sort and uniq data

Hi All, After Sorting directories and files i have got following output as below, now i only want the strings common in them, so the actual output should be as below in the bottom. How do i do that? Thanks -adsi File to be modified:- Common Components for ----> AA... (4 Replies)
Discussion started by: asirohi
4 Replies

10. Shell Programming and Scripting

How to replicate data using Uniq or awk

Hi, I have this scenario; where there are two classes:- apple and orange. 1,2,3,4,5,6,apple 1,1,0,4,2,3,apple 1,3,3,3,3,4,apple 1,1,1,1,1,1,orange 1,2,3,1,1,1,orange Basically for apple, i have 3 entries in the file, and for orange, I have 2 entries. Im trying to edit the file and find... (5 Replies)
Discussion started by: ahjiefreak
5 Replies
Login or Register to Ask a Question