Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Filter a list of IPs

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filter a list of IPs
# 8  
Old 02-07-2011
Quote:
Originally Posted by Franklin52
Try this:
Code:
awk 'NR==FNR{a[$0]; next}!($0 in a)' list dumpfile

This one worked and boy is it faster than the GREP. Thanks Franklin52

yinyuemi, i'm testing yours out now.


CORRECTION:

This logic actually didn't work. i didn't get an error with it but it didn't filter or delete the 37 list of IPs from the main list
Last edited by senormarquez; 02-08-2011 at 12:01 AM.. Reason: correction
# 9  
Old 02-07-2011
How about this to load (and strip) all your IP*.txt files and then process dumpfile, it also ensures IPs contain valid (0-255) values.

Code:
awk '
BEGIN {
  p = "([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])"
  p = p "\\." p "\\." p "\\." p
}
$0 ~ p { gsub("[ \t]",""); a[$0] }
END {
  while (getline < "dumpfile")
     if (!($0 in a)) print $0;
}' IP*.txt

Last edited by Chubler_XL; 02-08-2011 at 09:14 PM..
# 10  
Old 02-08-2011
Quote:
Originally Posted by yinyuemi
a little change, hopelly it works
Code:
awk 'NR==FNR{A[$0]=1;next}A[$0]!=1' deletelist mainfile


This one gave me a different error
Code:
awk: Internal software error in the tostring function on .
 The input line number is 5.88657e+06. The file is ip_02072011.dat.
 The source line number is 1.

ip_02072011.dat=mainfile
37ips.txt=deletelist
# 11  
Old 02-08-2011
Is this on HPUX?
The HPUX Resource Centre Forum talks about a patch for a memory leek.

BTW how did my code go?
Last edited by Chubler_XL; 02-08-2011 at 09:21 PM.. Reason: typo in URL
# 12  
Old 02-09-2011
yeah, it is HPUX.

i couldn't get ehe awk codes to work so i went with the grep.
the last tme i tried i kept getting the error

awk: Internal software error in the tostring function on .
The input line number is 5.88657e+06. The file is ip_02072011.dat.
The source line number is 1.

thanks for the link. i will read up on the memory leak
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Filter and delete numbers in a list

Hello, I've got a list of a single column numbers like 3000.66 3002.55 3062.23 3242.12 etc... I would like to delete all numbers higher than for example 3060.00 and lower than 2990.00 How can I do that? Thanks in advance (2 Replies)
Discussion started by: Board27
2 Replies

2. AIX

Get List of users+IPs

Hello! We have an AIX box (6.3), we are looking for a way to get list of logged in users and their IP. The issue we have is running who -ub shows hostname and we are wanting to get the ip. Any suggestions of ideas? Thanks in advance for any help! (4 Replies)
Discussion started by: entropy1980
4 Replies

3. Shell Programming and Scripting

Help with shell script - filter txt file full of ips

Hello again gentlemen. I would like to make a shell script to 'optimize' a plain text full of IPs. Let's suppose to have this text file: 1.192.63.253-1.192.63.253 1.0.234.46/32 1.1.128.0/17 1.116.0.0/14 1.177.1.157-1.177.1.157 1.23.22.19 1.192.61.0-1.192.61.99 8.6.6.6 I want to... (2 Replies)
Discussion started by: accolito
2 Replies

4. Shell Programming and Scripting

Get Ips from a list file

Hi Everyone, I typed a command: awk '{ print $1}' $LOGFILE | sort | uniq -c | sort -nr > $DEST/a.txt And I got file a.txt which show 6 1.1.1.1 3 2.2.2.2 2 3.3.3.3 1 4.4.4.4 Just now, I want to get exact ips which has first column > 5 to a file b.txt. In this situation, the... (5 Replies)
Discussion started by: testcase
5 Replies

5. Shell Programming and Scripting

Copying a files from a filter list and creating their associated parent directories

Hello all, I'm trying to copy all files within a specified directory to another location based on a find filter of mtime -1 (Solaris OS). The issue that I'm having is that in the destination directory, I want to retain the source directory structure while copying over only the files that have... (4 Replies)
Discussion started by: hunter55
4 Replies

6. Shell Programming and Scripting

Filter only gz files from list of subdirectories

Hi, I have a very big directory structure that consists of many sub-directories inside.There are around 50 ".gz" files under this dir structure. I want to copy all the gz files alone to a seperate location. Plz help me. (2 Replies)
Discussion started by: villain41
2 Replies

7. Shell Programming and Scripting

List of IPs & database lookup

I am trying to feed a list of IP's to do lookups from a database. My script works only for the first IP but all subsequent IPs output as 'unknown'. #!/usr/bin/php -q <? $ip = file('ip.txt'); foreach ($ip as $ip_num => $ip) { echo $ip; $out=sprintf("%u", ip2long($ip)); ... (1 Reply)
Discussion started by: hazno
1 Replies

8. Shell Programming and Scripting

Filter the file list greater then the size specified by user

HI, Can any tell me how to filter the list of files greater than the size specified by user. The size should be provided by user as an input. Regards shiva (6 Replies)
Discussion started by: shivu
6 Replies

9. AIX

Get the list, filter and delete the files on remote server

Hi, I need to login to a remote server. Go to a particular path. Get the lists of files on that path.There may be n number of files. I need to delete only those files from above created list which are 7 days older. I have achieved above using ftp protocol, but now the constraint has... (0 Replies)
Discussion started by: mail_amitnagpal
0 Replies
Login or Register to Ask a Question