Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Difficulty searching for IP address in a file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Difficulty searching for IP address in a file
# 1  
Old 05-07-2018
Difficulty searching for IP address in a file
Hi,

Below is the command I use to search for IP address in a file.


Code:
find ./ -type f \( -name "*.txt" -or -name "*.xml" \) | xargs grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"

As an output I found three IP address and below is how they look.

Code:
#Listen 112.34.56.58:80
Listen 112.134.56.58:80 #This is an IP
Listen 112.34.55.58:80

The problem is I do not want any IP in output which begin with "#" i.e

Code:
# Listen 12.34.56.78:80

Because, in my case anything starting with a # usually means comments and should be ignored i.e not listed.
Can you please tell me how should I tweak my command not to display IPs mentioned in lines that start with "#" character?

Please note: Any "#" after the IP should be considered and grep like below:
Code:
Listen 112.134.56.58:80 #This is an IP


Thank you.
Last edited by Don Cragun; 05-11-2018 at 02:08 PM.. Reason: Get rid of all of the FONT, COLOR, & QUOTE tags; add missing CODE tags.
# 2  
Old 05-07-2018
Maybe using an additional step to get just the IP address. Something like:
Code:
find ./ -type f \( -name "*.txt" -or -name "*.xml" \) | xargs grep -oE "^ *[^#].*\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"

or
Code:
find ./ -type f \( -name "*.txt" -or -name "*.xml" \) | xargs grep -oE "^ *[^#].*\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" | awk '{print $NF}'

Last edited by rdrtx1; 05-07-2018 at 07:28 PM..
# 3  
Old 05-07-2018
If your grep has the -P option you could use a lookahead to discard commented IPs.

Also here I added some more validation to discard illegal IP addresses (eg 192.999.1.1):

Code:
... | xargs grep  -oP '[^#]*\b\K(((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)(\.|\b)){4})'

# 4  
Old 05-08-2018
grep -o breaks the rule "grep returns whole lines".
So combining it with other options like -v produces experimental results.
Rather than trying the experimental -P that might differ between grep versions, I would go to real perl, like
Code:
perl -lne '/^[^#]*\b((\d{1,3}[.]){3}\d{1,3})\b/ and print $1'

or
Code:
perl -lne 's/#.*//; /\b((\d{1,3}[.]){3}\d{1,3})\b/ and print $1'

The latter leads to sed, indeed you can use sed to pre-filter the lines and then pipe to grep
Code:
sed 's/#.*//' | grep -Eo '\b([0-9]{1,3}[.]){3}[0-9]{1,3}\b'

# 5  
Old 05-11-2018
Hammer & Screwdriver
I tried all the suggestions but none of the helped.

A file has the below entries

Code:
#Listen 112.34.56.58:80
Listen 112.134.56.58:80 #This is an IP
Listen 112.34.55.58:80

Expectation is that the search should yield 2 / 3 IPs mentioned in the file. Yes, the entry which starts with the hash(#) i.e. #Listen 112.34.56.58:80 should not be display.

However, all the three entries are displayed which are listed in the file. One can create a file with these entries and test to see the problem themselves.

Can you please suggest please?
Last edited by Don Cragun; 05-11-2018 at 02:09 PM.. Reason: Change QUOTE tags to CODE tags.
# 6  
Old 05-11-2018
For me it works
Code:
cat file
#Listen 112.34.56.58:80
Listen 112.134.56.58:80 #This is an IP
Listen 112.34.55.58:80 
perl -lne 's/#.*//; /\b((\d{1,3}[.]){3}\d{1,3})\b/ and print $1' file
112.134.56.58
112.34.55.58
perl -lne '/^[^#]*\b((\d{1,3}[.]){3}\d{1,3})\b/ and print $1' file
112.134.56.58
112.34.55.58
sed 's/#.*//' file | grep -Eo '\b([0-9]{1,3}[.]){3}[0-9]{1,3}\b'
112.134.56.58
112.34.55.58

Could you please copy/paste your command/result?
# 7  
Old 05-12-2018
Hammer & Screwdriver
I m trying the below command but I do not see any output even when I expect it to.
Code:
sed 's/#.*//' | find ./ -type f \( -name "*.txt" -or -name "*.xml" \) | grep -Eo '\b([0-9]{1,3}[.]){3}[0-9]{1,3}\b'

I do not wish to use perl.
Here is my OS details: Linux mycomp 3.10.0-693.17.1.el7.x86_64 #1 SMP Sun Jan 14 10:36:03 EST 2018 x86_64 x86_64 x86_64 GNU/Linux

I was anyways able to get this to work by rewriting the command as a script.

Code:
 for i in $( find ./ -type f \( -name "*.txt" -or -name "*.xml" \) );
 do
 sed 's/#.*//' "$i" | grep -Eo '\b([0-9]{1,3}[.]){3}[0-9]{1,3}\b'
 if [ $? -eq 0 ]; then
 echo "File:$i"
 fi
 done

Last edited by mohtashims; 05-12-2018 at 02:17 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Difficulty with set - command

Hi , i have a shell script with the first two lines as #! /bin/ksh set -x when i try opening the file from unix using the command 'sh filename.sh' , i am getting the below error : invalid optionline 2: set: - Pls help Use code tags for you code and data... (1 Reply)
Discussion started by: Rajankum
1 Replies

2. Shell Programming and Scripting

difficulty with awk

hello folks, i am stuck with this awk command. i need to calculate the sum of a column of values on a flatfile and i am using the following command : awk -F"|" '{x += $10} END {print "Sum: "x}' standard_csv_file1.out that flatfile contains 180 fields and i am getting the... (5 Replies)
Discussion started by: jdsony
5 Replies

3. Shell Programming and Scripting

difficulty in formatting a file.

a file containing following data (part of it).... 1907594 201012 31 11 5837737 201012 41 18 257402.88 201101 31 11 7500 201101 33 1 115618.5 201101 41 11 556330 201102 31 12 481783.5 201102 41 20 2827732.13 201103 31 71 85253 201103 33 2 4479588.07 201103 41 90 7120 201104 21 1 ... (4 Replies)
Discussion started by: guptam
4 Replies

4. Shell Programming and Scripting

searching a file with a specified text without using conventional file searching commands

without using conventional file searching commands like find etc, is it possible to locate a file if i just know that the file that i'm searching for contains a particular text like "Hello world" or something? (5 Replies)
Discussion started by: arindamlive
5 Replies

5. Shell Programming and Scripting

Loop difficulty

hi all I am new to unix and want to create a loop to repeat the loop and before that it ask me to do so.I know "while" may help but I put it in my work and getting stuk with it.any help appreciated. (13 Replies)
Discussion started by: samsami1971
13 Replies

6. Emergency UNIX and Linux Support

TCL scripting - searching for a IP address in a string

Hi All, Can anyone please help me with the regular expression/code snippet to search for an IP address in a string with Tcl scripting. Example string "OSPF_NBRUP OSPF neighbor 16.138.181.15 (realm ospf-v2 e1-0/0/0:37.0 area 0.0.0.0) state changed from Full to Down due to KillNbr" In the... (6 Replies)
Discussion started by: Mr. Zer0
6 Replies

7. Shell Programming and Scripting

Perl Difficulty

Hi, I am trying to upload a file to a SQL database table. The column type is IMAGE. I am looking for a solution to upload a word doc file. I tried 3 approaches. 1) my $fileToStore = "mytest.doc"; open IPFILE, "<", $name; binmode IPFILE; while (<IPFILE>) { $fileToStore .= $_; } close... (1 Reply)
Discussion started by: b.paramanatti
1 Replies

8. UNIX for Dummies Questions & Answers

Reverse Proxy difficulty

Hi I am trying to set up two hosts in a reverse proxy. The reverse proxy already has 8 servers running perfectly, but they are all simply mapping pure addresses, which I have registered internally and externally. The latest two I wish to add are a bit different, they are app servers, one... (1 Reply)
Discussion started by: rboekdrukker
1 Replies

9. UNIX for Dummies Questions & Answers

Having difficulty with UNIX concept. Please help!

Hi, I would be very happy if someone could help me please. I am relatively new to UNIX, and still learning. My understanding of things are: Say I have a PC running Windows. This machine has a name. If I have 10 PC's, then I have 10 names, one for each PC. Each PC is independent of the other.... (4 Replies)
Discussion started by: ALon
4 Replies
Login or Register to Ask a Question