Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Tips/advise on alternative to doing egrep -v

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Tips/advise on alternative to doing egrep -v
# 1  
Old 12-19-2019
Tips/advise on alternative to doing egrep -v
Hi all,

At the moment, I am doing the following to exclude some exception strings. The more I need to exclude, the longer the string becomes and it has become error prone as I edit the list manually.

Code:
$ cat output.txt
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.101 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.105 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.98 user=mickey
host=192.168.1.111 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.102 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.104 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.9 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.103 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.107 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.123 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.108 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.109 user=mickey
host=192.168.1.99 user=mickey

$ egrep "192.168.1.101|192.168.1.102|192.168.1.103|192.168.1.123" output.txt
host=192.168.1.101 user=mickey
host=192.168.1.102 user=mickey
host=192.168.1.103 user=mickey
host=192.168.1.123 user=mickey

$ egrep -v "192.168.1.101|192.168.1.102|192.168.1.103|192.168.1.123" output.txt
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.105 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.98 user=mickey
host=192.168.1.111 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.104 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.9 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.107 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.108 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.99 user=mickey
host=192.168.1.109 user=mickey
host=192.168.1.99 user=mickey

I presume I can do something like assign several variables and then concatenate them together and will be trying this out later, so something like below:


Code:
$ 

set1="192.168.1.101|192.168.1.102"
set2="192.168.1.111|192.168.1.112"
all_set="${set1}|${set2}"
egrep -v "${all_set}" output.txt

I am sure someone can suggest a better and more efficient way of doing this. I am hoping to be able to use an exception file and use that as an exclusion list when parsing output.txt but can't find an example of how to do it like that. So if I need to exclude more search string, then I just edit that exception file. And that exception file can contain other things to exclude too which is a more efficient way of doing a search <file> but exclude <strings>.

Please advise of tips and examples that I can try.
Last edited by newbie_01; 12-19-2019 at 06:59 PM.. Reason: More info
# 2  
Old 12-20-2019
If your exclude.txt contains content like :
Code:
192.168.1.101
192.168.1.102
192.168.1.103
192.168.1.123

And you wish to exclude from your output.txt based on IP address in exclude.txt, this could be a start.
Code:
awk 'NR==FNR { a[$0] } { wo=$0; gsub("[a-z,=]","",$1); if ( !( $1 in a ) ) print wo } ' exclude.txt input.txt

What other things you wish to exclude except IP address ?
Above is just a simple example.

Hope that helps
Regards
Peasant.
# 3  
Old 12-20-2019
I think your current concern is about overlong lines.
The | divider is an egrep thing.
In grep and fgrep you can have a newline.
Code:
fgrep -v "192.168.1.101
192.168.1.102
192.168.1.103
192.168.1.123" output.txt

But you should also be concerned about exactness.
The fgrep takes a dot as is, while in grep and egrep a dot means "any character". So fgrep is more exact here.
Still each search item can be a part of the whole, for example
fgrep "10.168.1.13" can find "10.168.1.13" and "110.168.1.13" and "10.168.1.136".
# 4  
Old 12-20-2019
You should be aware of several things that can catch you:-
  • Using egrep is the same as grep -E so the string passed is an Extended Regular Expression. Along with | as an 'or' separator, it also means that the . is a wildcard for a single character. Searching for 192.168 will also match 192g168
  • You can group expressions or characters using [expression] so you can consolidate your search/exclude.

I'm not clear what the overall requirement for this is, but I think you are looking for sessions for user mickey that are/aren't from a specific set up IP addresses. Might I suggest:
Code:
egrep -v ^host=192\.168\.1\.10[123]|192\.168\.1\.123

If this is all there is, then it may be to better blend these together like 192\.168\.(10[123]|123)




I hope that this helps,
Robin
# 5  
Old 12-20-2019
Hi
Code:
grep -vFf exlude.txt original.txt

option -F allows you to not escape special characters such as dot in your exclude file
Last edited by nezabudka; 12-20-2019 at 10:01 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Don't have tree, need advise to differentiate dir from file from this alternative that uses find

Hi, I don't have tree on the Solaris server and our SA don't want to install it. I found this example from One Line Linux Command to Print Out Directory Tree Listing | systemBash that more or less does what I am mainly looking for. Example run is as below: $: find ./ | sed -e... (2 Replies)
Discussion started by: newbie_01
2 Replies

2. Solaris

Experts !!! Please advise

Hi, I work on sun Solaris. Am hosting few web services on my server which are accessed over the internet. Now to check whether the web service is responding or not, i first have to log in to the web service URL. If it doesn't respond there, i come back to my server box and restart the service... (4 Replies)
Discussion started by: sting672744
4 Replies

3. HP-UX

alternative for egrep -o on HP-UX

Hello to all board members!! I have a problem on a HP-UX system. I should write a script. Therefore I need to search after IP addresses in the output of a command. On Debian this works: ifconfig | egrep -o "{1,3}\.{1,3}\.{1,3}\.{1,3}" The script where i need this is not ifconfig, but... (2 Replies)
Discussion started by: vostro
2 Replies

4. UNIX for Dummies Questions & Answers

Advise on HP unix

Hi everyone I need help i want to learn Unix and do some certification on Unix Hp the reason why i choose Hp is because the company that i work for uses HP and interested in knowing a lt more about what i do i montor the unix server (disk space CPU usage memory utilization,proceeses job... (5 Replies)
Discussion started by: liema
5 Replies

5. Linux

Please advise me.

Hello all, I have a question, and would like some advice please. I am a windows guy by trade....5 years in the Marines is where I learnt a lot of what i know. I took a junior level sys admin job...learned a bit more...and now I do IT security. All of this happened in the last 8 years. So I'm 27... (2 Replies)
Discussion started by: Quality
2 Replies

6. UNIX for Dummies Questions & Answers

search ")" with egrep - egrep: syntax error

Hi Guys, we have a shell script which basically query the Database which retrieves huge data and use the data with "egrep" . Now there is some data which contains characters like "abc)" and the same is used like below : "egrep (.+\|GDPRAB16\|GDPR/11702 96 abc)\|$ temp.txt" now while... (7 Replies)
Discussion started by: sagarjani
7 Replies

7. UNIX for Dummies Questions & Answers

your advise on mysql, please

My intention is to set up a mysql on unix. Could you provide some info, please, useful for realization of the plan (other than "read the documentation" that will be done anyway)? vaguely yours` sehrguey (2 Replies)
Discussion started by: serguey
2 Replies

8. UNIX for Dummies Questions & Answers

Egrep cheat sheet anywhere? Looking for meaning of egrep -c

Hi I've been searching google and have not found what egrep -c means. Does anyone know where I can get a cheat sheet or what that -c means? thanks, Linda (2 Replies)
Discussion started by: leelm
2 Replies

9. UNIX for Dummies Questions & Answers

I am one of the newbies, please advise

I am new to UNIX and Linux. I have some experiences with Windows server. I am thinking to start with those OS (Unix/Linux) and more specifically with the OS for the server. however, i have no idea which one would i start first, unix or linux? Because i also dont know how they are different. ... (3 Replies)
Discussion started by: sanlen
3 Replies
Login or Register to Ask a Question