Grep that I should know but I can't remember!


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Grep that I should know but I can't remember!
# 1  
Old 01-03-2012
Grep that I should know but I can't remember!

I have a master list of hosts, and a list of bad hosts. I want to filter out the bad hosts from the master list. I was trying a few for loops but it's not providing the desired result:
Code:
for i in $(cat master_host_list);do grep -iv $i bad_host_list;done | sort | uniq

# won't work because it performs each iteration of the master_host_list which eventually includes results from previous exclusions of the bad_host_list that get overwritten in my sort/uniq
 
for i in $(cat bad_host_list);do grep -iv $i master_host_list;done | sort | uniq

# same thing

I know I did this a few months ago w/ ease and for the life of me I can't remember how I did it. Thanks in advance.
# 2  
Old 01-03-2012
Code:
grep -vf bad_host_list master_host_list

I don't think you need the sort and uniq unless there are duplicates in the master list.
# 3  
Old 01-03-2012
You need to check the invalid list for each entry and only echo if not present ie.
Code:
 for i in $(cat master_list );do grep -i $i invalid_hosts >/dev/null || echo $i;done

EDIT, Methyl's solution uses a gnu grep modifier (-f) which is brilliantly handy but not reliably portable.

Last edited by Skrynesaver; 01-03-2012 at 10:15 AM..
# 4  
Old 01-03-2012
Mythl's response uses the -f flag which is standard in many versions of grep If your distribution doesn't recognise it, then have a look at the manual page for fgrep instead.



Robin
Liverpool/Blackburn
UK
# 5  
Old 01-03-2012
Quote:
Originally Posted by methyl
Code:
grep -vf bad_host_list master_host_list

I don't think you need the sort and uniq unless there are duplicates in the master list.
Yes - that's it I remember now! Thank you so much. Yes I recall that my company uses GNU and non-GNU standard packages so I'll keep the latter in mind. Thank you all - what a relief.
# 6  
Old 01-03-2012
Nothing non-standard about "grep -f". Nice to know that GNU have followed unix standards for once.
On my unix system "grep" "fgrep" and "egrep" are actually the same binary program. It's been a long time since they were distinct programs in unix.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

What is the best way to remember UNIX / Linux Commands?

Hi all, I'm new to UNIX world and new to this forum. As I observed there are lot of commands that needs to be remembered in UNIX programming. I'm into DevOps and can anyone please tell me what are all the important commands that are useful for DevOps Engineer. NOTE: Please correct me if my... (3 Replies)
Discussion started by: maxlink
3 Replies

2. Post Here to Contact Site Administrators and Moderators

Log Out vs Remember Me

Howdy, I clicked the rememberer me when I log in, and evidently I really do not understand what that means. I had hoped that at least it would remember my user name for the next time that I log in. However, when I log out, I see a message about cookies being removed and one other thing that I... (1 Reply)
Discussion started by: danuke
1 Replies

3. What is on Your Mind?

Anyone remember this cute unix ad?

It showed a cleaning woman (probably in the evening, after most of the other employees had left work) happily typing commands on a dot matrix terminal (could've been a DEC LA120, IIRC) just because "unix is so easy to use, even a cleaning woman can use it!". If you know where to find a scanned... (2 Replies)
Discussion started by: mathiasbage
2 Replies

4. Shell Programming and Scripting

remember processed files

Hello dear community! I have the following task to accomplish: there is a directory with approximately 2 thousand files. I have to write a script which would randomly extract 200 files on the first run. On the second run it should extract again 200 files but that files mustn't intersect with... (5 Replies)
Discussion started by: sidorenko
5 Replies

5. Forum Support Area for Unregistered Users & Account Problems

Two Login's - Remember only one.

Hi All, Have only recently returned to Unix.com due to other activities (*oh the shame of it all). Anyways, when I initially came back to redesigned/revamped site I couldn't remember any of my logins. I'd originally thought that I'd used the UID of 'Cameron' but couldn't remember any details... (2 Replies)
Discussion started by: Cameron
2 Replies

6. UNIX for Dummies Questions & Answers

scripts remember the previous parameter???

I have a simple script. I have tried this in Bourne, Korn and C shells in my AIX ...no luck unset STARTQUEUE # Check parameters if then echo "*E* Batch Queue parameter is missing" else $BATCHQ/bin/setup.sh STARTQUEUE=$1 # If the queue doesn't exist, create it ... (3 Replies)
Discussion started by: KenL
3 Replies

7. UNIX for Dummies Questions & Answers

remember last visited line in vim

hi, I know we can do this; but dont know how.. I open a file using vim..browse thru it and then say :wq after reaching some line; The next time I open the same file, I want vim to position the cursor on the line where I left last time; anyone? (2 Replies)
Discussion started by: spopuri
2 Replies

8. Shell Programming and Scripting

AWK--does anyone remember it

I am trying to run awk on a 55 page Word document. I wanted to delete every occurrence of <company>, <script>, </scripts> from the file then cut & paste all of the appropriate fields to an Excel spreadsheet. Also the code is suppose to replace the dates in a new format such as "xxxx-xx-xx" ... (2 Replies)
Discussion started by: cnitadesigner
2 Replies

9. Shell Programming and Scripting

It's been awhile...help me remember

Well it's been a long time since I have used any OS besides apples and windows (raising my son). My principal would like our teachers to use UNIX as their mail system. That's not a problem, the mail system is like riding a bike you never forget. Here's my problem. She wants me to write a script... (2 Replies)
Discussion started by: catbad
2 Replies
Login or Register to Ask a Question