Uniq not doing what I want it to


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Uniq not doing what I want it to
# 1  
Old 10-28-2011
Uniq not doing what I want it to

I have a master list of servers. I also have a list of servers I'm not supposed to touch. I'm trying to filter out the list servers that I'm not supposed to touch from the master list of servers, so I will have a "master list of servers I can touch". When I try to filter these I'm not getting the desired result:

Code:
for i in $(cat dont_touch_list);do grep -iv $i master_list; done | uniq > master_touch_list

What this is doing, it appears, is echoing the master_list into the master_touch_list for each interation of the dont_touch_list. What am I doing wrong?
# 2  
Old 10-28-2011
The problem is with the grep "-v" option, it prints all lines that don't match a pattern, in your case the value of ${i}.

Try:
Code:
for i in $(cat dont_touch_list);do grep -iv $i master_list; done | sort | uniq > master_touch_list

# 3  
Old 10-28-2011
Try this...
Code:
grep -vf dont_touch_list master_list > master_touch_list

--ahamed
This User Gave Thanks to ahamed101 For This Post:
# 4  
Old 10-28-2011
You can probably use the join utility to do what you want to do.

For example, if do-not-touch file contains:
Code:
a
b
c

and master-list file contains:
Code:
a
b
c
d
e
f

Assuming the files are sorted:
Code:
$ join -v 2 do-not-touch master-list
d
e
f

This User Gave Thanks to fpmurphy For This Post:
# 5  
Old 10-28-2011
Quote:
Originally Posted by felipe.vinturin
The problem is with the grep "-v" option, it prints all lines that don't match a pattern, in your case the value of ${i}.

Try:
Code:
for i in $(cat dont_touch_list);do grep -iv $i master_list; done | sort | uniq > master_touch_list

This did not produce the desired result. For some reason the master_touch_list still had items in it from the dont_touch_list. But thank you for the suggestion!

Quote:
Originally Posted by ahamed101
Try this...
Code:
grep -vf dont_touch_list master_list > master_touch_list

--ahamed
Without going through and individually checking each and every item in the master_touch_list this appears to have worked. Running a for loop grepping each item from the dont_touch_list in the master_touch_list produced no results so I think I'm good. Appreciate your help and I'll keep murphy's suggestion in mind as well.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Uniq help

hello I want to check on first column duplicates and print the unique first and second columns My trial output is not generating what I needed, i.e the second column. thanks in advance (5 Replies)
Discussion started by: bhargavpbk88
5 Replies

2. OS X (Apple)

Uniq tag

Hello, I have recently imported all my scripts into OS X. There is one unix command that is problematic in the new environment: sort temp7.txt | uniq -u -w4 > temp8.txt The system doesn't seem to like the -w tag. Is there an alternative command? Here is the objective: Data: 1234 aaa... (1 Reply)
Discussion started by: palex
1 Replies

3. Shell Programming and Scripting

uniq commmand

Hi I'm having a file (extract.txt) which contains lots of repeated values i want to extract it only with unique values,while am using the the uniq command It result's with 1510 lines this too have the duplicates,pls help me on this. I attached my extract.txt file as attachment here. ... (2 Replies)
Discussion started by: thelakbe
2 Replies

4. Shell Programming and Scripting

uniq -c

When I do uniq -c on a list of sorted numbers, for eg: 1 1 2 2 2 3 3 4 It outputs 2 1 3 2 2 3 1 4. Now, is there a way to sort on the column that "uniq -c" produced? (2 Replies)
Discussion started by: prasanna1157
2 Replies

5. Shell Programming and Scripting

Keep the last uniq record only

Hi folks, Below is the content of a file 'tmp.dat', and I want to keep the uniq record (key by first column). However, the uniq record should be the last record. 302293022|2|744124889|744124889 302293022|3|744124889|744124889 302293022|4|744124889|744124889 302293022|5|744124889|744124889... (4 Replies)
Discussion started by: ChicagoBlues
4 Replies

6. Shell Programming and Scripting

Help with uniq command

I call.... cat 1.txt | uniq -c Sample of whats in 1.txt vmstat cd exit w cd cd cd newgrp xinit f cd cd cd rlogin (2 Replies)
Discussion started by: Bandit390
2 Replies

7. UNIX for Dummies Questions & Answers

Difference between plain "uniq" and "uniq -u"

Dear all, It's not entirely clear to me from manpage the difference between them. Why we still need "-u" flag? - monkfan (3 Replies)
Discussion started by: monkfan
3 Replies

8. Shell Programming and Scripting

How to use uniq on a certain field?

How can I use uniq on a certain field or what else could I use? If I want to use uniq on the second field and the output would remove one of the lines with a 5. bob 5 hand jane 3 leg jon 4 head chris 5 lungs (1 Reply)
Discussion started by: Bandit390
1 Replies

9. Shell Programming and Scripting

who - uniq output

Hi, I'd like to have a script what takes the 'who' output and grabs the user names and outputs just the user name, and no duplicates. I know I could do something like: who | awk '{print $1}' | uniq -u but I'd like to stay away from using the 'uniq' comand and just use awk. Thanks (5 Replies)
Discussion started by: lochraven
5 Replies

10. HP-UX

help on UniQ

All, Can anybody provide me the links to the documentation on UniQPrint? I need to prepare some documents to help my co-workers to learn UniQPrint. Regards, Vishal (0 Replies)
Discussion started by: vishal_ranjan
0 Replies
Login or Register to Ask a Question