search for a string in a text file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting search for a string in a text file
# 1  
Old 12-05-2010
search for a string in a text file

I want to write a script to check for duplicates
For example: I have a text file with information in the format of /etc/passwd
alice:x:1008:555:William Williams:/home/bill:/bin/bash
bob:x:1018:588:Bobs Boos:/home/bob:/bin/bash
bob:x:1019:528:Robt Ross:/home/bob:/bin/bash
james:x:1012:518:Tilly James:/home/bob:/bin/bash

I want to simply check if there are duplicate users and if there are, output the line to standard error. So in the example above since bob appears twice my output would simply generate something like:
Error duplicate user
bob:x:1018:588:Bobs Boos:/home/bob:/bin/bash
bob:x:1019:528:Robt Ross:/home/bob:/bin/bash

Right now I have a while loop that reads each line and stores each piece of information in a variable using awk -F that is delimited with ":". After storing my username I am not too sure on the best approach to check to see if it already exists.

Some parts of my code:
while read line; do
user=`echo $line | awk -F : '{print $1}'`
match=`grep $user $1`($1 is the txtfile)
if [ $? -ne 0 ]; then
echo "Unique user"
else
echo "Not unique user"
then somehow grep those lines and output it

fi

The matching does not produce the right results
Suggestions?
# 2  
Old 12-05-2010
Code:
awk -F: '{print "^"$1":"}' /etc/passwd | uniq -d | grep -f - /etc/passwd


Last edited by ctsgnb; 12-05-2010 at 06:07 AM..
# 3  
Old 12-05-2010
Perhaps it is best to include the field separator so bob and bobette do not pass as duplicates and also introduce a sort step because uniq only works on adjacent matching lines
Code:
awk -F: '{print "^"$1 FS}' /etc/passwd | sort | uniq -d | grep -f -  /etc/passwd

This would do the same thing with the userid field...
Code:
awk -F: '{print "^[^:]*:x:"$3":"}' /etc/passwd | sort | uniq -d | grep -f -  /etc/passwd

Or combined in one awk, something like this:
Code:
awk -F: 'NR==FNR{A[$1]++;B[$3]++;next}A[$1]>1||B[$3]>1' /etc/passwd /etc/passwd


Last edited by Scrutinizer; 12-05-2010 at 05:45 AM..
# 4  
Old 12-05-2010
Read /etc/passwd for only one time.
Code:
awk -F: '{a[$1]=(a[$1]=="")?$0:a[$1] ORS $0;b[$1]++}END {for (i in b) if (b[i]>1)print a[i]}' /etc/passwd

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Search a string and display its location on the entire string and make a text file

I want to search a small string in a large string and find the locations of the string. For this I used grep "string" -ob <file name where the large string is stored>. Now this gives me the locations of that string. Now how do I store these locations in a text file. Please use CODE tags as... (7 Replies)
Discussion started by: ANKIT ROY
7 Replies

2. Shell Programming and Scripting

Read in search strings from text file, search for string in second text file and output to CSV

Hi guys, I have a text file named file1.txt that is formatted like this: 001 , ID , 20000 002 , Name , Brandon 003 , Phone_Number , 616-234-1999 004 , SSNumber , 234-23-234 005 , Model , Toyota 007 , Engine ,V8 008 , GPS , OFF and I have file2.txt formatted like this: ... (2 Replies)
Discussion started by: An0mander
2 Replies

3. Programming

Search a string in a text file in C

Hello guys, i want some help please in coding that program , " A mini dictionary" the file looks like : Waver --- To be hesitated retirement --- life after end of career The user enter the word , and then it prints the meaning of it Please use CODE tags when... (4 Replies)
Discussion started by: Alyy
4 Replies

4. Shell Programming and Scripting

Search a string in a text file and add another string at the end of line

Dear All I am having a text file which is having more than 200 lines. EX: 001010122 12000 BIB 12000 11200 1200003 001010122 2000 AND 12000 11200 1200003 001010122 12000 KVB 12000 11200 1200003 In the above file i want to search for string KVB... (5 Replies)
Discussion started by: suryanarayana
5 Replies

5. Shell Programming and Scripting

Search a string in a text file and add another string at the particular position of a line

I am having a text file which is having more than 200 lines. EX: 001010122 12000 BIB 12000 11200 1200003 001010122 2000 AND 12000 11200 1200003 001010122 12000 KVB 12000 11200 1200003 In the above file i want to search for string KVB and add/replace... (1 Reply)
Discussion started by: suryanarayana
1 Replies

6. Shell Programming and Scripting

Search text file, then grep next instance of string

I need to be able to search for a beginning line header, then use grep or something else to get the very next instance of a particular string, which will ALWAYS be in "Line5". What I have is some data that appears like this: Line1 Line2 Line3 Line4 Line5 Line6 Line7 Line1 Line2 ...... (4 Replies)
Discussion started by: Akilleez
4 Replies

7. UNIX for Dummies Questions & Answers

Unix search a string in the text file

File name : Sample.txt Actually i would like to read <schema>Oracle<schema> string from input file and return only once database as my output. Please advise me. Moved to appropriate forum. (1 Reply)
Discussion started by: balajikalai
1 Replies

8. UNIX for Dummies Questions & Answers

how can search a String in one text file and replace the whole line in another file

i am very new to UNIX plz help me in this scenario i have two text files as below file1.txt name=Rajakumar. Discipline=Electronics and communication. Designation=software Engineer. file2.txt name=Kannan. Discipline=Mechanical. Designation=CADD Design Engineer. ... (6 Replies)
Discussion started by: kkraja
6 Replies

9. Shell Programming and Scripting

Perl: Search for string on line then search and replace text

Hi All, I have a file that I need to be able to find a pattern match on a line, search that line for a text pattern, and replace that text. An example of 4 lines in my file is: 1. MatchText_randomNumberOfText moreData ReplaceMe moreData 2. MatchText_randomNumberOfText moreData moreData... (4 Replies)
Discussion started by: Crypto
4 Replies

10. Shell Programming and Scripting

appending string to text file based on search string

Hi, I need to append string "Hi" to the beginning of the lines containing some specific string. How can I achieve that? Please help. Malay (1 Reply)
Discussion started by: malaymaru
1 Replies
Login or Register to Ask a Question