more than 10 identical lines


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers more than 10 identical lines
# 1  
Old 08-16-2011
more than 10 identical lines

I have a file that looks like this 10 user1s, 5 user2s and 10 users3.

10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.2 user2
10.10.1.2 user2
10.10.1.2 user2
10.10.1.2 user2
10.10.1.2 user2
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3

if $1 (10.10.1.1) has more or equal to 10, grep 10.10.1.1 and email. same, if $1 (10.10.1.3) has more or equal to 10, grep 10.10.1.3 and email.
# 2  
Old 08-16-2011
What is the grep supposed to do?

Code:
C=1
OLD=""
while read IP U
do
        if [ "$IP $U" = "$OLD" ]
        then
                C=`expr $C + 1`
        else
                C=1
        fi

        if [ "$C" -ge 10 ]
        do
                echo "user $U has done this 10 times in a row" | sendmail
                C=1
        done

        OLD="$IP $U"
done < filename

# 3  
Old 08-16-2011
Code:
# cat /tmp/yourinputfile|uniq -c|awk '{ if($1 >= 10) print $2}'

This will just print to terminal. You can modify it to mail. Left as an exercise for OP Smilie
This User Gave Thanks to dude2cool For This Post:
# 4  
Old 08-16-2011
thanks, but output should look like this when all said and done:

10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.1 user1
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
10.10.1.3 user3
# 5  
Old 08-16-2011
Use a for loop, iterate over the values and grep out of your input file.

Code:
for i in `command`
do
grep $i /tmp/yourinputfile
done

or use xargs
Pipe output from command to xargs

Code:
outputfromcommand|xargs  -I {} grep '{}' /tmp/yourinputfile

Pick your poison Smilie

Last edited by dude2cool; 08-16-2011 at 03:07 PM.. Reason: Edit
# 6  
Old 08-16-2011
Quote:
Originally Posted by dude2cool
Code:
for i in `command`
do
grep $i /tmp/yourinputfile
done

This is, again, a useless use of cat, inefficient and occasionally dangerous.

Code:
 command > /tmp/$$

while read LINE
do
        ...
done < /tmp/$$

rm -f /tmp/$$

I don't think your xargs version is going to work, since it will take multiple arguments beyond the first one to be filenames. If your system supports xargs -n, the -n 1 parameter would fix that.
# 7  
Old 08-16-2011
As per above post from corona, trying to be a better script-er (is that even a word ? Smilie ), eliminating the use of the useless cat, here is a cat free version. Looking forward to more advice, if this can be further improved.


Code:
uniq -c /tmp/yourinputfile|awk '{ if($1 >= 10) print $2}'|while read line
do
grep -i $line /tmp/yourinputfile
done

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk - If field value of consecutive records are the identical print portion of lines

I have some data that looks like this: PXD= ZW< 1,6 QR> QRJ== 1,2(5) QR> QRJ== 4,1(2) QR> QRJ== 4,2 QRB= QRB 4,2 QWM QWM 6,2 R<C ZW< 11,2 R<H= R<J= 6,1 R>H XZJ= 1,2(2) R>H XZJ= 2,6(2) R>H XZJ= 4,1(2) R>H XZJ= 6,2 RDP RDP 1,2 What I would like to do is if fields $1 and $2 are... (5 Replies)
Discussion started by: jvoot
5 Replies

2. UNIX for Beginners Questions & Answers

How to delete identical lines while leaving one undeleted?

Hi, I have a file as follows. file1 Hello Hi His Hi Hi Hungry hi so I want to delete identical lines while leaving one of them undeleted. So desired output will be Hello Hi (2 Replies)
Discussion started by: beginner_99
2 Replies

3. Shell Programming and Scripting

Combine identical lines and average the one variable field

I have the following file 299899 chrX_299716_300082 196 78.2903 299991 chrX_299982_300000 18.2538 Tajd:0.745591 FayWu:-0.245701 T2:1.45 299899 chrX_299716_300082 196 78.2903 299991 chrX_299982_300000 18.2538 Tajd:0.745591 FayWu:-0.245701 T2:0.283 311027 chrX_310892_311162 300 91.6452... (2 Replies)
Discussion started by: jfern
2 Replies

4. Shell Programming and Scripting

sed print all lines between second and third identical lines

I am trying to extract a table of data (mysql query output) from a log file. I need to print everything below the header and not past the end of the table. I have spent many hours searching with little progress. I am matching the regexp +-\{99\} with no problem. I just can't figure out how to print... (5 Replies)
Discussion started by: godfreydanials
5 Replies

5. Shell Programming and Scripting

Scan a file in realtime and execute certain commands on encountering 5 consecutive identical lines

Mysql log has something like below: I need a bash shell script that will do the following: 1) The script will scan the mysql.log file constantly in real time (something like tail -F mysql.log) 2) If it encounters 5 consecutive identical lines then it would invoke some commands (say... (4 Replies)
Discussion started by: proactiveaditya
4 Replies

6. Ubuntu

reinstall identical system?

I have my Ubuntu system nicely tailored to my needs, with specific software installed, and other things removed. I'd like to build a new PC, and have the identical software configuration on it. Is there some easy way to export the list of installed software to a file? Then have the new machine... (3 Replies)
Discussion started by: lupin..the..3rd
3 Replies

7. Shell Programming and Scripting

to get two almost identical rows into one

Hi All, I am having hard time in getting two almost identical rows into one, I know how to do if starting word is unique, anyhow this is my problem Input File: issue1 5167 dum 1 1 kkk 7888 dum 2 1 ffff 7888 dum 2 2 llll 7888 dum 3 1 eee 7888 issue2 7667 dum 2 1 jjjj 8999 dum 2 2 jjjj... (3 Replies)
Discussion started by: ricky315
3 Replies

8. Shell Programming and Scripting

Ignore identical lines

Hello Experts, I have two files called "old" and "new". My old file contains 10 lines and my new file contains 10 + "n" lines. The first field in both these files contain ID. I sort these two files on ID. I am interested in only the lines that are in the new file and not in old. I tried... (4 Replies)
Discussion started by: forumthreads
4 Replies

9. Shell Programming and Scripting

replace 2 identical strings on different lines

I am looking to replace two or more strings on different lines using sed, but not with the same variable. IE # cat xxx.file <abc> abc def ghi abc def ghi abc def ghi currently I can only change each line with the same pattern: # sed -e '/<abc>/!s/abc\(.*\)/jkl mno/' xxx.file abc jkl mno... (3 Replies)
Discussion started by: prkfriryce
3 Replies

10. UNIX for Dummies Questions & Answers

How to check the files are Identical or not?

Hi, I am taking backup in every hr and every day, I want to programmatically check my backup files are identical with original files. Any methods are available in Unix scripts? Any idea.? I don't want to download both and compare, I looking for idea to develop a script to read both files and... (3 Replies)
Discussion started by: redlotus72
3 Replies
Login or Register to Ask a Question