Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

ksh Remove and replace repeated in file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting ksh Remove and replace repeated in file
# 1  
Old 12-01-2012
ksh Remove and replace repeated in file
Hi, i need to read a line from a file and count the number of times it appear in, then continuous to the second line with the same. So when i count a line i have to remove all duplicates in the file to not count it another time.

Code:
while read line
do
    n=$(grep -c $line File)
    print "$line : $n"
    sed "/$line/d" > Aux
    mv Aux File
done < File

I try with a file like this:

Code:
Antonio
Carlos
Roberto
Antonio
Roberto

And return:

Code:
Antonio : 2
Carlos : 1
Roberto : 2
Antonio : 0
Roberto : 0

The problem i think is that i cant replace a file with mv wich is opened, but i dont know. if you have a better way to solve the problem iŽll be gratefull.

Srry for my bad English Smilie
Thanks in advance.
Last edited by ToniX; 12-01-2012 at 07:48 AM..
# 2  
Old 12-01-2012
Try

Code:
awk 'NR==FNR{A[$0]++;next}{print $0,":",A[$0]?A[$0]:"0";delete A[$0]}' file file

This User Gave Thanks to pamu For This Post:
ToniX
# 3  
Old 12-01-2012
Returns the same at post 1:

Code:
Antonio : 2
Carlos : 1
Roberto : 2
Antonio : 0
Roberto : 0

With the repeated count. I dont know much about the order awk, but im learning right now.

---------- Post updated at 08:00 AM ---------- Previous update was at 07:32 AM ----------

I just solved it, thanks pamu for your help. I need an extra file to store all the non repeated lines and then search in the repeated lines file.

Code:

Code:
sort File.txt | uniq > Aux
while read Auxline
do
    n=$(grep -c $Auxline File.txt)
    print "$Auxline: $n"
done < Aux

# 4  
Old 12-01-2012
Or try..
Code:
awk '{a[$0]++;next}END{for(i in a)print i,a[i]}' filename

This User Gave Thanks to michaelrozar17 For This Post:
ToniX
# 5  
Old 12-01-2012
Quote:
Originally Posted by michaelrozar17
Or try..
Code:
awk '{a[$0]++;next}END{for(i in a)print i,a[i]}' filename

I try it and it works too, thanks Smilie. awk is more powerful than I thought Smilie
# 6  
Old 12-01-2012
Code:
sort File.txt | uniq -c

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicate lines which has been repeated 4 times

Remove duplicate lines which has been repeated 4 times attached test.txt below command tried and not getting expect output. for i in `cat test.txt | uniq` do num=`cat test.txt | grep $i | wc -l` echo $i $num done test.txt ... (17 Replies)
Discussion started by: Kalia
17 Replies

2. Shell Programming and Scripting

Remove repeated letter words

Hi, I have this text file with these words and I need help with removing words with repeated letter from these lines. 1 ama 5 bib 29 bob 2 bub 5 civic 2 dad 10 deed 1 denned 335 did 1 eeee 1 eeeee 2 eke 8... (4 Replies)
Discussion started by: crepe6
4 Replies

3. Shell Programming and Scripting

ksh Loop through file one, remove lines from file two

Good Afternoon, I start with a file named biglist.txt. I have another file smallerlist. txt I want to remove the lines from smallerlist.txt from biglist.txt and leave those lines that do not reside in smallerlist.txt. Thanks !! (2 Replies)
Discussion started by: popeye
2 Replies

4. Shell Programming and Scripting

Find repeated word and take sum of the second field to it ,for all the repeated words in awk

Hi below is the input file, i need to find repeated words and sum up the values of it which is second field from the repeated work.Im trying but getting no where close to it.Kindly give me a hint on how to go about it Input fruits,apple,20,fruits,mango,20,veg,carrot,12,veg,raddish,30... (11 Replies)
Discussion started by: 100bees
11 Replies

5. Shell Programming and Scripting

Remove regularly repeated lines

How can i delete some regular repeated lines in a file? example: in_file EDGE 1 2 12 EDGE 2 3 23 EDGE 3 4 34 EDGE 5 6 56 EDGE 6 7 67 EDGE 7 8 78 EDGE 9 10 910 EDGE 10 11 1011 EDGE 11 12 1112 EDGE 13 14 1314 EDGE 14 15 1415 EDGE 15 16 1516 EDGE 17 18 1718 EDGE 18 19 1819 EDGE 19... (8 Replies)
Discussion started by: saeed.soltani
8 Replies

6. Shell Programming and Scripting

remove brackets and put it in a column and remove repeated entry

Hi all, I want to remove the remove bracket sign ( ) and put in the separate column I also want to remove the repeated entry like in first row in below input (PA156) is repeated ESR1 (PA156) leflunomide (PA450192) (PA156) leflunomide (PA450192) CHST3 (PA26503) docetaxel... (2 Replies)
Discussion started by: manigrover
2 Replies

7. Shell Programming and Scripting

remove anything after repeated string pattern found

HI, Can anyone help me with a script. i/p calc 1 2 3 4 5 6 7 8 calc 4 5 6 calc 7 8 9 o/p calc 1 2 3 4 5 6 7 8 calc 4 5 6 i.e remove anything after where two times the string calc is found. thanks (3 Replies)
Discussion started by: Indra2011
3 Replies

8. Shell Programming and Scripting

Remove repeated line using Perl

I am new to Perl and in text file of around 1000 lines having around 500 repeated line which I felt is no use and want to remove these line.so can somebody help in same for providing sample code how can i remove these repeated line in a file. (11 Replies)
Discussion started by: dinesh.4126
11 Replies

9. Shell Programming and Scripting

remove and replace text in a file

Hello all, How would I go to a particular line in a file and remove certain text from it and replace with something that I want it to be there. like: file /etc/abc now look for line HOME="/export/xyz" in /etc/abc and then replace with HOME=/"export/xyz1" thanks in advance guys. (1 Reply)
Discussion started by: solaix14
1 Replies

10. Shell Programming and Scripting

Find/replace to new file: ksh -> perl

I have korn shell script that genretaets 100 file based on template replacing the number. The template file is as below: $ cat template file number: NUMBER The shell script is as below: $ cat gen.sh #!/bin/ksh i=1; while ((i <= 100)); do sed "s/NUMBER/$i/" template > file_${i} ((... (1 Reply)
Discussion started by: McLan
1 Replies
Login or Register to Ask a Question