Search Results

Sure. First of all, I'd made a mistake in my earlier script. Corrected now in my post.
It sorts the input file on the first field (delimited by commas) so that the duplicate records (w.r.t. the...

With some assumptions:
sort -t, -k1,1 file|awk -F, 'p1==$1{if(p) print p0;p=0;print;next}{p1=$1;p0=$0;p=1}'

Hi,

Try this one,


awk -F, '{a[$1]++;if(v[$1]){v[$1]=v[$1] ORS $0;}else{v[$1]=$0;}} END {for (i in a) if (a[i]>=2) print v[i]}' file



If you want to disply only the duplicated lines,

...

And one needn't wait till the file is read completely, to determine/print the duplicate records:
awk 'a[$0]++==1' file

Try this:

'{a[$0]++} END {for (i in a) if (a[i]>=2) print i}' file

grep -f search_file * | awk -F: '{a[$2]++} END {for (i in a) if (a[i]>=50) print i}'

I am not sure if I can recommend another smart approach, but I believe my approach will run much faster than pamu's script since he/she is using looping structures.

Did you try running both &...

try something like this..

It may take some time.

while read line
do
c=0
while read files
do
grep "$line" "$files" && c++
done<All_file_list
if [[ "$c" -gt 50 ]]
then
echo "$line 50...

$ cat tst
2,3,4
1,2,3
1,2,3
2,3,4
1,2,3
2,3,4
1,2,3
1,2,3
0,1,2
$ awk -F, '{a[$2]+=$3}END{for (i in a) print "Column2 with value " i " has its sum of column3 = " a[i]}' tst
Column2 with...

This counts the occurrences of all values in the second column.

awk 'arr[$2]++ {next}
END {for (i in arr) {print i, arr[i]}} ' HB291-0W-A4.txt | sort -n > newfile

You mean something like :
awk -F, '{print $2}' HB291-0W-A4.txt | sort | uniq -cor like :
awk -F, '{++a[$2]}END{for(i in a) print i " appears " a[i] " times"}' HB291-0W-A4.txt?

We're talking bash now. Assigning the array either way is fine; num=( 086 088 143 291 292) may be a bit more efficient. The ${...} construct will allow you to access the array's features and...

When I tried pamu's awk script with the input files:
file1:
AAAA 23 45
AAAA 101 203
AAAB 44 56
AAAC 34 65
AAAD 34 87

file2:
AAAA 34 54
AAAA 201 202
AAAA 301 302
AAAE 34 56
AAAE 234 456...

Ok. Try this.. It will certainly give idea about the data. From which file it is coming.(not tested- You can change spacing as how you want)

awk...

If you don't have any problem with spacings..:D

awk 'FNR==NR{if(!a[$1]){a[$1]=$0}else{a[$0]=$0};next}{if(a[$1]){print a[$1],$2,$3;delete a[$1]}else{print $1"\t"$2,$3}}END{for(i in...

When I ran the commands suggested by pamu, I got a syntax error from the join command (the operands need to follow the options when using a standards conforming shell). Changing the join command to:...

I don't know what could be the problem here(may be OS version)
I am using BASH.
$ join file1 file2 -a 1 -a 2
AAAA 23 45 34 54
AAAB 44 56
AAAC 34 65
AAAD 34 87
AAAE 34 56

I did small mistake...

You better use join.

try

join file1 file2 -a 1 -a 2with awk..

awk 'FNR==NR{a[$1]=$0;next}{if(a[$1]){print a[$1],$2,$3;delete a[$1]}else{print $1"\t"$2,$3}}END{for(i in a){if(a[i]){print...

awk 'FNR==NR{s+=$2;next} (s1+=$2) >=(s/10) {print FNR;exit}' myFile myFile

How do you propose to do that? Start from the top and keep summing until you get to 10%? Start from the highest value and go down? Each selection method will give a different answer.

Here is...

Replace if f in *AA* with for f in *AA* Same with other loop as well.

This is a little bit longer, but I believe it does what you want:
awk -F "," 'NR == FNR {
A[$1 FS $2]=$0
next
}
$1 FS $2 in A {
printf("%s", A[$1 FS $2])
for(i...

awk version:
awk 'NR==FNR{A[$1]; next} FILENAME!=ARGV[1] && $1 in A{print $0,FILENAME}' file * | sort -k1,1 -k4,4



--
@pamu: probably need to fix $x to avoid incomplete matches, for example:...

try this..

while read x y
do
grep "$x" * | grep -v "^file" | awk -F ":" -v line="$x" '{print line,$2,$1}' >> output_file
done<file

oh i see it. i used $prev in one spot and $last in another. sorry. usually test things first but it was just a quick logic example. seems you'll also have to add something to deal with first file.....