Search Results

for exact word matches, try:
awk '
NR==FNR {a[$1]=$1; next;}
{l=0; for (i=1; i<=NF; i++) if ($i in a) {print ; next;}}
' words_file infile
for exact word match count of 1 (as shown), try:
awk '...

@Scrutinizer: Nice, but that has got a problem with larger duplicate count later in the line. For a line like R1 AA AA AA AA - - CC CC TT TT TT it yields
R1 1 1 1 1 0 0 -1 -1 0 0 0
Small...

Alternative without sorting:

awk '
BEGIN {
A["AA"]; A["CC"]; A["GG"]; A["TT"]
}
NR>1 {
minkey=""; max=0
for(i=2; i<=NF; i++) if($i in A) {
A[$i]++
...

A bit verbose - probably can be done without sorting, but....
awk -f rita.awk myFile where rita.awk is:

function quicksort(data, left, right, i, last)
{
if (left >= right) # do nothing...

I don't quite understand your output given your description and the input.
My understanding given your sample input would be - counting '-' as 0-s:

- C1 C2 C3 C4 C5
R1 1 1 0 0 -1
R2 -1 1 1 1 1...

this is still confusing....
Looks like you're using 4 diff names: tin/fin for the first 2 tables AND rid/fid for the last table.
I must be thick-skulled today, but I don't get it.
Sorry, maybe...

It is difficult to match column names "fid" with "fin" and almost impossible to match "rid" with "tin". So I discontinued that.
Given the column names match (fid=fid, rid=rid), tryawk 'FILENAME...

Hmmm... this is a bit confusing to parse this explanation.
Could you try it one more time with just ONE example and a desired output, pls.
I'm failing to see what keys you're using to find a match...

Try alsoawk 'FNR==1 {FNo++
if (FNo < 4) print > "out_t" substr(FILENAME, length(FILENAME))}

FNo < 4 {DT[NR]=$0","FILENAME
...

I am assuming there is a missing comma at line 7 in file number one. Try:

awk -F, '
NR==FNR {
A[$0]
next
}
FNR==1 {
close(f)
f="out_t" ++c
}
FNR==1 || $1 FS $2...

Although I'm not a friend of one liners, this one is really short:awk 'FNR==NR {W[$1,$3]=$2; W[$1, $4]=-$2;next} {SUM[$1]+=W[$2, $3]} END {for (s in SUM) print s, SUM[s]}' file3 file4
gr1 20
gr2...

While I must learn AWK "one of these days soon"(TM) the following Perl solution does the job required.

skrynesaver@busybox ~/$ cat tmp/tmp.pl
#!/usr/bin/perl
use strict;
open (my...

You could also try something like this:
awk '
NR == 1 {
# Note that this copies in the input header to the output, but the
# output will NOT include field 6 from the input.
print
next
}...

I adjusted your original code to support N counts/accumulators (a pair per each column). I use isnum function from Wikipedia


awk '
function isnum(x){
return(x==x+0);
}
{
...

I want to edit the sample input slightly to accommodate NA missing values, and remove the leading spaces. For a finite number of columns you can of course use arr1, arr2, arr3 etc like you have used...

Please show us some sample input, show us the output you expect from that sample input, tell us what OS and shell you're using, and tell us something about the ranges of bounds we should expect on...

This should take less memory:
wak '
{
i=$1 SUBSEP $2
}
NR==FNR {
if(i in A) {
if(A[i] && A[i]!=$4 && A[i]!=substr($4,2) substr($4,1,1)) A[i]=x
}
else {
...

Yes, Scrutinizer's script works with unsorted data.
Yes, this section prints the 1st line then jumps to the next line.
The script takes lots of memory that is processed in the END section, after...

Another way:
awk '
NR==1{
print
next
}
{
i=$1 SUBSEP $2
if(i in A) {
if(B[i]!=$4 && C[i]!=$4) N[i]
}
else {
A[i]=$0
B[i]=$4
...

Is your file sorted? How many rows?

---------- Post updated 01-11-15 at 03:50 PM ---------- Previous update was 01-10-15 at 10:40 PM ----------

A very crude solution, maybe the experts can help...

Being a bit short on time i will deal only with the first part:



No, your solution is good. It is possible to do the same in awk, though, and if you want to use awk anyway for the second part...

I need all the intermediate steps because, the intermediate files go into different pipelines once produced, you are right,some of the awk scripts can be combined and made more streamlined..

Once the input file is sorted, you could run sth likeawk 'NR==1 {P=$1}
P != $1 {if (N && D) print O; O=DL=""; N=0; D=0; P=$1}
{O=O DL $0;...

50GB: Without ordering, you cannot reasonably expect to be able to solve the problem.
You also cannot create arrays that are billions of items long because you would spend years searching for...

Hi Ravinder,
I think you understand what it is doing. Here is a different way to comment the code that takes more lines to describe, but fits in an 80 column screen:
awk '
BEGIN { # Set input and...