Iterating over a list using awk, when variable


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Iterating over a list using awk, when variable
# 1  
Old 12-17-2011
Iterating over a list using awk, when variable

Hi,

I've recently started using Awk commands as i need it to pull information out of a file and give me the mean value of a series of numbers. Here is the code i run on my Infile and it works just fine.

Code:
awk  '{if ($1 == "Mam189") print $0}' Infile | awk '{if ($1 != $2) print $0}' | awk '{sum+=$3} END { print "Mam189 = ",sum/NR}'

However i need to run this code thousands of times replacing the word "Mam189" with a different Mam# each time.

Essentially i would like to feed in a list, like this:
Mam189
Mam220
Mam18
Mam5
Mam100

and then do something like a for loop
Code:
for i in `cat list`; 
do;
awk  '{if ($1 == "$i") print $0}' Infile | awk '{if ($1 != $2) print $0}' | awk '{sum+=$3} END { print "$i= ",sum/NR}'
done;

This however does not work. I was wondering if anyone knew of a work around or a different approach i could take to do this?
# 2  
Old 12-17-2011
If you post a sample of the original unmodified input and example of the expected/desired output, it would be definitely easier.
# 3  
Old 12-17-2011
Hi,

Apologies, ive been looking at this for so long, i forget that its actually not that clear.

The original file that i am working with looks somthing like this


Quote:
Mam189 Mam329 84.07
Mam189 Mam82 66.37
Mam189 Mam83 66.81
Mam82 Mam487 71.24
Mam82 Mam484 70.80
Mam82 Mam150 73.89
Mam426 Mam181 78.32
Mam426 Mam264 78.76
Mam426 Mam485 78.76

In the first colum is DNA sequence data for Mam189, Mam82 and Mam426 that i am comparing to other sequence data, listed in the second colum. The thrid colum is the sequence similarity between the data listed in colum 1 and colum 2.

The code above essentially takes everything that Mam189 is compared to and gives an average of the third colum. The output would read
Quote:
Mam189 = 72.41666667
I would like to be able to feed in a list of names, so that the code would be able to iterate over a list and give the average for Mam189, Mam82, Mam426.

I've only had experience using a for loop before, but it wont work in this case. Not sure what the best approach is to deal with this.

Any advice would be much appreciated.
# 4  
Old 12-17-2011
Try this:


$
Code:
gawk 'NR==FNR{ a[$1] += $3; b[$1]++} NR!=FNR{ for(key in a) {if($1==key)print key,a[key]/b[key]}}' /tmp/1 /tmp/2

Mam189 72.4167
Mam426 78.6133

I used the following sample files:

Quote:
~ $ cat /tmp/1
Mam189 Mam329 84.07
Mam189 Mam82 66.37
Mam189 Mam83 66.81
Mam82 Mam487 71.24
Mam82 Mam484 70.80
Mam82 Mam150 73.89
Mam426 Mam181 78.32
Mam426 Mam264 78.76
Mam426 Mam485 78.76

~ $ cat /tmp/2
Mam189
Mam426
This User Gave Thanks to dude2cool For This Post:
# 5  
Old 12-18-2011
In addition, this will calculate the average per DNA sequence for all the entries:

Code:
awk 'END {
  for (d in dna)
    print d, dna[d]/count[d]
  }
{
  dna[$1]   += $3
  count[$1] ++
  }' infile

This User Gave Thanks to radoulov For This Post:
# 6  
Old 12-18-2011
Thanks for awk/gawk

Awesome thanks, that works great!

Best,
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Question on iterating array elements

Hi, I am trying to do something similar to the for loop example from KSH For Loop Array: Iterate Through Array Values $: cat y.ksh #!/bin/ksh # set array called nameservers set -A nameservers 192.168.1.1 192.168.1.5 202.54.1.5 # print all name servers for i in ${nameservers} do ... (3 Replies)
Discussion started by: newbie_01
3 Replies

2. Shell Programming and Scripting

Iterating awk over a directory of files?

I need to run the same awk function over an entire directly of files. This is the awk: awk '{$(NF+1)=1}1' Is there a way that I can run this command once over all of the files, along the lines of: awk '{$(NF+1)=1}1' * so that I do not have to run this several times? My main concern is... (2 Replies)
Discussion started by: owwow14
2 Replies

3. Shell Programming and Scripting

Bash Script Iterating Question

I am trying to look through one of my directories to remove certain files. I am pretty new to Unix and bash so I just need a little help in starting this. I know I would have to write two loops one to iterate the directories and one to iterate the files. How would I write the loops? (3 Replies)
Discussion started by: totoro125
3 Replies

4. Shell Programming and Scripting

Iterating over subdirectories and dealing with files within them

Hello, I am working on a coding project for a class and to test the program I have created, I have come up with 100 different test cases. The program takes four text files as input, so each of the test cases is contained in a folder with four files. I have a folder called 'tests', within which... (1 Reply)
Discussion started by: dpryor
1 Replies

5. Programming

Iterating and calloc questions.

Whilst creating the function readjust_descr I have stumble across what may be a problem or something that might just work. I was hoping someone could look at the code below and tell me if readjust_descr will clear all null pointers from the structure descr_list. struct descr descr_list =... (6 Replies)
Discussion started by: Errigour
6 Replies

6. Shell Programming and Scripting

using awk for setting variable but change the output of this variable within awk

Hi all, Hope someone can help me out here. I have this BASH script (see below) My problem lies with the variable path. The output of the command find will give me several fields. The 9th field is the path. I want to captured that and the I want to filter this to a specific level. The... (6 Replies)
Discussion started by: Cowardly
6 Replies

7. Shell Programming and Scripting

Iterating through two arrays in a single loop

Hey everyone. Is it possible to use two arrays in a loop? Basically what I am trying to do is iterate through the elements in an array, and, based on a condition, use the current position in the array to point to that index in the array. Here's the loop structure I'm looking for: ... (1 Reply)
Discussion started by: msarro
1 Replies

8. Shell Programming and Scripting

iterating over results from sqlplus

hi all, i am writing a ksh script, i am logging into an oracle db via sqlplus and running a select statement but i dont know how i can store the results from the sql so i can iterate over it and do more operations over it. I dont want to write a stored procedure with a cursor since i need to... (2 Replies)
Discussion started by: cesarNZ
2 Replies

9. Shell Programming and Scripting

Problem iterating in while loop

I am facing a problem in replacing the file contents by iterating through the list. My present code: Code: #!/bin/bash# TFILE="/tmp/vinay/testb_1.txt" while read linedo aline="$line" echo $aline code=`echo $aline|cut -d ',' -f1` country=`echo $aline|cut -d... (5 Replies)
Discussion started by: av_vinay
5 Replies

10. Shell Programming and Scripting

Problem iterating through PATH entries with spaces

I have a Bash script on Cygwin that tries to iterate through the directory entries in PATH. I need to convert the PATH value to a form that I can iterate through with "for var in $list; do". For instance, an excerpt from my PATH value is this: :/c/Program Files/Windows Imaging/:/c/Program... (2 Replies)
Discussion started by: dkarr
2 Replies
Login or Register to Ask a Question