Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Pipe 2 bash loops together

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pipe 2 bash loops together
# 8  
Old 10-09-2015
Are you aware that mayhap all of what you try (two for loops working on several files) might be done in one single awk script? But without knowing more details that's difficult to say for sure.
This User Gave Thanks to RudiC For This Post:
cmccabe
# 9  
Old 10-09-2015
Basically, the two individual awk scripts work like this:

The awk below matches the contents of the input.txt using file.txt as the search file. The input is 60 records and the search file is ~250,000, so it only returns the 60 matches.

Code:
 
awk '
    FNR==NR{d[$0]; next;}          
    {                              
        for(k in d){               
            pat="(^|;)"k":";       
            if($2 ~ pat){
                print;             
                break;
            }
        }
    }' input.txt file.txt > match.txt

The awk below combines and averages all the matches as well as returns how many bases were in each match.

Code:
 
awk '{k=$1 OFS $2;
    s[k]+=$4; c[k]++}
  END{for(i in s)
    print i, s[i]/c[i], c[i] " bases" }' match.txt > average.txt

I use two loops to perform all that in one command. There probably is a better way to do that, but I am a scientist learning awk and not quite there yet. Thank you for your help and for the ideas Smilie.
# 10  
Old 11-16-2015
So if I wanted to string 3 for loops together in bash, is the below the best way to do so:

Basically, use specific files to run on every /*baseA_counts.txt in the directory. Thank you Smilie.

Code:
for f in /home/cmccabe/Desktop/HiQ/*base_counts.txt ; do
     bname=`basename $f`
     pref=${bname%%.txt}
     awk -f /home/cmccabe/Desktop/match.awk /home/cmccabe/Desktop/panels/PCD_unix_corrected.bed $f > /home/cmccabe/Desktop/HiQ/${pref}_PCD_coverage.bed
done
for f in /home/cmccabe/Desktop/HiQ/*base_counts.txt ; do
     bname=`basename $f`
     pref=${bname%%.txt}
     awk -f /home/cmccabe/Desktop/match.awk /home/cmccabe/Desktop/panels/BMF_unix_corrected.bed $f > /home/cmccabe/Desktop/HiQ/${pref}_BMF_coverage.bed
done
for f in /home/cmccabe/Desktop/HiQ/*base_counts.txt ; do
     bname=`basename $f`
     pref=${bname%%.txt}
     awk -f /home/cmccabe/Desktop/match.awk /home/cmccabe/Desktop/panels/Pah_unix_corrected.bed $f > /home/cmccabe/Desktop/HiQ/${pref}_PAH_coverage.bed
done

Last edited by cmccabe; 11-16-2015 at 01:28 PM.. Reason: FIXED FORMAT
# 11  
Old 11-16-2015
In fact I don't understand what you are doing in above construct (as details like samples are missing), but just from looking at it it doesn't feel right...
Why don't you run all three awks in one for loop?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Correlation Between 3 Different Loops using Bash

I have 3 loops that I use to determine the permission level of AWS user accounts. This array lists the AWS policy ARN (Amazon Resource Name): for ((policy_index=0;policy_index<${#aws_managed_policies};++policy_index)); do aws_policy_arn="${aws_managed_policies}" ... (1 Reply)
Discussion started by: bluethundr
1 Replies

2. Shell Programming and Scripting

Bash: How to use read with conditions & loops

Hello, Below I try to control that the input is good an IP : #!/bin/bash cp /home/scripts/choice_interfaces.txt /home/scripts/interfaces.txt chmod 644 /home/scripts/interfaces.txt echo -e "Please enter the network informations into the /etc/network/interfaces file, complete them below... (9 Replies)
Discussion started by: Arnaudh78
9 Replies

3. Shell Programming and Scripting

bash loops

hello i'm writing a script and I want to use a for loop inside a while loop as following: while read line; do echo $line for i in $vrm; do echo $i done done < './contacts' when i use just the while loop it prints the lines from file ./contacts just... (13 Replies)
Discussion started by: vlm
13 Replies

4. Shell Programming and Scripting

while loops and variables under bash

Hi, This is probably going to be very simple but i came across something i can't quite explain. Here is the situation: i have a list of files, which i'd like to process one by one (get the size, make some tests, whatever) and generate some statistics using different variables. Something... (5 Replies)
Discussion started by: m69w
5 Replies

5. Homework & Coursework Questions

Bash if and loops help

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: Your shell script should continue to execute until the user selects option 4 2. Relevant commands, code,... (2 Replies)
Discussion started by: boyboy1212
2 Replies

6. UNIX for Dummies Questions & Answers

Bash loops and variable scope

Hi All, I've been researching this problem and I am pretty sure that the issue is related to the while loop and the piping. There are plenty of other threads about this issue that recommend removing the pipe and using redirection. However, I haven't been able to get it working using the ssh and... (1 Reply)
Discussion started by: 1skydive
1 Replies

7. UNIX for Dummies Questions & Answers

A copy paste problem with loops in bash

Hello All, i have a really strange copy paste problem. When I write some loops in an editor for example: for j in 1 2 3 do echo "$j" done and I want to paste it to the shell, the result in the shell is: for j in 1 2 3; do e; other commands work fine and if a copy paste... (4 Replies)
Discussion started by: creamcheese
4 Replies

8. Shell Programming and Scripting

bash scripting: using multiple 'for loops'??

Hey guys, I'm kinda a noob at scripting. I am trying to create a script that uses multiple for loops with the lsiutility to monitor disk health on a system. The script runs, but it will continually echo an infinite number of LogVolumes when there are only 2 per virtual disk on my server. It's... (2 Replies)
Discussion started by: tank126
2 Replies

9. Shell Programming and Scripting

[bash] IF is eating my loops

Hi! Could someone explain me why the below code is printing the contents of IF block 5 times instead of 0? #!/bin/bash VAR1="something" VAR2="something" for((i=0;i<10;i++)) do if(($VAR1=~$VAR2)) then echo VAR1: $VAR1 echo... (3 Replies)
Discussion started by: machinogodzilla
3 Replies

10. Shell Programming and Scripting

arrays and while loops in bash

hi guys, i have an array called ARRAY which has elements in it... i am trying to assign elements of ARRAY to master_array.. i get a =: command not found error.. i=0 while do ${master_array}=${ARRAY} ((i++)) done is there something i am missing? (4 Replies)
Discussion started by: npatwardhan
4 Replies
Login or Register to Ask a Question