Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to use a loop for multiple files in a folder to run awk command?
Top Forums Shell Programming and Scripting How to use a loop for multiple files in a folder to run awk command? Post 302940849 by sajmar on Thursday 9th of April 2015 12:31:28 PM
Old 04-09-2015
How to use a loop for multiple files in a folder to run awk command?
Dear folks

I have two data set which there names are "final.map" and "1.geno" and look like this structures:
final.map:
Code:
gi|358485511|ref|NC_006088.3| 2044
 
 gi|358485511|ref|NC_006088.3| 2048
 gi|358485511|ref|NC_006088.3| 2187
 gi|358485511|ref|NC_006088.3| 17654
 
 gi|358485511|ref|NC_006088.3| 17666


1.geno:
Code:
gi|358485511|ref|NC_006088.3| 2048   G C 0 1 1
 gi|358485511|ref|NC_006088.3| 17654 A G 1 1 2
 
 gi|358485511|ref|NC_006088.3| 17666 A G 0 1 1
 
 gi|358485511|ref|NC_006088.3| 17785 G A 0 1 1
 gi|358485511|ref|NC_006088.3| 30347 G C 1 1 2


Now, I am trying to run this command below:

Code:
awk -f example.awk 1.geno final.map > 1.dat


In this command "example.awk" contains the below command:
NR==FNR{a[$1,$2]=$5" "$6" "$7;next}{print $1,$2,a[$1,$2]?a[$1,$2]:"0 0 0"}


the output of the awk command give us "1.dat" which is
Code:
gi|358485511|ref|NC_006088.3| 2044   0 0 0
 
gi|358485511|ref|NC_006088.3| 2048   0 1 1
 
 gi|358485511|ref|NC_006088.3| 2187   0 0 0
 
 gi|358485511|ref|NC_006088.3| 17654 1 1 2
 
 gi|358485511|ref|NC_006088.3| 17666 0 1 1




My problem is I have around 300 *.geno files which I want to get *.dat out of the awk command. I am knowing the loop in unix could be helpful but I think I am using the loop wrong in some way.

Could any one give me an idea how to avoid to run awk each time separately and do it at one time by looping?
Last edited by vgersh99; 04-09-2015 at 02:07 PM.. Reason: code tags, please!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to run awk command having multiple lines

Hi, Can u see the code below. set xyz = `cat testt1.txt | awk '/-----/{\ print $1 }\ ' | tail -1` I need to execute it in c shell . What is wrong with the above command. When i write everything on a single line then it is working. Can anybody help me . (0 Replies)
Discussion started by: nani_g
0 Replies

2. UNIX for Dummies Questions & Answers

Foreach loop to run a perl script on multiple files

Hi, I have thousands of files in a directory that have the following 2 formats: 289620178.aln 289620179.aln 289620180.aln 289620183.aln 289620184.aln 289620185.aln 289620186.aln 289620187.aln 289620188.aln 289620189.aln 289620190.aln 289620192.aln.... and: alnCDS_1.fasta (1 Reply)
Discussion started by: greptastic
1 Replies

3. Shell Programming and Scripting

How to run multiple awk files

I'm trying some thing like this. But not working It worked for bash files Now I want some thing like that along with multiple input files by redirecting their outputs as inputs of next command like below Could you guyz p0lz help me on this #!/usr/bin/awk -f BEGIN { } script1a.awk... (2 Replies)
Discussion started by: repinementer
2 Replies

4. Shell Programming and Scripting

using mv command for moving multiple files in a folder

Hi, I have a requirement where I need to move Bunch of folders containing multiple files to another archive location. i want to use mv command .I am thinking when we use mv command to move directory does it create directory 1st and then move all the files ? e.g source... (4 Replies)
Discussion started by: rkmbcbs
4 Replies

5. Shell Programming and Scripting

Receiving 'ambiguous redirect' when trying to run command against multiple files

I came across the command string on https://www.unix.com/shell-programming-scripting/141885-awk-removing-data-before-after-pattern.html which was what I was looking for to be able to remove data before a certain pattern. However, outputting the result to a file seems to work on an individual basis... (4 Replies)
Discussion started by: HLee1981
4 Replies

6. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as... (2 Replies)
Discussion started by: ida1215
2 Replies

7. Shell Programming and Scripting

Script for telnet and run one command kill it and run another command using while loop

( sleep 3 echo ${LOGIN} sleep 2 echo ${PSWD} sleep 2 while read line do echo "$line" PID=$? sleep 2 kill -9 $PID done < temp sleep 5 echo "exit" ) | telnet ${HOST} while is executing only command and exits. (5 Replies)
Discussion started by: sooda
5 Replies

8. Shell Programming and Scripting

FOR loop with multiple files as input and awk

Hi all , i want to pass multiple files as input to a for loop for i in file1 file2 file3 do some awk action < $i >> $i.out done but im getting error in that for loop is the way i use to pass files to awk using for correct and 2.we can directly pass multiple files to awk as... (7 Replies)
Discussion started by: zozoo
7 Replies

9. UNIX for Dummies Questions & Answers

Loop awk command on files in a folder

Hi, I'd like to loop an action over all files with given extension within a folder. The "main" action is: awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }' input.txt > output.txt The input.txt should be every file in the folder with *.subVCF extension; and the output should be a file... (3 Replies)
Discussion started by: dovah
3 Replies

10. Programming

Run sed and awk in multiple files in adirectory

Dear linux users I was running around of 200 djob for a Blastp search in a cluster. All my input files were protein fasta file (prot.fna.1, prot.fna.2 ...prot.fna.200). The output of each individual slurm job is located in a corresponding file ending with *test (prot.fna.1.test,... (10 Replies)
Discussion started by: Dieunel
10 Replies

LEARN ABOUT MOJAVE

git-http-push

GIT-HTTP-PUSH(1)                                                    Git Manual                                                    GIT-HTTP-PUSH(1)

NAME

       git-http-push - Push objects over HTTP/DAV to another repository

SYNOPSIS

       git http-push [--all] [--dry-run] [--force] [--verbose] <url> <ref> [<ref>...]

DESCRIPTION

       Sends missing objects to remote repository, and updates the remote branch.

       NOTE: This command is temporarily disabled if your libcurl is older than 7.16, as the combination has been reported not to work and
       sometimes corrupts repository.

OPTIONS

       --all
           Do not assume that the remote repository is complete in its current state, and verify all objects in the entire local ref's history
           exist in the remote repository.

       --force
           Usually, the command refuses to update a remote ref that is not an ancestor of the local ref used to overwrite it. This flag disables
           the check. What this means is that the remote repository can lose commits; use it with care.

       --dry-run
           Do everything except actually send the updates.

       --verbose
           Report the list of objects being walked locally and the list of objects successfully sent to the remote repository.

       -d, -D
           Remove <ref> from remote repository. The specified branch cannot be the remote HEAD. If -d is specified the following other conditions
           must also be met:

           o   Remote HEAD must resolve to an object that exists locally

           o   Specified branch resolves to an object that exists locally

           o   Specified branch is an ancestor of the remote HEAD

       <ref>...
           The remote refs to update.

SPECIFYING THE REFS

       A <ref> specification can be either a single pattern, or a pair of such patterns separated by a colon ":" (this means that a ref name
       cannot have a colon in it). A single pattern <name> is just a shorthand for <name>:<name>.

       Each pattern pair consists of the source side (before the colon) and the destination side (after the colon). The ref to be pushed is
       determined by finding a match that matches the source side, and where it is pushed is determined by using the destination side.

       o   It is an error if <src> does not match exactly one of the local refs.

       o   If <dst> does not match any remote ref, either

           o   it has to start with "refs/"; <dst> is used as the destination literally in this case.

           o   <src> == <dst> and the ref that matched the <src> must not exist in the set of remote refs; the ref matched <src> locally is used
               as the name of the destination.

       Without '--force`, the <src> ref is stored at the remote only if <dst> does not exist, or <dst> is a proper subset (i.e. an ancestor) of
       <src>. This check, known as "fast-forward check", is performed in order to avoid accidentally overwriting the remote ref and lose other
       peoples' commits from there.

       With --force, the fast-forward check is disabled for all refs.

       Optionally, a <ref> parameter can be prefixed with a plus + sign to disable the fast-forward check only on that ref.

GIT

       Part of the git(1) suite

Git 2.17.1                                                          10/05/2018                                                    GIT-HTTP-PUSH(1)
All times are GMT -4. The time now is 05:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy