Sponsored Content
Top Forums Shell Programming and Scripting How to use a loop for multiple files in a folder to run awk command? Post 302940849 by sajmar on Thursday 9th of April 2015 12:31:28 PM
Old 04-09-2015
How to use a loop for multiple files in a folder to run awk command?

Dear folks

I have two data set which there names are "final.map" and "1.geno" and look like this structures:
final.map:
Code:
gi|358485511|ref|NC_006088.3| 2044
 
 gi|358485511|ref|NC_006088.3| 2048
 gi|358485511|ref|NC_006088.3| 2187
 gi|358485511|ref|NC_006088.3| 17654
 
 gi|358485511|ref|NC_006088.3| 17666


1.geno:
Code:
gi|358485511|ref|NC_006088.3| 2048   G C 0 1 1
 gi|358485511|ref|NC_006088.3| 17654 A G 1 1 2
 
 gi|358485511|ref|NC_006088.3| 17666 A G 0 1 1
 
 gi|358485511|ref|NC_006088.3| 17785 G A 0 1 1
 gi|358485511|ref|NC_006088.3| 30347 G C 1 1 2


Now, I am trying to run this command below:

Code:
awk -f example.awk 1.geno final.map > 1.dat


In this command "example.awk" contains the below command:
NR==FNR{a[$1,$2]=$5" "$6" "$7;next}{print $1,$2,a[$1,$2]?a[$1,$2]:"0 0 0"}


the output of the awk command give us "1.dat" which is
Code:
gi|358485511|ref|NC_006088.3| 2044   0 0 0
 
gi|358485511|ref|NC_006088.3| 2048   0 1 1
 
 gi|358485511|ref|NC_006088.3| 2187   0 0 0
 
 gi|358485511|ref|NC_006088.3| 17654 1 1 2
 
 gi|358485511|ref|NC_006088.3| 17666 0 1 1




My problem is I have around 300 *.geno files which I want to get *.dat out of the awk command. I am knowing the loop in unix could be helpful but I think I am using the loop wrong in some way.

Could any one give me an idea how to avoid to run awk each time separately and do it at one time by looping?

Last edited by vgersh99; 04-09-2015 at 02:07 PM.. Reason: code tags, please!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to run awk command having multiple lines

Hi, Can u see the code below. set xyz = `cat testt1.txt | awk '/-----/{\ print $1 }\ ' | tail -1` I need to execute it in c shell . What is wrong with the above command. When i write everything on a single line then it is working. Can anybody help me . (0 Replies)
Discussion started by: nani_g
0 Replies

2. UNIX for Dummies Questions & Answers

Foreach loop to run a perl script on multiple files

Hi, I have thousands of files in a directory that have the following 2 formats: 289620178.aln 289620179.aln 289620180.aln 289620183.aln 289620184.aln 289620185.aln 289620186.aln 289620187.aln 289620188.aln 289620189.aln 289620190.aln 289620192.aln.... and: alnCDS_1.fasta (1 Reply)
Discussion started by: greptastic
1 Replies

3. Shell Programming and Scripting

How to run multiple awk files

I'm trying some thing like this. But not working It worked for bash files Now I want some thing like that along with multiple input files by redirecting their outputs as inputs of next command like below Could you guyz p0lz help me on this #!/usr/bin/awk -f BEGIN { } script1a.awk... (2 Replies)
Discussion started by: repinementer
2 Replies

4. Shell Programming and Scripting

using mv command for moving multiple files in a folder

Hi, I have a requirement where I need to move Bunch of folders containing multiple files to another archive location. i want to use mv command .I am thinking when we use mv command to move directory does it create directory 1st and then move all the files ? e.g source... (4 Replies)
Discussion started by: rkmbcbs
4 Replies

5. Shell Programming and Scripting

Receiving 'ambiguous redirect' when trying to run command against multiple files

I came across the command string on https://www.unix.com/shell-programming-scripting/141885-awk-removing-data-before-after-pattern.html which was what I was looking for to be able to remove data before a certain pattern. However, outputting the result to a file seems to work on an individual basis... (4 Replies)
Discussion started by: HLee1981
4 Replies

6. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as... (2 Replies)
Discussion started by: ida1215
2 Replies

7. Shell Programming and Scripting

Script for telnet and run one command kill it and run another command using while loop

( sleep 3 echo ${LOGIN} sleep 2 echo ${PSWD} sleep 2 while read line do echo "$line" PID=$? sleep 2 kill -9 $PID done < temp sleep 5 echo "exit" ) | telnet ${HOST} while is executing only command and exits. (5 Replies)
Discussion started by: sooda
5 Replies

8. Shell Programming and Scripting

FOR loop with multiple files as input and awk

Hi all , i want to pass multiple files as input to a for loop for i in file1 file2 file3 do some awk action < $i >> $i.out done but im getting error in that for loop is the way i use to pass files to awk using for correct and 2.we can directly pass multiple files to awk as... (7 Replies)
Discussion started by: zozoo
7 Replies

9. UNIX for Dummies Questions & Answers

Loop awk command on files in a folder

Hi, I'd like to loop an action over all files with given extension within a folder. The "main" action is: awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }' input.txt > output.txt The input.txt should be every file in the folder with *.subVCF extension; and the output should be a file... (3 Replies)
Discussion started by: dovah
3 Replies

10. Programming

Run sed and awk in multiple files in adirectory

Dear linux users I was running around of 200 djob for a Blastp search in a cluster. All my input files were protein fasta file (prot.fna.1, prot.fna.2 ...prot.fna.200). The output of each individual slurm job is located in a corresponding file ending with *test (prot.fna.1.test,... (10 Replies)
Discussion started by: Dieunel
10 Replies
ypxfr(1M)																 ypxfr(1M)

NAME
ypxfr, ypxfr_1perday, ypxfr_1perhour, ypxfr_2perday - transfer NIS database from server to local node SYNOPSIS
domain] host] domain] tid prog server] mapname Remarks The Network Information Service (NIS) was formerly known as Yellow Pages (YP). Although the name has changed, the functionality of the service remains the same. DESCRIPTION
copies a Network Information Service (NIS) map (database) to the local host from a NIS server by using the NIS services. A map can be copied regardless of its age, or it can be copied depending on whether its modification time (order number) is more recent than that of the local map. The command creates a temporary map in directory where domain is the NIS domain. The command fills the map with mapname entries, obtains the map parameters (master and order number), and loads them. It then clears the old version of mapname and moves the temporary map to the existing mapname. If is run interactively, it writes messages to standard output. If is invoked without a controlling terminal and if the log file exists, appends all its messages to that file. Since is usually run from the superuser's file (see crontab(1)) or by (see yppush(1M)), the log file can retain a record of what attempted and what the results were. To maintain consistency between NIS servers, should be executed periodically for every map in the NIS database. Different maps change at different rates. For example, the map may not change for months at a time, and might therefore be checked for changes only once a day, such as in the early morning hours. However, may change several times per day, so hourly checks for updates might be more appropriate. A file can perform these periodic checks and transfers automatically. Rather than having a separate file for each map, requests can be grouped in a shell script to update several maps at once. Example scripts (mnemonically named) are in and They serve as reasonable rough drafts that can be changed as appropriate. Refer to ypfiles(4) and ypserv(1M) for an overview of the Network Information Service. Options recognizes the following options: Preserve the resolver flag in the map during transfer. Do not send a "clear current map" request to the local process. Use this flag if is not running locally when you are running Otherwise, complains that it cannot talk to the local and the transfer fails. If is running locally, do not use this flag. Copy the map from a NIS server in domain rather than the domain returned by (see domainname(1)). Force the map to be copied, even if its order number at the remote NIS server is not more recent than the order number of the local map. Obtain the map from host, regardless of its master server. If this option is not used, asks the NIS service for the master's host name and tries to obtain its map. The host can be a name or an IP address of the form Specify a source domain from which to transfer a map that should be the same across domains (such as the map. When invokes it specifies that should call back a process (that initiated the transfer) at the host server, registered as program number prog, and waiting for a response to transaction tid. AUTHOR
was developed by Sun Microsystems, Inc. FILES
Log file The following scripts are suggested for use with Run one transfer per day Run two transfers per day Hourly transfers of "volatile" maps SEE ALSO
crontab(1), domainname(1), cron(1M), ypinit(1M), yppush(1M), ypserv(1M), ypfiles(4). ypxfr(1M)
All times are GMT -4. The time now is 02:24 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy