Sponsored Content
Top Forums Shell Programming and Scripting How to use grep in a loop using a bash script? Post 302976350 by aberg on Tuesday 28th of June 2016 12:53:38 PM
Old 06-28-2016
How to use grep in a loop using a bash script?

Dear all,

Please help with the following.

I have a file, let's call it data.txt, that has 3 columns and approx 700,000 lines, and looks like this:

Code:
rs1234  A  C
rs1236  T  G
rs2345  G  T

Moderator's Comments:
Mod Comment Please use code tags as required by forum rules!


I have a second file, called reference.txt, which has one column with about 500,000 lines, and contains some, but not all of the values of column 1 in data.txt. e.g.

Code:
rs1234
rs2345
...

I want to 'grep' out all the lines in data.txt that have a match in reference.txt, so that I end with:

Code:
rs1234  A  C
rs2345  G  T

I have tried:
Code:
cat data.txt | grep -f reference.txt > output.txt

But this was taking far too long.

I therefore thought I might need to loop it using a bash script. I had a go, but got nowhere with the following:

Code:
for i in reference.txt; do
grep "$i" data.txt
done

I am sure that this must be quite simple to do, but would be grateful for your help with this.

Thank you,

AB

Last edited by RudiC; 06-29-2016 at 06:26 AM.. Reason: Added code tags.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

loop does not execute in bash script?

I have a very basic bash shell script, which has many "while... done; for .... done" loop clauses, like the following ~~ #!/bin/bash while blablalba; do .... done < /tmp/file for line in `cat blablabla`; do grep $line /tmp/raw ; done > /tmp/1; while blablalba2; do .... done <... (2 Replies)
Discussion started by: fedora
2 Replies

2. Shell Programming and Scripting

error in bash script 'if' loop

SEND_MESSAGE=test echo $SEND_MESSAGE if then echo `date` > update_dt_ccaps.lst echo "The file transfer failed" >> update_dt_ccaps.lst SEND_MESSAGE=false fi The above code is showing error in bash shell as : ./test: line 5: [: test: integer expression expected ... (2 Replies)
Discussion started by: DILEEP410
2 Replies

3. Shell Programming and Scripting

Getting error on for loop - bash script

Hi, I am working on bash script after a long time. I am getting error near done statement while running a for loop snippet. The error says "Syntax error near unexpcted token 'done'" please suggest what could be wrong. here is the snippet elements=${#option_arr} //an array of values... (1 Reply)
Discussion started by: arundhati_s
1 Replies

4. Shell Programming and Scripting

(BASH) Using a loop variable to grep something in a file?

Hi, I have a loop running until a variable L that is read previously in the full script. I'd like to grep some information in an input file at a line that contains the value of the loop parameter $i. I've tried to use grep, but the problem is nothing is written in the FILE files. It seems grep... (5 Replies)
Discussion started by: DMini
5 Replies

5. UNIX for Dummies Questions & Answers

Problem with multiple grep in bash loop

Hello, I am trying to create a matrix of 0's and 1's depending on whether a gene and sample name are found in the same line in a file called results.txt. An example of the results.txt file is (tab-delimited): Sample1 Gene1 ## Gene2 ## Sample2 Gene2 ## Gene 4 ## Sample3 Gene3 ... (2 Replies)
Discussion started by: InfoSeeker2
2 Replies

6. Shell Programming and Scripting

Expect script called in loop from Bash Script

Having issues with an expect script. I've been scripting bash, python, etc... for a couple years now, but just started to try and use Expect. Trying to create a script that takes in some arguments, and then for now, just runs a pwd command(for testing, final will be command I pass). Here is... (0 Replies)
Discussion started by: cbo0485
0 Replies

7. Shell Programming and Scripting

Bash script - loop question

Hi Folks, I have a loop that goes through an array and the output is funky. sample: array=( 19.239.211.30 ) for i in "${array}" do echo $i iperf -c $i -P 10 -x CSV -f b -t 50 | awk 'END{print '$i',$6}' >> $file done Output: 19.239.211.30 19.2390.2110.3 8746886 seems that when... (2 Replies)
Discussion started by: nitrohuffer2001
2 Replies

8. Shell Programming and Scripting

While loop with input in a bash script

I have the following while loop that I put in a script, demo.sh: while read rna; do aawork=$(echo "${rna}" | sed -n -e 's/\(...\)\1 /gp' | sed -f rna.sed) echo "$aawork" | sed 's/ //g' echo "$aawork" | tr ' ' '\012' | sort | sed '/^$/d' | uniq -c | sed 's/*\(*\) \(.*\)/\2: \... (3 Replies)
Discussion started by: faizlo
3 Replies

9. UNIX for Beginners Questions & Answers

Help with date in bash script for loop from YYYYMMDDHHMM

Hi everyone I need some help I want to create an script which does some processing it takes the two arguments 201901010000 and 201901020200 - so YYYMMDDHHMM I want to split processing into hours from start until end, I dont get why this works but when I add to a future variable... (1 Reply)
Discussion started by: kl1ngac1k
1 Replies

10. UNIX for Beginners Questions & Answers

Help with a bash loop script

Create a single bash script that does the following: a. Print out the number of occurrences for each motif that is found in the bacterial genome and output to a file called motif_count.txt b. Create a fasta file for each motif (so 3 in total) which contains all of the genes and their... (6 Replies)
Discussion started by: dre
6 Replies
Locale::Script(3perl)					 Perl Programmers Reference Guide				     Locale::Script(3perl)

NAME
Locale::Script - standard codes for script identification SYNOPSIS
use Locale::Script; $script = code2script('phnx'); # 'Phoenician' $code = script2code('Phoenician'); # 'Phnx' $code = script2code('Phoenician', LOCALE_CODE_NUMERIC); # 115 @codes = all_script_codes(); @scripts = all_script_names(); DESCRIPTION
The "Locale::Script" module provides access to standards codes used for identifying scripts, such as those defined in ISO 15924. Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 15924 four-letter codes will be used. SUPPORTED CODE SETS
There are several different code sets you can use for identifying scripts. The ones currently supported are: alpha This is a set of four-letter (capitalized) codes from ISO 15924 such as 'Phnx' for Phoenician. This code set is identified with the symbol "LOCALE_SCRIPT_ALPHA". The Zxxx, Zyyy, and Zzzz codes are not used. This is the default code set. numeric This is a set of three-digit numeric codes from ISO 15924 such as 115 for Phoenician. This code set is identified with the symbol "LOCALE_SCRIPT_NUMERIC". ROUTINES
code2script ( CODE [,CODESET] ) script2code ( NAME [,CODESET] ) script_code2code ( CODE ,CODESET ,CODESET2 ) all_script_codes ( [CODESET] ) all_script_names ( [CODESET] ) Locale::Script::rename_script ( CODE ,NEW_NAME [,CODESET] ) Locale::Script::add_script ( CODE ,NAME [,CODESET] ) Locale::Script::delete_script ( CODE [,CODESET] ) Locale::Script::add_script_alias ( NAME ,NEW_NAME ) Locale::Script::delete_script_alias ( NAME ) Locale::Script::rename_script_code ( CODE ,NEW_CODE [,CODESET] ) Locale::Script::add_script_code_alias ( CODE ,NEW_CODE [,CODESET] ) Locale::Script::delete_script_code_alias ( CODE [,CODESET] ) These routines are all documented in the Locale::Codes man page. SEE ALSO
Locale::Codes Locale::Constants http://www.unicode.org/iso15924/ Home page for ISO 15924. AUTHOR
See Locale::Codes for full author history. Currently maintained by Sullivan Beck (sbeck@cpan.org). COPYRIGHT
Copyright (c) 1997-2001 Canon Research Centre Europe (CRE). Copyright (c) 2001-2010 Neil Bowers Copyright (c) 2010-2011 Sullivan Beck This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2011-09-26 Locale::Script(3perl)
All times are GMT -4. The time now is 12:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy