Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Filter and merge 2 files problem Post 302951694 by Yarinka on Sunday 9th of August 2015 08:00:09 PM
Old 08-09-2015
Filter and merge 2 files problem

Hi,

I'm trying to combine two files which have 1 column in common and filter out rows I don't need.

File 1:
Code:
ID       Start       End       Matched       Coverage
1       1       254     1515    5.96
2       1       135     402     2.98


File 2 (has 2 rows per entry):
Code:
>1 254:17:30:28.6351:1.62947
AAAAAAAAAAAAAAAAAAAAAACAGCTAAAGTTGAGGATTTCAAACAGAAAAGCAACA
>2 135:11:15:14.3786:0.609257
AAAAAAAAAAAAAAAAAAAAAACCTTCCCTCGGTCTGATATGTCTTCATTTACAATGCT

so I do filtering of the File 1 with awk (awk '$5>0 {print}' file 1 > file1_filtered) and want to merge File 1_filtered and File 2 retaining only sequences with IDs that passes >5 filter. Ideally I want to have file with 3 columns: ID Sequence Coverage:

Code:
ID       Sequence       Coverage
1       AAAAAAA......    5.96
2       AAAAAAAA....     2.98

I am a biologist making very first steps in bash scripting so I would greatly appreciate any comments or explanations on how it should work.

I use Cygwin on Windows.

Many thanks!

Moderator's Comments:
Mod Comment edit by bakunin: please use CODE-tags for file content too. Thank you.

Last edited by bakunin; 08-09-2015 at 09:17 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

2. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

3. Shell Programming and Scripting

zip code filter problem using AWK need help

Hello all; I have a large file (csv file) with addresses. I am trying to filter out specific entries based on zip code from a particular column of data. However my awk statement can not account for zipcode that begins with a certain pattern. It finds the pattern anywhere within the zipcode. ... (3 Replies)
Discussion started by: nelsonsierra
3 Replies

4. Shell Programming and Scripting

Filter files and print

Hi, I need to filter and store the files ends with log extension in the array and need to write the file names in the array to a file. I need to use array to derive this solution. Please help me out. Thanks (2 Replies)
Discussion started by: Sekar1
2 Replies

5. Programming

Problem with Mail merge in perl

This could be a simple problem for the perl experts I am trying mail merge in perl The header file is head1 which is as under ... (6 Replies)
Discussion started by: sunnyboy
6 Replies

6. Shell Programming and Scripting

how to filter files with given format

Hi, all, I have files like: nameserver 216.66.22.2 ; tserv1.ash1.ipv6.he.net. tserv13.ash1.ipv6.he.net. nameserver 216.66.38.58 ; tserv1.tor1.ipv6.he.net. tserv21.tor1.ipv6.he.net. nameserver 216.218.221.6 ;... (3 Replies)
Discussion started by: esolvepolito
3 Replies

7. Shell Programming and Scripting

Problem with filter data using sed command

Hi, I am using the following command(sed) to get the key/value pair from the string String="{ "test":"test message", "testmessage":"subscription is active, charge successfully} " }" status=$( echo $String | sed -e 's/^.*\("testmessage":*\).*$/\1/') echo $status i am getting this... (2 Replies)
Discussion started by: nanthagopal
2 Replies

8. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

9. Shell Programming and Scripting

Copying the files after filter

Hi Guys, i want copy the all files another direcotry after filtering the command. and tried as like below...it's not working. ls -ltr|awk '{print $9}'|grep "images\|\.htm"|cp *.* /home/oracle Thanks (13 Replies)
Discussion started by: bmk
13 Replies

10. Shell Programming and Scripting

Merge files and generate a resume in two files

Dear Gents, Please I need your help... I need small script :) to do the following. I have a thousand of files in a folder produced daily. I need first to merge all files called. txt (0009.txt, 0010.txt, 0011.txt) and and to output a resume of all information on 2 separate files in csv... (14 Replies)
Discussion started by: jiam912
14 Replies
Filter::Boolean(3pm)					User Contributed Perl Documentation				      Filter::Boolean(3pm)

NAME
Log::Log4perl::Filter::Boolean - Special filter to combine the results of others SYNOPSIS
log4perl.logger = WARN, AppWarn, AppError log4perl.filter.Match1 = sub { /let this through/ } log4perl.filter.Match2 = sub { /and that, too/ } log4perl.filter.MyBoolean = Log::Log4perl::Filter::Boolean log4perl.filter.MyBoolean.logic = Match1 || Match2 log4perl.appender.Screen = Log::Dispatch::Screen log4perl.appender.Screen.Filter = MyBoolean log4perl.appender.Screen.layout = Log::Log4perl::Layout::SimpleLayout DESCRIPTION
Sometimes, it's useful to combine the output of various filters to arrive at a log/no log decision. While Log4j, Log4perl's mother ship, chose to implement this feature as a filter chain, similar to Linux' IP chains, Log4perl tries a different approach. Typically, filter results will not need to be passed along in chains but combined in a programmatic manner using boolean logic. "Log if this filter says 'yes' and that filter says 'no'" is a fairly common requirement but hard to implement as a chain. "Log::Log4perl::Filter::Boolean" is a special predefined custom filter for Log4perl which combines the results of other custom filters in arbitrary ways, using boolean expressions: log4perl.logger = WARN, AppWarn, AppError log4perl.filter.Match1 = sub { /let this through/ } log4perl.filter.Match2 = sub { /and that, too/ } log4perl.filter.MyBoolean = Log::Log4perl::Filter::Boolean log4perl.filter.MyBoolean.logic = Match1 || Match2 log4perl.appender.Screen = Log::Dispatch::Screen log4perl.appender.Screen.Filter = MyBoolean log4perl.appender.Screen.layout = Log::Log4perl::Layout::SimpleLayout "Log::Log4perl::Filter::Boolean"'s boolean expressions allow for combining different appenders by name using AND (&& or &), OR (|| or |) and NOT (!) as logical expressions. Parentheses are used for grouping. Precedence follows standard Perl. Here's a bunch of examples: Match1 && !Match2 # Match1 and not Match2 !(Match1 || Match2) # Neither Match1 nor Match2 (Match1 && Match2) || Match3 # Both Match1 and Match2 or Match3 SEE ALSO
Log::Log4perl::Filter, Log::Log4perl::Filter::LevelMatch, Log::Log4perl::Filter::LevelRange, Log::Log4perl::Filter::StringRange COPYRIGHT AND LICENSE
Copyright 2002-2009 by Mike Schilli <m@perlmeister.com> and Kevin Goess <cpan@goess.org>. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2010-07-21 Filter::Boolean(3pm)
All times are GMT -4. The time now is 12:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy