Sponsored Content
Top Forums Web Development Perl join two files by "common" column Post 302494415 by z1dane on Monday 7th of February 2011 08:30:19 AM
Old 02-07-2011
Hi yifangt,

No problems. chomp by the way operates on the default input ($_), so you can just specify chomp instead of chomp $_ Smilie

The computation will take a while as you pointed out. I think it might be worth fixing up the first file so that everything is systematic e.g. having a standardised assembly notation, so you don't need to use a regular expression. Once that is fixed up, you can just use a hash to see if the key exists.

As for your second approach of deleting elements in the hash, look up the delete() function.

Good luck and happy coding!

Dave
This User Gave Thanks to z1dane For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

2. Shell Programming and Scripting

Join multiple files based on 1 common column

I have n files (for ex:64 files) with one similar column. Is it possible to combine them all based on that column ? file1 ax100 20 30 40 ax200 22 33 44 file2 ax100 10 20 40 ax200 12 13 44 file2 ax100 0 0 4 ax200 2 3 4 (9 Replies)
Discussion started by: quincyjones
9 Replies

3. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

4. UNIX for Dummies Questions & Answers

how to join two files using "Join" command with one common field in this problem?

file1: Toronto:12439755:1076359:July 1, 1867:6 Quebec City:7560592:1542056:July 1, 1867:5 Halifax:938134:55284:July 1, 1867:4 Fredericton:751400:72908:July 1, 1867:3 Winnipeg:1170300:647797:July 15, 1870:7 Victoria:4168123:944735:July 20, 1871:10 Charlottetown:137900:5660:July 1, 1873:2... (2 Replies)
Discussion started by: mindfreak
2 Replies

5. Shell Programming and Scripting

Substituting comma "," for dot "." in a specific column when comma"," is a delimiter

Hi, I'm dealing with an issue and losing a lot of hours figuring out how i would solve this. I have an input file which looks like this: ('BLABLA +200-GRS','Serviço ','TarifaçãoServiço','wap.bla.us.0000000121',2985,0,55,' de conversão em escada','Dia','Domingos') ('BLABLA +200-GRR','Serviço... (6 Replies)
Discussion started by: poliver
6 Replies

6. UNIX for Dummies Questions & Answers

How to use the the join command to join multiple files by a common column

Hi, I have 20 tab delimited text files that have a common column (column 1). The files are named GSM1.txt through GSM20.txt. Each file has 3 columns (2 other columns in addition to the first common column). I want to write a script to join the files by the first common column so that in the... (5 Replies)
Discussion started by: evelibertine
5 Replies

7. Shell Programming and Scripting

Problem of Perl's "join" function

$ perl -e '@f=("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","1","911"); print join("\t",@f)."\n";' aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa ... (5 Replies)
Discussion started by: carloszhang
5 Replies

8. UNIX for Dummies Questions & Answers

How to join 2 .txt files based on a common column?

Hi all, I'm trying to join two .txt file tab delimitated based on a common column. File 1 transcript_id gene_id length effective_length expected_count TPM FPKM IsoPct comp1000201_c0_seq1 comp1000201_c0 337 183.51 0.00 0.00 0.00 0.00 comp1000297_c0_seq1 ... (1 Reply)
Discussion started by: alisrpp
1 Replies

9. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test... (2 Replies)
Discussion started by: Hiroshi
2 Replies

10. Shell Programming and Scripting

Join, merge, fill NULL the void columns of multiples files like sql "LEFT JOIN" by using awk

Hello, This post is already here but want to do this with another way Merge multiples files with multiples duplicates keys by filling "NULL" the void columns for anothers joinning files file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv: 1|123|jojo 1|NULL|bibi... (2 Replies)
Discussion started by: yjacknewton
2 Replies
Perl6::Slurp(3pm)					User Contributed Perl Documentation					 Perl6::Slurp(3pm)

NAME
Perl6::Slurp - Implements the Perl 6 'slurp' built-in SYNOPSIS
use Perl6::Slurp; # Slurp a file by name... $file_contents = slurp 'filename'; $file_contents = slurp '<filename'; $file_contents = slurp '<', 'filename'; $file_contents = slurp '+<', 'filename'; # Slurp a file via an (already open!) handle... $file_contents = slurp *STDIN; $file_contents = slurp $filehandle; $file_contents = slurp IO::File->new('filename'); # Slurp a string... $str_contents = slurp $string; $str_contents = slurp '<', $string; # Slurp a pipe... $str_contents = slurp 'tail -20 $filename |'; $str_contents = slurp '-|', 'tail', -20, $filename; # Slurp with no source slurps from whatever $_ indicates... for (@files) { $contents .= slurp; } # ...or from the entire ARGV list, if $_ is undefined... $_ = undef; $ARGV_contents = slurp; # Specify I/O layers as part of mode... $file_contents = slurp '<:raw', $file; $file_contents = slurp '<:utf8', $file; $file_contents = slurp '<:raw :utf8', $file; # Specify I/O layers as separate options... $file_contents = slurp $file, {raw=>1}; $file_contents = slurp $file, {utf8=>1}; $file_contents = slurp $file, {raw=>1}, {utf8=>1}; $file_contents = slurp $file, [raw=>1, utf8=>1]; # Specify input record separator... $file_contents = slurp $file, {irs=>" "}; $file_contents = slurp '<', $file, {irs=>" "}; $file_contents = slurp {irs=>" "}, $file; # Input record separator can be regex... $file_contents = slurp $file, {irs=>qr/ +/}; $file_contents = slurp '<', $file, {irs=>qr/ +| {2,}}; # Specify autochomping... $file_contents = slurp $file, {chomp=>1}; $file_contents = slurp {chomp=>1}, $file; $file_contents = slurp $file, {chomp=>1, irs=>" "}; $file_contents = slurp $file, {chomp=>1, irs=>qr/ +/}; # Specify autochomping that replaces irs # with another string... $file_contents = slurp $file, {irs=>" ", chomp=>" "}; $file_contents = slurp $file, {chomp=>" "}, {irs=>qr/ +/}; # Specify autochomping that replaces # irs with a dynamically computed string... my $n = 1; $file_contents = slurp $file, {chomp=>sub{ " #line ".$n++." "}; # Slurp in a list context... @lines = slurp 'filename'; @lines = slurp $filehandle; @lines = slurp $string; @lines = slurp '<:utf8', 'filename', {irs=>"x{2020}", chomp=>" "}; DESCRIPTION
"slurp" takes: o a filename, o a filehandle, o a typeglob reference, o an IO::File object, or o a scalar reference, converts it to an input stream if necessary, and reads in the entire stream. If "slurp" fails to set up or read the stream, it throws an exception. If no data source is specified "slurp" uses the value of $_ as the source. If $_ is undefined, "slurp" uses the @ARGV list, and magically slurps the contents of all the sources listed in @ARGV. Note that the same magic is also applied if you explicitly slurp <*ARGV>, so the following three input operations: $contents = join "", <ARGV>; $contents = slurp *ARGV; $/ = undef; $contents = slurp; are identical in effect. In a scalar context "slurp" returns the stream contents as a single string. If the stream is at EOF, it returns an empty string. In a list context, it splits the contents after the appropriate input record separator and returns the resulting list of strings. You can set the input record separator ("{ irs => $your_irs_here}") for the input operation. The separator can be specified as a string or a regex. Note that an explicit input record separator has no effect in a scalar context, since "slurp" always reads in everything anyway. In a list context, changing the separator can change how the input is broken up within the list that is returned. If an input record separator is not explicitly specified, "slurp" defaults to " " (not to the current value of $/ X since Perl 6 doesn't have a $/); You can also tell "slurp" to automagically "chomp" the input as it is read in, by specifying: ("{ chomp => 1 }") Better still, you can tell "slurp" to automagically "chomp" the input and replace what it chomps with another string, by specifying: ("{ chomp => "another string" }") You can also tell "slurp" to compute the replacement string on-the-fly by specifying a subroutine as the "chomp" value: ("{ chomp => sub{...} }"). This subroutine is passed the string being chomped off, so for example you could squeeze single newlines to a single space and multiple conseqcutive newlines to a two newlines with: sub squeeze { my ($removed) = @_; if ($removed =~ tr/ / / == 1) { return " " } else { return " "; } } print slurp(*DATA, {irs=>qr/[ ]* +/, chomp=>&squeeze}), " "; Which would transform: This is the first paragraph This is the second paragraph This, the third This one is the very last to: This is the first paragraph This is the second paragraph This, the third This one is the very last Autochomping works in both scalar and list contexts. In scalar contexts every instance of the input record separator will be removed (or replaced) within the returned string. In list context, each list item returned with its terminating separator removed (or replaced). You can specify I/O layers, either using the Perl 5 notation: slurp "<:layer1 :layer2 :etc", $filename; or as an array of options: slurp $filename, [layer1=>1, layer2=>1, etc=>1]; slurp [layer1=>1, layer2=>1, etc=>1], $filename; or as individual options (each of which must be in a separate hash): slurp $filename, {layer1=>1}, {layer2=>1}, {etc=>1}; slurp {layer1=>1}, {layer2=>1}, {etc=>1}, $filename; (...which, of course, would look much cooler in Perl 6: # Perl 6 only :-( slurp $filename, :layer1 :layer2 :etc; slurp :layer1 :layer2 :etc, $filename; ) A common mistake is to put all the options together in one hash: slurp $filename, {layer1=>1, layer2=>1, etc=>1}; This is almost always a disaster, since the order of I/O layers is usually critical, and placing them all in one hash effectively randomizes that order. Use an array instead: slurp $filename, [layer1=>1, layer2=>1, etc=>1]; WARNING
The syntax and semantics of Perl 6 is still being finalized and consequently is at any time subject to change. That means the same caveat applies to this module. DEPENDENCIES
Requires: Perl 5.8.0, Perl6::Export AUTHOR
Damian Conway (damian@conway.org) COPYRIGHT
Copyright (c) 2003-2012, Damian Conway. All Rights Reserved. This module is free software. It may be used, redistributed and/or modified under the same terms as Perl itself. perl v5.14.2 2012-06-14 Perl6::Slurp(3pm)
All times are GMT -4. The time now is 02:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy