Sponsored Content
Top Forums Shell Programming and Scripting merge two text files of different size on common index Post 302518736 by LMHmedchem on Sunday 1st of May 2011 03:59:21 PM
Old 05-01-2011
I found a bug in my data where the first two files had a different header name for one header. The header row is now correct, more or less.

There still seems to be an issue in that the last column has three columns of space delimited data in it.
Code:
sumSO2Am                 SUB_ID     SOURCE                                          
0                 10584046   ChemBridge                                      
0                 10580948   ChemBridge                                      
0                 10580812   ChemBridge                                      
0                 10580337   ChemBridge                                      
0                 10579979   ChemBridge                                      
0                 10579233   ChemBridge

The last two, SUB_ID and SOURCE are duplicate cols (already occur at $3,$4). These come from $3, $4 in a2. Each row should end with the sumSO2Am field.

I don't see where that is happening in the command, or I just don't get it. I see how the first 7 fields are being printed, but not the rest of each row. I can post some short test files if that would help.

LMHmedchem
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

2. Shell Programming and Scripting

How to remove common file names from text files

I'm running on freebsd -- with a default shell of csh. I have two files named A and B. Each line of each file contains a file name. How can I write a script that removes all the file names in file B from A. I tried to use perl to create a huge regular expression with "|" separating the file... (2 Replies)
Discussion started by: siegfried
2 Replies

3. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should... (1 Reply)
Discussion started by: evelibertine
1 Replies

4. UNIX for Dummies Questions & Answers

Merge two files with common IDs but unequal number of rows

Hi, I have two files that I would like to merge and think that there should be a solution using awk. The files look something like this: file 1 IDX1 IDY1 IDX2 IDY2 IDX3 IDY3 file 2 IDY1 dataA data1 IDY2 dataB data2 IDY3 dataC data3 Desired output IDX1 IDY1 dataA data1 IDX2 ... (5 Replies)
Discussion started by: katie8856
5 Replies

5. Shell Programming and Scripting

script to merge two files on an index

I have a need to merge two files on the value of an index column. input file 1 id filePath MDL_NUMBER 1 MFCD00008104.mol MFCD00008104 2 MFCD00012849.mol MFCD00012849 3 MFCD00037597.mol MFCD00037597 4 MFCD00064558.mol MFCD00064558 5 MFCD00064559.mol MFCD00064559 input file 2 ... (9 Replies)
Discussion started by: LMHmedchem
9 Replies

6. Shell Programming and Scripting

Merge files based on both common and uncommon rows

Hi, I have two files A (2190 rows) and file B (1100 rows). I want to merge the contents of two files based on common field, also I need the unmatched rows from file A file A: ABC XYZ PQR file B: >LMN|chr1:11000-12456: >ABC|chr15:176578-187678: >PQR|chr3:14567-15866: output... (3 Replies)
Discussion started by: Diya123
3 Replies

7. Shell Programming and Scripting

Find matched patterns in a column of 2 files with different size and merge them

Hi, i have input files like below:- input1 Name Seq_ID NewID Scores MT1 A0QZX3 1.65 277.4 IVO A0QZX3 1.65 244.5 HPO A0QZX3 1.65 240.5 RgP A0Q3PP 5.32 241.0 GX1 LPSZ3S 96.1 216.9 MEL LPSS3X 4.23 204.1 LDD LPSS3X 4.23 100.2 input2 Fac AddName NewID ... (9 Replies)
Discussion started by: redse171
9 Replies

8. Shell Programming and Scripting

Merge multiple files with common header

Hi all, Say i have multiple files x1 x2 x3 x4, all with common header (date, time, year, age),, How can I merge them to one singe file "X" in shell scripting Thanks for your suggestions. (2 Replies)
Discussion started by: msarguru
2 Replies

9. UNIX for Dummies Questions & Answers

Merge selective columns from files based on common key

Hi, I am trying to selectively merge two files based on keys reported in the 1st column. File1: #file1-header1 file1-header2 111 qwe rtz uio 198 asd fgh jkl 165 yxc 789 poi uzt rew 89 lkj File2: #file2-header2 file2-header2 165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies

10. Shell Programming and Scripting

Merge multiple tab delimited files with index checking

Hello, I have 40 data files where the first three columns are the same (in theory) and the 4th column is different. Here is an example of three files, file 2: A_f0_r179_pred.txt Id Group Name E0 1 V N(,)'1 0.2904 2 V N(,)'2 0.3180 3 V N(,)'3 0.3277 4 V N(,)'4 0.3675 5 V N(,)'5 0.3456 ... (8 Replies)
Discussion started by: LMHmedchem
8 Replies
Template::Plugin::Table(3)				User Contributed Perl Documentation				Template::Plugin::Table(3)

NAME
Template::Plugin::Table - Plugin to present data in a table SYNOPSIS
[% USE table(list, rows=n, cols=n, overlap=n, pad=0) %] [% FOREACH item IN table.row(n) %] [% item %] [% END %] [% FOREACH item IN table.col(n) %] [% item %] [% END %] [% FOREACH row IN table.rows %] [% FOREACH item IN row %] [% item %] [% END %] [% END %] [% FOREACH col IN table.cols %] [% col.first %] - [% col.last %] ([% col.size %] entries) [% END %] DESCRIPTION
The "Table" plugin allows you to format a list of data items into a virtual table. When you create a "Table" plugin via the "USE" directive, simply pass a list reference as the first parameter and then specify a fixed number of rows or columns. [% USE Table(list, rows=5) %] [% USE table(list, cols=5) %] The "Table" plugin name can also be specified in lower case as shown in the second example above. You can also specify an alternative variable name for the plugin as per regular Template Toolkit syntax. [% USE mydata = table(list, rows=5) %] The plugin then presents a table based view on the data set. The data isn't actually reorganised in any way but is available via the "row()", "col()", "rows()" and "cols()" as if formatted into a simple two dimensional table of "n" rows x "n" columns. So if we had a sample "alphabet" list contained the letters '"a"' to '"z"', the above "USE" directives would create plugins that represented the following views of the alphabet. [% USE table(alphabet, ... %] rows=5 cols=5 a f k p u z a g m s y b g l q v b h n t z c h m r w c i o u d i n s x d j p v e j o t y e k q w f l r x We can request a particular row or column using the "row()" and "col()" methods. [% USE table(alphabet, rows=5) %] [% FOREACH item = table.row(0) %] # [% item %] set to each of [ a f k p u z ] in turn [% END %] [% FOREACH item = table.col(2) %] # [% item %] set to each of [ m n o p q r ] in turn [% END %] Data in rows is returned from left to right, columns from top to bottom. The first row/column is 0. By default, rows or columns that contain empty values will be padded with the undefined value to fill it to the same size as all other rows or columns. For example, the last row (row 4) in the first example would contain the values "[ e j o t y undef ]". The Template Toolkit will safely accept these undefined values and print a empty string. You can also use the IF directive to test if the value is set. [% FOREACH item = table.row(4) %] [% IF item %] Item: [% item %] [% END %] [% END %] You can explicitly disable the "pad" option when creating the plugin to returned shortened rows/columns where the data is empty. [% USE table(alphabet, cols=5, pad=0) %] [% FOREACH item = table.col(4) %] # [% item %] set to each of 'y z' [% END %] The "rows()" method returns all rows/columns in the table as a reference to a list of rows (themselves list references). The "row()" methods when called without any arguments calls "rows()" to return all rows in the table. Ditto for "cols()" and "col()". [% USE table(alphabet, cols=5) %] [% FOREACH row = table.rows %] [% FOREACH item = row %] [% item %] [% END %] [% END %] The Template Toolkit provides the "first", "last" and "size" virtual methods that can be called on list references to return the first/last entry or the number of entries in a list. The following example shows how we might use this to provide an alphabetical index split into 3 even parts. [% USE table(alphabet, cols=3, pad=0) %] [% FOREACH group = table.col %] [ [% group.first %] - [% group.last %] ([% group.size %] letters) ] [% END %] This produces the following output: [ a - i (9 letters) ] [ j - r (9 letters) ] [ s - z (8 letters) ] We can also use the general purpose "join" virtual method which joins the items of the list using the connecting string specified. [% USE table(alphabet, cols=5) %] [% FOREACH row = table.rows %] [% row.join(' - ') %] [% END %] Data in the table is ordered downwards rather than across but can easily be transformed on output. For example, to format our data in 5 columns with data ordered across rather than down, we specify "rows=5" to order the data as such: a f . . b g . c h d i e j and then iterate down through each column (a-e, f-j, etc.) printing the data across. a b c d e f g h i j . . . Example code to do so would be much like the following: [% USE table(alphabet, rows=3) %] [% FOREACH cols = table.cols %] [% FOREACH item = cols %] [% item %] [% END %] [% END %] Output: a b c d e f g h i j . . . In addition to a list reference, the "Table" plugin constructor may be passed a reference to a Template::Iterator object or subclass thereof. The Template::Iterator get_all() method is first called on the iterator to return all remaining items. These are then available via the usual Table interface. [% USE DBI(dsn,user,pass) -%] # query() returns an iterator [% results = DBI.query('SELECT * FROM alphabet ORDER BY letter') %] # pass into Table plugin [% USE table(results, rows=8 overlap=1 pad=0) -%] [% FOREACH row = table.cols -%] [% row.first.letter %] - [% row.last.letter %]: [% row.join(', ') %] [% END %] AUTHOR
Andy Wardley <abw@wardley.org> <http://wardley.org/> COPYRIGHT
Copyright (C) 1996-2007 Andy Wardley. All Rights Reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
Template::Plugin perl v5.16.3 2011-12-20 Template::Plugin::Table(3)
All times are GMT -4. The time now is 04:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy