Need script to merge two spreadsheets


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Need script to merge two spreadsheets
# 1  
Old 05-02-2012
Need script to merge two spreadsheets

Hello all, I am very new to scripting and I have a challenging problem I would like to write a code for. Essentially, I would like to merge two tab-delimited spreadsheets that have identical column and row labels, one contains numbers, the other contains labels (text and numbers):

Spreadsheet 1
PHP Code:
gene  W  X  Y  Z
I     1  2  3  4
II    5  6  7  8
III   9 10 11 12
IV   13 14 15 16 
Spreadsheet 2
PHP Code:
gene W   X   Y   Z
I    d1          
II          a3   
III  b1
IV              c1 
Merged Spreadsheet
PHP Code:
gene W   X   Y   Z
I    1   2   3   4
     d1
II   5   6   7   8
             a3
III  9   10  11  12
     b1
IV   13  14  15  16
                 c1 
*Preferably, the column labels (W,X,Y,Z) would not have to be in the same order in both spreadsheets.

Any help appreciated!!
Thanks
# 2  
Old 05-02-2012
Are these spreadsheets tab-separated, or what?

Are they truly labelled with roman numerals?
# 3  
Old 05-02-2012
Yes they are tab-delimited columns although I can work with comma delimited aswell. All labels and data points would be a combination of both text and numbers. ie. Sample1 Sample2 Sample3 for the column labels, and DFAGF4, 345RGT, gfg234, for the row labels. Everything would be treated as a string I'm assuming.
# 4  
Old 05-02-2012
Okay, sounds reasonable enough. Working on something.
# 5  
Old 05-02-2012
Code:
$ cat merge.awk

BEGIN { FS="\t"; OFS="\t" }

NR==1 {
        print
        MAX=NF
        for(N=1; N<=NF; N++)    ORDER[$N]=N
        next
}

NR==FNR { D[$1]=$0; next }

# Pick up order of fields from second file
FNR==1 {
        for(N=1; N<=NF; N++)    REORDER[N]=ORDER[$N];
        next
}

$1 in D {
        print D[$1]
        for(N=2; N<=MAX; N++)
                printf("%s%s", OFS, $(REORDER[N]));
        printf("\n");
}

$ cat file1

gene    W       X       Y       Z
I       1       2       3       4
II      5       6       7       8
III     9       10      11      12
IV      13      14      15      16

# Note the altered column order
$ cat file2

gene    W       Y       X       Z
I       d1
II              a3
III     b1
IV                              c1

$ awk -f merge.awk file1 file2

gene    W       X       Y       Z
I       1       2       3       4
        d1
II      5       6       7       8
                        a3
III     9       10      11      12
        b1
IV      13      14      15      16
                                c1

$

# 6  
Old 05-02-2012
Damn you work fast, they must be paying you well. There seems to be a bug, not quite sure how to explain it, but I've attached the two inputs and the output I got. Perhaps my example was too simple, I have 27 columns and 711 rows in my file.

Look forward to hearing back!

Thanks. Smilie
# 7  
Old 05-03-2012
try this

Code:
 
 paste -d "|" file1 file2 | sed 's/|/\n/g' | tail -n +2

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script to merge and delete lines

POLY_STATS { EqName 103_tri Id 123 act_polyCount 1 act_polyValue 0 } POLY_STATS { EqName 103_tri Id 123 pass_polyCount 2 pass_polyValue 0 } POLY_STATS { EqName 103 Id 123 mes_polyCount 2 mes_polyValue 0 (5 Replies)
Discussion started by: Jag02
5 Replies

2. Shell Programming and Scripting

How to merge Expect script inside shell script?

Hi I have two scripts one is Expect and other is shell. I want to merge Expect code in to Shell script so that i can run it using only one script. Can somebody help me out ? Order to execute: Run Expect_install.sh first and then when installation completes run runTests.sh shell script. ... (1 Reply)
Discussion started by: ashish_neekhra
1 Replies

3. Shell Programming and Scripting

script to merge two files on an index

I have a need to merge two files on the value of an index column. input file 1 id filePath MDL_NUMBER 1 MFCD00008104.mol MFCD00008104 2 MFCD00012849.mol MFCD00012849 3 MFCD00037597.mol MFCD00037597 4 MFCD00064558.mol MFCD00064558 5 MFCD00064559.mol MFCD00064559 input file 2 ... (9 Replies)
Discussion started by: LMHmedchem
9 Replies

4. Shell Programming and Scripting

Help with merge Shell Script

I have a file test.log The content of the file is : a:R_yz:1 a:R_cd:2 a:F_bc:0 a:F_xx:3 b:R_dg:5 b:R_gf:1 b:F_fd:4 I want the output is : :a R_yz 1 3 3 : R_cd 2 : F_bc 0 : F_xx 3 :b R_dg 5 6 4 : R_gf 1 : F_fd 4 (8 Replies)
Discussion started by: mnmonu
8 Replies

5. Shell Programming and Scripting

script to merge multiple line

Dear all, I am new to this community. I need a script that will merge two lines of a text file based on some character. for example- Input asfdas fas sdfhksd sdf ss 45 gf# gjsdh fhjsd yiu oiuio ioioiuii sdgjdshsdg sdhfjk sdfhsd sdf sdf sdf wer wer we# qqwewq qwe qwe wer# fsdf sdf... (8 Replies)
Discussion started by: wildhorse
8 Replies

6. Shell Programming and Scripting

merge two files via looping script

Hi all, I hope you can help me. I got a file a and a file b File a contains a b c d e f g h File b contains 1 2 3 (8 Replies)
Discussion started by: stinkefisch
8 Replies

7. Shell Programming and Scripting

script to merge xml files with options

Hi, I have a very basic knowledge of shell scripting & would like some help with a little problem I have. I sometimes use a program calle phronix & sometimes like to compare its results which are *.xml files. Which is easy enough but a friend wants to avoid typing the path to the files.... (2 Replies)
Discussion started by: ptrbee
2 Replies

8. Shell Programming and Scripting

script needed to merge two files and report differences

Hello, I have two txt files that look like this: db.0.0.0.0: Total number of NS records = 1 db.127.0.0.0: Total number of NS records = 1 Total number of PTR records = 1 db.172.19.0.0: Total number of NS records = 1 Total number of PTR records = 3 db.172.19.59.0: Total... (8 Replies)
Discussion started by: richsark
8 Replies

9. Shell Programming and Scripting

Merge two files in windows using perl script

Hi I want to merge two or more files using perl in windows only(Just like Paste command in Unix script) . How can i do this.Is ther any single command to do this? Thanks Kunal (1 Reply)
Discussion started by: kunal_dixit
1 Replies

10. Shell Programming and Scripting

shell script to merge files

Can anybody help me out with this problem " a shell program that takes one or any number of file names as input; sorts the lines of each file in ascending order and displays the non blank lines of each sorted file and merge them as one combined sorted file. The program generates an error... (1 Reply)
Discussion started by: arya
1 Replies
Login or Register to Ask a Question