Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help with joining files and adding headers to files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help with joining files and adding headers to files
# 1  
Old 05-22-2012
Help with joining files and adding headers to files
Hi,

I have about 20 tab delimited text files that have non sequential numbering such as:

UCD2.summary.txt
UCD45.summary.txt
UCD56.summery.txt

The first column of each file has the same number of lines and content. The next 2 column have data points:

i.e UCD2.summary.txt:

Code:
a   8.9   9.6
b   5.6   68
c   8.5   52

UCD45.summary.txt:

Code:
a   4.2   8.5
b   5.5   56
c   5.6   12


There are no headers for these files. I would like to join all these files together since the first column has the same data. However I need to be able to tell which file each value came from so I need to add headers.

The output file would look like this with header files:

Code:
probeID   UCD2-value1   UCD2-value2  UCD45-value1 UCD45-value2
a                  8.9               9.6                4.2               8.5
b                  5.6                68                5.5               56
c                  8.5                52                5.6               12


I am very new to linux and perl and would love some help accomplishing the output above. Thanks!

Ryan

Moderator's Comments:
Mod Comment Code tags for code, please.
Last edited by Corona688; 05-22-2012 at 02:10 PM..
# 2  
Old 05-22-2012
Code:
$ echo header > filename
$ join file1 file2 >> filename
$ cat filename

header
a 8.9 9.6 4.2 8.5
b 5.6 68 5.5 56
c 8.5 52 5.6 12

$

# 3  
Old 05-22-2012
Thanks for the quick reply.
However this won't work because I don't have a header for each of the columns. I need to know for each column where the data came from.

Thanks!
# 4  
Old 05-22-2012
You'll have to get that information from somewhere, and nothing in your post suggests where it does come from, so I think we need more information.
# 5  
Old 05-22-2012
Sorry for not being clear.

I would like the output to be as so:

Code:
probe id   "FILENAMEA-info1"  "FILENAMEA-info2"  "FILENAMEB-info1"  "FILENAMEB-info2"
a                value                    value                  value
b                value                    value                  value
c                value                    value                  value

The first column would have the hearder "probeID"
2nd colum would have the filename+info as a header
3r column would have the filename+info as a header
and etc...

Does that make sense?
Thanks
# 6  
Old 05-22-2012
Yes, I see what you want now, sorry for being dense.

Working on something.
# 7  
Old 05-22-2012
Code:
$ cat jn.awk

BEGIN { OFS="\t";       }

F!=FILENAME {
        F=FILENAME;

        for(N=1; N<=NF; N++)    COL=COL OFS FILENAME"-info"N;
}

{
        D[$1]=D[$1] " " $0;
        if(!($1 in O))
        {
                O[++ORDER]=$1;
                O[$1]=1
        }
}

END {
        print substr(COL,2);
        for(N=1; N<=ORDER; N++)
        {
                $0=substr(D[O[N]], 2);
                $1=$1;
                print;
        }
}

$ awk -f jn.awk data1 data2

data1-info1     data1-info2     data1-info3     data2-info1     data2-info2    data2-info3
a       8.9     9.6     a       4.2     8.5
b       5.6     68      b       5.5     56
c       8.5     52      c       5.6     12

$

Perhaps not the most efficient but an all-in-one solution.
This User Gave Thanks to Corona688 For This Post:
rrdavis
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining two files by retaining the headers.

Hi Experts, I want to join two files(file1 ,file2) which are having tab separated values,by sorting them on key column(ID) and want to redirect the output to other file(output file) along with the headers from both the files. cat file1: ID Name phone_no 205 mno 90808 200 xyz 32003... (9 Replies)
Discussion started by: bharathbangalor
9 Replies

2. Shell Programming and Scripting

Joining 2 Files

File "A" (column names: Nickname Number GB) Nickname Number GB PROD_DB0034 100A 16 ASMIL1B_DATA_003 100B 16 PSPROD_0000 1014 36 PSPROD_0001 100D 223 ..... File "B" (column names: TYPE DEVICE NUMBER SIZE) TYPE DEVICE NUMBER SIZE 1750500 hdisk2 100A 16384 1750500 hdisk3 ... (4 Replies)
Discussion started by: Daniel Gate
4 Replies

3. Shell Programming and Scripting

Joining two files into one

Hi experts, I'm quite newbie here!! I have two seperate files. Contents of file like below File 1: 6213019212001 8063737 File:2 15703784 I want to join these two files into one where content will be File 3: 6213019212001 8063737 15703784 Regards, Ray Seilden (1 Reply)
Discussion started by: RayanS
1 Replies

4. Shell Programming and Scripting

Joining Three Files

Hi guys, I have three files which needs to be joined to a single file. File 1: Col a, Col b, Col c File 2: Col 1a, Col 1b File 3: Col 2a, Col 2b Output: Col 1a, Col 2a, Col a, Col b, Col c. All the files are comma delimited. I need to join Col b with Col 1b and need to... (17 Replies)
Discussion started by: mac4rfree
17 Replies

5. Shell Programming and Scripting

renaming files or adding a name in the beginning of all files in a folder

Hi All I have a folder that contains hundreds of file with a names 3.msa 4.msa 21.msa 6.msa 345.msa 456.msa 98.msa ... ... ... I need rename each of this file by adding "core_" in the begiining of each file such as core_3.msa core_4.msa core_21.msa (4 Replies)
Discussion started by: Lucky Ali
4 Replies

6. Shell Programming and Scripting

joining two or more files

i have three files file a has contents 123 234 238 file b has contents 189 567 567 and file c has contents qwe ert ery (1 Reply)
Discussion started by: tomjones
1 Replies

7. Shell Programming and Scripting

Merging of files with different headers to make combined headers file

Hi , I have a typical situation. I have 4 files and with different headers (number of headers is varible ). I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only). For example - File 1 H1|H2|H3|H4 11|12|13|14 21|22|23|23... (1 Reply)
Discussion started by: marut_ashu
1 Replies

8. Shell Programming and Scripting

Joining files

Hi, Whats the unix function to join multiple files? is it cat? so I have multiple files in the same format and I want to join then by row eg. FILE1 1 3 1 3 1 3 1 3 FILE2 2 4 2 4 2 4 (1 Reply)
Discussion started by: kylle345
1 Replies

9. Shell Programming and Scripting

Help with joining two files

Greetings, all. I've got a project that requires I join two data files together, then do some processing and output. Everything must be done in a shell script, using standard unix tools. The files look like the following: File_1 Layout: Acct#,Subacct#,Descrip Sample: ... (3 Replies)
Discussion started by: rjlohman
3 Replies

10. UNIX for Dummies Questions & Answers

joining files

Hi, Could anyone help me ? I'm trying to join two files, but no common field are on them. So I think on generate \000\ sequence to add for each line on both files, so then will be able to join these files. Any idea? Thanks in advance, (2 Replies)
Discussion started by: Manu
2 Replies
Login or Register to Ask a Question