Visit Our UNIX and Linux User Community


Merging of files with different headers to make combined headers file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merging of files with different headers to make combined headers file
# 1  
Old 08-06-2009
Lightbulb Merging of files with different headers to make combined headers file

Hi ,
I have a typical situation. I have 4 files and with different headers (number of headers is varible ).
I need to make such a merged file which will have headers combined from all files (comman coluns should appear once only).

For example -
File 1
H1|H2|H3|H4
11|12|13|14
21|22|23|23

File 2

H3|H5|H6
31|32|33
41|42|43

Merged file of these two will be -

H1|H2|H3|H4|H5|H6
11|12|13|14|NA||NA
21|22|23|23|NA|NA
NA|NA|31|NA|32|33
NA|NA|41|NA|42|43

Please help me for the same.
# 2  
Old 08-07-2009
if already know the maximum and sequence of headers, should below easy perl script help you some.

If not, may need extra efforts to make out the header hash first and then use below perl script.


Code:
my %headers=(H1=>1,H2=>2,H3=>3,H4=>4,H5=>5,H6=>6);
my @header_arr = sort {$headers{$a}<=>$headers{$b}} keys %headers;
print join "|", @header_arr;
print "\n";
my %missed;
my $max=$#header_arr;
while(<DATA>){
	chomp;
	my @tmp=split("[|]",$_);
	my %hash=%headers;
	if($.==1){
		map {delete $hash{$_}} @tmp;
		map {$tt{$_}=1} values %hash;
		next;
	}
	else{
		for(my $i=1;$i<=$max+1;$i++){
			if(exists $tt{$i}){
				print "NA|";
			}
			else{
				my $tmp=shift @tmp;
				print $tmp,"|";
			}
		}
	}
	print "\n";
}
__DATA__
H1|H3|H6
12|31|33
23|41|43


Previous Thread | Next Thread
Test Your Knowledge in Computers #487
Difficulty: Easy
A program's entry point is always defined by a main function or method.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to compare files and validate order of headers

The below awk verifies the count and order of each text file in the directory. The script does execute and produce output, however the order of the headers are not compared to key. The portion in bold is supposed to do that. If the order of the headers in each text file is the same as key, then... (0 Replies)
Discussion started by: cmccabe
0 Replies

2. Shell Programming and Scripting

Row bind multiple csv files having different column headers

All, I guess by this time someone asked this kind of question, but sorry I am unable to find after a deep search. Here is my request I have many files out of which 2 sample files provided below. File-1 (with A,B as column headers) A,B 1,2 File-2 (with C, D as column headers) C,D 4,5 I... (7 Replies)
Discussion started by: ks_reddy
7 Replies

3. Shell Programming and Scripting

Joining two files by retaining the headers.

Hi Experts, I want to join two files(file1 ,file2) which are having tab separated values,by sorting them on key column(ID) and want to redirect the output to other file(output file) along with the headers from both the files. cat file1: ID Name phone_no 205 mno 90808 200 xyz 32003... (9 Replies)
Discussion started by: bharathbangalor
9 Replies

4. Shell Programming and Scripting

Merging File with headers

Hi I need to merge 4 files. The issue i am facing is all the files have headers and i do not want them in the final output file. Can anybody suggest how to do it? (5 Replies)
Discussion started by: Arun Mishra
5 Replies

5. Shell Programming and Scripting

Faster command to remove headers for files in a directory

Good evening Im new at unix shell scripting and im planning to script a shell that removes headers for about 120 files in a directory and each file contains about 200000 lines in average. i know i will loop files to process each one and ive found in this great forum different solutions... (5 Replies)
Discussion started by: alexcol
5 Replies

6. Shell Programming and Scripting

Editing File Headers

Hey Guys, Absolute neewbie here. I am trying to see if it is possible to edit headers/meta-data of files in Mac OSX. I am basically trying to change an audio file header to read 16bit instead of 24bit. We have an issue with some of our software and it regularly exports 16bit audio files with... (3 Replies)
Discussion started by: andysuperaudiom
3 Replies

7. Shell Programming and Scripting

Multiple headers in a file

Hi , I have a .txt file in which I have multiple headers, the header record starts with $ symbol...like the first column name is $Account. I have to keep the header in the first line and delete all the remaining headers which are in the file. I tried using sort adc.txt | uniq -u , but my... (7 Replies)
Discussion started by: gaur.deepti
7 Replies

8. UNIX for Dummies Questions & Answers

reading file headers

Hello, I have done much googling on this, but apparently not using the right keywords. I am assuming there is some kind of header for each file on a disk which stores information such as mod time, access time, etc. I have two questions: 1) is there a way to read this header directly,... (2 Replies)
Discussion started by: Allasso
2 Replies

9. Shell Programming and Scripting

Remove text between headers while leaving headers intact

Hi, I'm trying to strip all lines between two headers in a file: ### BEGIN ### Text to remove, contains all kinds of characters ... Antispyware-Downloadserver.com (Germany)=http://www.antispyware-downloadserver.c om/updates/ Antispyware-Downloadserver.com #2... (3 Replies)
Discussion started by: Trones
3 Replies

10. UNIX for Dummies Questions & Answers

help:how to remove headers in output file

Hi I am running a script (which compares two directory contents) for which I am getting an output of 70 pages in which few pages are blank so I was able to delete those blank lines. But I also want to delete the headers present for each page. can any one help me by providing the code... (1 Reply)
Discussion started by: raj_thota
1 Replies

Featured Tech Videos