Merging two files in UNIX Post: 302875963

Sponsored Content

Top Forums Shell Programming and Scripting Merging two files in UNIX Post 302875963 by Akshay Hegde on Thursday 21st of November 2013 07:02:35 AM

11-21-2013

Moderator

Quote:

Originally Posted by bharathbangalor

Hi Experts,
Need urgent solution for a problem.

I have two files file1 and file2. file1 is tab separated and file2 is comma separated.
I need to merge both the files into single file based on CUST_ID by retaining the headers of file1

Matching CUST_IDs should be placed one below the other in the output file.If any correspnding column is missing in file 2 it has to be replaced with 'NA';

Code:

cat file1:(tab separated)

CUST_ID    Name    Place    acc_no
1    abc    blr|chn|dlh    500|600|700
2    xyz    hyd|blr    888|999
3    mno    blr|hyd    111|222|333


cat file2:(coma separated)
CUST_ID,Name,acc_no
1,abc,500|600|700
2,xyz,111|222|333


Required output:

CUST_ID,Name,Place,acc_no
1,abc,blr|chn|dlh,500|600|700
1,abc,NA,500|600|700
2,xyz,hyd|blr,888|999
2,xyz,NA,111|222|333

Try:

Code:

$ awk -F'[ ,]' 'FNR==NR{gsub(/[[:space:]]/,",");A[$2]=$0;next}A[$2]{$3="NA,"$3;$0=FNR==1 ? A[$2]: A[$2] RS $0}A[$2]'  OFS=\, file1 file2

Resulting

Code:

CUST_ID,Name,Place,acc_no
1,abc,blr|chn|dlh,500|600|700
1,abc,NA,500|600|700
2,xyz,hyd|blr,888|999
2,xyz,NA,111|222|333

Last edited by Akshay Hegde; 11-21-2013 at 08:49 AM.. Reason: missed OFS

Akshay Hegde

View Public Profile for Akshay Hegde

Find all posts by Akshay Hegde

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Merging 2 .CSV files in Unix

I need a little help as I am a complete novice at scripting in unix. However, i am posed with an issue...:eek: i have two csv files in the following format@ FILE1.CSV: HEADER HEADER Header , , HEADER 001X ,,200 002X ,,300 003X ...

2. Shell Programming and Scripting

merging files

Thanks in advance I have 2 files having key field in each.I would like to join both on common key.I have used join but not sucessful. The files are attached here . what i Want in the output is on the key field SLS OFFR . I have used join commd but not successful. File one ======= SNO ...

3. UNIX for Dummies Questions & Answers

Merging files

Hi i have two files say file 1 contents are A B C D E I have file2 contents are B E F G C K I want to have new file like A B

4. Shell Programming and Scripting

merging two files

Friends, os: redhat enterprise linux/SCO UNIX5.0 I have two files and I would like to merge on given key value. Now I have tried with join commd but it does not supporte multiple delimiters. and if records length is not fixed. join -a1 5 -a2 1 -t -o file1 file2 > outname Can any...

5. Shell Programming and Scripting

Merging 2 files

Hi, I have got two files 1.txt 1111|apple| 2222|orange| 2.txt 1111|1234|000000000004356| 1111|1234|000000001111| 1111|1234|002000011112| 2222|5678|000000002222| 2222|9102|000000002222| I need to merge these two so that my out put looks like below: Search code being used should be...

6. Shell Programming and Scripting

Help with merging files

i would like to merge two files that have the same format but have different data. i would like to create one output file that contains information from both the original files.:rolleyes:

7. UNIX for Dummies Questions & Answers

Merging two files

Hi, I have two files a.txt and b.txt. a.txt 1 2 3 4 b.txt a b c d e I want to generate a file c.txt by merging these two file and the resultant file would contain c.txt 1

8. Shell Programming and Scripting

Help with merging 2 files into 1

::::::::: ::FileA:: ::::::::: A1-------A2--------A3---A4---A5-- ================================= AC5VXVLT-XX---------------------- B57E434--XXXX1-----MMMM-ZZZ--111- C325G20--XXXXX3----CCCC------3332 DC35S51--XXXXY1----DDDD------44X- DC35S52--XXXXY2----DDDD------44Y-...

9. Shell Programming and Scripting

Merging Very large CSV files in Unix

Hi, I have two very large CSV files, which I want to merge (equi-join) based on a key (column). One of the file (say F1) would have ~30 MM records and 700 columns. The other file (~f2) would have same # of records and lesser columns (say 50). I want to create an output file joining on a...

10. Shell Programming and Scripting

Merging Files in UNIX shell script

I have the urge to merge some files using unix shell script but I'm very new using this language and I haven't succeeded yet. The requirement is to merge the header, body and footer into one file with the name "ANY-NAME" in below example. To identify which files should be merged, I have flagged...

LEARN ABOUT REDHAT

merge

MERGE(1)						      General Commands Manual							  MERGE(1)

NAME

       merge - three-way file merge

SYNOPSIS

       merge [ options ] file1 file2 file3

DESCRIPTION

       merge  incorporates all changes that lead from file2 to file3 into file1.  The result ordinarily goes into file1.  merge is useful for com-
       bining separate changes to an original.	Suppose file2 is the original, and both file1 and file3 are modifications of  file2.   Then  merge
       combines both changes.

       A  conflict  occurs  if	both  file1 and file3 have changes in a common segment of lines.  If a conflict is found, merge normally outputs a
       warning and brackets the conflict with <<<<<<< and >>>>>>> lines.  A typical conflict will look like this:

	      <<<<<<< file A
	      lines in file A
	      =======
	      lines in file B
	      >>>>>>> file B

       If there are conflicts, the user should edit the result and delete one of the alternatives.

OPTIONS

       -A     Output conflicts using the -A style of diff3(1), if supported by diff3.  This merges all changes leading from file2  to  file3  into
	      file1, and generates the most verbose output.

       -E, -e These  options  specify conflict styles that generate less information than -A.  See diff3(1) for details.  The default is -E.  With
	      -e, merge does not warn about conflicts.

       -L label
	      This option may be given up to three times, and specifies labels to be used in place of the corresponding  file  names  in  conflict
	      reports.	 That  is, merge -L x -L y -L z a b c generates output that looks like it came from files x, y and z instead of from files
	      a, b and c.

       -p     Send results to standard output instead of overwriting file1.

       -q     Quiet; do not warn about conflicts.  -V Print 's version number.

DIAGNOSTICS

       Exit status is 0 for no conflicts, 1 for some conflicts, 2 for trouble.

IDENTIFICATION

       Author: Walter F. Tichy.
       Manual Page Revision: 5.7; Release Date: 1995/06/01.
       Copyright (C) 1982, 1988, 1989 Walter F. Tichy.
       Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995 Paul Eggert.

SEE ALSO

       diff3(1), diff(1), rcsmerge(1), co(1).

BUGS

       It normally does not make sense to merge binary files as if they were text, but merge tries to do it anyway.

GNU
								    1995/06/01								  MERGE(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Merging 2 .CSV files in Unix

Discussion started by: chachabronson

2. Shell Programming and Scripting

merging files

Discussion started by: vakharia Mahesh

3. UNIX for Dummies Questions & Answers

Merging files

Discussion started by: ssuresh1999

4. Shell Programming and Scripting

merging two files

Discussion started by: vakharia Mahesh

5. Shell Programming and Scripting

Merging 2 files

Discussion started by: jisha

6. Shell Programming and Scripting

Help with merging files

Discussion started by: joe black

7. UNIX for Dummies Questions & Answers

Merging two files

Discussion started by: siba.s.nayak

8. Shell Programming and Scripting

Help with merging 2 files into 1

Discussion started by: lordsmiter

9. Shell Programming and Scripting

Merging Very large CSV files in Unix

Discussion started by: student_007

10. Shell Programming and Scripting

Merging Files in UNIX shell script

Discussion started by: black_soul

LEARN ABOUT REDHAT

merge