Concatenate 2 files data


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Concatenate 2 files data
# 1  
Old 10-01-2013
Concatenate 2 files data

Hi,

I have one file as:
Code:
$cat file1
abc  pqr  123
def  wxy  234
xyz  ghi   567

and another file as
Code:
$cat file2
345
456
987

I want the output of file2 to be appended in file1. And file 1 should look like:
Code:
$cat file1
abc  pqr  123  345
def  wxy  234  456
xyz  ghi   567  987

Please let me know how to move forward with this.
Thanks in advance.

Last edited by Don Cragun; 10-01-2013 at 07:34 AM.. Reason: Add CODE tags
# 2  
Old 10-01-2013
Code:
paste file1 file2

# 3  
Old 10-01-2013
Since you want two spaces between joined lines and want to overwrite one of your input files, try:
Code:
cp file1 tmp.$$ && paste -d " " tmp.$$ /dev/null file2 > file1 && rm tmp.$$

# 4  
Old 10-01-2013
Sir , Can you please help me to understand the command that you have suggested?
# 5  
Old 10-01-2013
Quote:
Originally Posted by Don Cragun
Since you want two spaces between joined lines and want to overwrite one of your input files, try:
Code:
cp file1 tmp.$$ && paste -d " " tmp.$$ /dev/null file2 > file1 && rm tmp.$$

1) Copy the contents of file1 to a file named tmp.$$ (note that this is Don Cragun's preference, you can change tmp.$$ to whatever name you want, myAux for example, see below), where $$ expands to the PID of your current shell (you can check this by running the following 2 commands and comparing their respective outputs echo $$ and ps aux | grep bash | grep -v grep)
Code:
gacanepa@debian:~$ echo $$
3428
gacanepa@debian:~$ ps aux | grep bash | grep -v grep
gacanepa  3428  0.0  1.3   6196  3380 pts/0    Ss   11:42   0:01 -bash
gacanepa@debian:~$

2) Merge files tmp.$$, /dev/null, and file2, line by line, and set the delimiter a space instead of a tab (that is what the -d option of the paste command is used for). The special file /dev/null is used here to add an extra space before the 4th column to match the same space separator as the other columns (which according to your example, seems to be 2 spaces).
3) Redirect the output to file1 and overwrite its contents.
4) Remove the auxiliary file tmp.$$ (or myAux, as seen below)
Without /dev/null:
Code:
gacanepa@debian:~/scripts/bash/tests$ cat file1
abc  pqr  123
def  wxy  234
xyz  ghi  567
gacanepa@debian:~/scripts/bash/tests$ cat file2
345
456
987
gacanepa@debian:~/scripts/bash/tests$ cp file1 myAux && paste -d " " myAux file2 > file1 && rm myAux
rm: remove regular file `myAux'? y
removed `myAux'
gacanepa@debian:~/scripts/bash/tests$ cat file1
abc  pqr  123 345
def  wxy  234 456
xyz  ghi  567 987
gacanepa@debian:~/scripts/bash/tests$

With /dev/null:
Code:
gacanepa@debian:~/scripts/bash/tests$ cat file1
abc  pqr  123
def  wxy  234
xyz  ghi  567
gacanepa@debian:~/scripts/bash/tests$ cat file2
345
456
987
gacanepa@debian:~/scripts/bash/tests$ cp file1 myAux && paste -d " " myAux /dev/null file2 > file1 && rm myAux
rm: remove regular file `myAux'? y
removed `myAux'
gacanepa@debian:~/scripts/bash/tests$ cat file1
abc  pqr  123  345
def  wxy  234  456
xyz  ghi  567  987
gacanepa@debian:~/scripts/bash/tests$

Hope it helped Smilie.

Last edited by gacanepa; 10-02-2013 at 09:29 AM..
# 6  
Old 10-02-2013
Quote:
Originally Posted by sidpatil
Sir , Can you please help me to understand the command that you have suggested?
Hi sidpatil,
I believe gacanepa already explained how the paste command portion of the AND list:
Code:
cp file1 tmp.$$ && paste -d " " tmp.$$ /dev/null file2 > file1 && rm tmp.$$

works.

If I had known the name of the script that Bhrigu was creating (assume scriptname for demonstration purposes), I would have chosen the name scriptname.$$ instead of tmp.$$ to make it more obvious what script created the temp file. Note that using scriptname.$$ rather than something like scriptname.tmp (or myAux) allows multiple copies of this script to be run concurrently without interfering with each other (assuming that the files being processed were parameterized rather than hard-coded into the script).

The AND list:
Code:
command1 && command2 && command3

executes command1. If and only if it completes successfully (terminates with exit code 0), then command2 will be run. And, if and only if, command2 also completes successfully, then command3 will be run. In this case, if the cp command completes successfully, we have a backup copy of file1. If the paste completes successfully, file1 has been replaced with the contents of file2 appended to the ends of the lines from the original file1. And if both of them succeed, we remove the backup copy of the initial contents of file1. So, if the cp fails, file1 has not been changed. If cp succeeds and paste fails, file1 can be returned to its original contents by copying the backup file back into file1.

This could also be done by:
Code:
paste -d " " file1 /dev/null file2 > tmp.$$ && mv tmp.$$ file1

but the permissions and ownership of file1 might change, and if there were any hard links to file1 when this script started, they would no longer be linked to file1 if this script completes successfully.

To parameterize the script and make sure that it was invoked with two pathname operands, I would tend to write a slightly longer version of this script:
Code:
#!/bin/ksh
IAm=${0##*/}     # basename of the path used to invoke this script
if [ $# -ne 2 ]  # verify arg count
then    printf "Invalid number of operands.\nUsage: %s path1 path2\n" "$IAm" >&2
fi
tmpf="$IAm.$$"   # create temp file name

# Do the work...
cp "$1" "$tmpf" && paste -d " " "$tmpf" /dev/null "$2" > "$1" && rm "$tmpf"

I usually use the Korn shell, but any shell that accepts basic POSIX conforming shell syntax (including, but not limited to, bash and ksh) will be just fine for this simple script.
This User Gave Thanks to Don Cragun For This Post:
# 7  
Old 10-02-2013
Don Cragun, thanks for that thorough explanation! Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Concatenate files and delete source files. Also have to add a comment.

- Concatenate files and delete source files. Also have to add a comment. - I need to concatenate 3 files which have the same characters in the beginning and have to remove those files and add a comment and the end. Example: cat REJ_FILE_ABC.txt REJ_FILE_XYZ.txt REJ_FILE_PQR.txt >... (0 Replies)
Discussion started by: eskay
0 Replies

2. UNIX for Dummies Questions & Answers

Concatenate files

Hi I am trying to learn linux step by step an i am wondering can i use cat command for concatenate files but i want to place context of file1 to a specific position in file2 place of file 2 and not at the end as it dose on default? Thank you. (3 Replies)
Discussion started by: iliya24
3 Replies

3. UNIX for Dummies Questions & Answers

Concatenate Several Files to One

Hi All, Need your help. I will need to concatenate around 100 files but each end of the file I will need to insert my name DIRT1228 on each of the file and before the next file is added and arrived with just one file for all the 100files. Appreciate your time. Dirt (6 Replies)
Discussion started by: dirt1228
6 Replies

4. Shell Programming and Scripting

Concatenate files

I have a file named "file1" which has the following data 10000 20000 30000 And I have a file named "file2" which has the following data ABC DEF XYZ My output should be 10000ABC 20000DEF (3 Replies)
Discussion started by: bobby1015
3 Replies

5. Shell Programming and Scripting

Concatenate files

Hi, I want to create a batch(bash) file to combine 23 files together. These files have the same extension. I want the final file is save to a given folder. Once it is done it will delete the 23 files. Thanks for help. Need script. (6 Replies)
Discussion started by: zhshqzyc
6 Replies

6. Shell Programming and Scripting

Concatenate files

I have directory structure sales_only under which i have multiple directories for each dealer example: ../../../Sales_Only/xxx_Dealer ../../../Sales_Only/yyy_Dealer ../../../Sales_Only/zzz_Dealer Every day i have one file produce under each directory when the process runs. The requirement... (3 Replies)
Discussion started by: mohanmuthu
3 Replies

7. UNIX for Dummies Questions & Answers

Concatenate column data, grouped by row info

Hello, I have data which looks like: 1 2 3 a x 0 0 a 0 p 0 a 0 0 0 b 0 b c b a 0 0 b 0 0 0 c q 0 s c 0 r 0 I would like to concatenate each of the column data, grouped by the row values, i.e. my... (4 Replies)
Discussion started by: Gussifinknottle
4 Replies

8. Shell Programming and Scripting

How to concatenate a string in every row data of a file??

Please look into the example. My source file is like, 00,57,3,2008-07-24 06:30:06 10,1,8025171,"1M00",17907023,2008-07-23 18:16:58 10,2,8025171,"1M00",17907023,2008-07-23 18:17:01 99,184 What should i do if i want output like... hello,00,57,3,2008-07-24 06:30:06... (7 Replies)
Discussion started by: pkumar3
7 Replies

9. Shell Programming and Scripting

Checking the directory and concatenate the data of all the log files in that dir

Hi Gurus, I am new to unix and need your help to make a shell script. I have a requirement, would appreciate if you could please help me on it: Requirement: ------------- I will pass 2 parameters in shell script 1). Directory name say errors 2). file extension say .log First of all this... (4 Replies)
Discussion started by: anshulinpc
4 Replies

10. Shell Programming and Scripting

How to cut, concatenate data in Shell Script

Hello All,, I'm very new to Unix and I'm more in SAP ABAP. I have a requirement for a shell script which will accept one parameter as Input file name(with directory) e.g : /sapdata/deubank/dbdirect/out/deu01deupe/in/deu01deupe20051207111320.pmt In the shell script I have to make two more... (2 Replies)
Discussion started by: vasan_srini
2 Replies
Login or Register to Ask a Question