Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

MERGE 13 files and add the file name at the end of each record

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting MERGE 13 files and add the file name at the end of each record
# 1  
Old 08-19-2008
MERGE 13 files and add the file name at the end of each record
Hi Gurus,
I have 13 comma(,) seperated files that i have to merge and create a single file which has file name attached at th end of each record in the out put file. Can any one please help me with writing a unix script with this issue?
Code:
test1.dat
BIG ID,Local ID,Bond Description,Liquidity,Maturity,Mat Sect,Price,Accr Int,Outs-Loc ,MVal-Loc,MVal-USD,MVal-EUR,Mac Dur,Mod Dur,Yield,Convexity,Coupon,Issue Date,Freq,ISIN
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,AT0000384938
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
 
test2.dat
BIG ID,Local ID,Bond Description,Liquidity,Maturity,Mat Sect,Price,Accr Int,Outs-Loc ,MVal-Loc,MVal-USD,MVal-EUR,Mac Dur,Mod Dur,Yield,Convexity,Coupon,Issue Date,Freq,ISIN
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,AT0000384938
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
 
test3.dat
BIG ID,Local ID,Bond Description,Liquidity,Maturity,Mat Sect,Price,Accr Int,Outs-Loc ,MVal-Loc,MVal-USD,MVal-EUR,Mac Dur,Mod Dur,Yield,Convexity,Coupon,Issue Date,Freq,ISIN
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,AT0000384938
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939
so on upto thirteen files
 
my output neeeds to be
outputfile.dat
BIG ID,Local ID,Bond Description,Liquidity,Maturity,Mat Sect,Price,Accr Int,Outs-Loc ,MVal-Loc,MVal-USD,MVal-EUR,Mac Dur,Mod Dur,Yield,Convexity,Coupon,Issue Date,Freq,ISIN,FINENAME
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,AT0000384938,test1.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test1.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test1.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 3',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test1.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 6',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,AT0000384938,test2.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 6',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test2.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 6',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test2.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 6',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test2.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 8',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,AT0000384938,test3.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 8',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test3.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 8',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test3.dat
AAA0024,BBB0024,sync  5.50%  RAGB  Jan  2010,Ben,15-Jan-2008,'1 - 8',101.075,2.87,8809.982,9157.532,14524.303,9157.532,1.427,1.363,4.703,3.205,5.5,22-Oct-1999,1,PT0000384939,test3.dat

I appreciate if any one can help me with this issue.
thanks
vkr
# 2  
Old 08-19-2008
try:
Code:
awk '{ print $0, FILENAME }'  test*.dat  > outputfile

Won't work if there are any other test<...?>.dat files in the directory thsat you don't want included.
# 3  
Old 08-19-2008
MySQL
awk '{printf "%s,%s\n",$0,FILENAME}' *.dat >new_filename
This User Gave Thanks to palsevlohit_123 For This Post:
A-V
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Need help to delete special characters exists only at the end of the each record in UNIX file?

Hi, I have a file in unix with 15 columns.It consists special characters(#,$,^M,@,*,% etc)at the end of the each record.I want to remove these special characters.I used the following: Sed -e 's/ /g;s/ */ /g' . But It is removing special characters exists everywhere in the file(begining,middle... (24 Replies)
Discussion started by: rakeshp
24 Replies

2. Shell Programming and Scripting

How to add trailer record at the end of the flat file in the unix ksh shell scripting?

Hi, How to add trailer record at the end of the flat file in the unix ksh shell scripting can you please let me know the procedure Regards Srikanth (3 Replies)
Discussion started by: srikanth_sagi
3 Replies

3. Shell Programming and Scripting

Need a ksh script for adding the space at the end of record in a flat file

Hi, I need a ksh script for the below requirement: i have a Delimited flat file with 200 records delimiter is '|~|' i need a script to insert space at the end if the record is ending with delimiter '|~|' if it didnt end with delimiter it should not append space. Example: ram|~|2|~| ... (16 Replies)
Discussion started by: srikanth_sagi
16 Replies

4. UNIX for Dummies Questions & Answers

Merge files and add file name to the end of each line

Hello everybody, I'm trying to merge a lot of files, but I want to include the filename to the end of each line. I've tried to use cat, but I got stuck. My files are for example: file01.001 123456 aaa ddd ee 458741 eee fff ee file02.003 478596 uuu ddd ee 145269 ttt fff ee ... (4 Replies)
Discussion started by: ernesto561
4 Replies

5. Shell Programming and Scripting

how to add blank spaces at the end of every record in a file.

hi, Does anyone has any idea in adding few blank spaces at the end of every record in a file. Eg: file.txt Baby Boy Kim 1234 Baby Boy Vik 1334 Desired output:- output.txt Baby Boy Kim 1234 Baby Boy Vik 1334 I want to add 10 blank spaces at the end every record in file.txt (3 Replies)
Discussion started by: techmoris
3 Replies

6. Shell Programming and Scripting

Add end of char \n on end of file

Hi, I want to add \n as a EOF at the end of file if it does't exist in a single command. How to do this? when I use command echo "1\n" > a.txt and od -c a.txt 0000000 1 \n \n 0000003 How does it differentiate \n and eof in this case? Regards, Venkat (1 Reply)
Discussion started by: svenkatareddy
1 Replies

7. Shell Programming and Scripting

How to merge and add?? Two Files

Sorry noob here...... I have 2 files like this: File A 8080000001 400 8080000002 300 8080000003 200 File B 8080000001 600 8080000002 200 8080000004 200 I want one file where File OUT 8080000001 1000 (6 Replies)
Discussion started by: Unice
6 Replies

8. Shell Programming and Scripting

append a record at the end of a file

Hi all, i have to append a record at the end of the file(a file which is already with some records).how do i do?please help me? is there any way of doing this with "SED" command.i am not sure.plz help me on this. would appreciate your ideas!!!! bye rao. (3 Replies)
Discussion started by: raoscb
3 Replies

9. Shell Programming and Scripting

Add 'ENDEND' on end of each record at position is 14-20

I have file format like below and I'm trying to modify this file. I need to add 'ENDEND' end of each record. 01 ASH01 1CTCTL EDPPOO STAND 01 ASH08 0020 A1TH 101 01 ASH09 0022 A1TH 102 01 ASH09 0022 A1TH 103 01 ASH02 2CTCTL ... (5 Replies)
Discussion started by: naveenkcl
5 Replies
Login or Register to Ask a Question