merge different files into different files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting merge different files into different files
# 1  
Old 06-25-2010
merge different files into different files

Hi,

we have files with the below naming convesion. i want to concatenate all the files into one file.

Source files.

Code:
Customer_information_Detail_1111_201010.csv
Product_information_Detail_1111_201010.csv
Relation_information_Detail_1111_201010.csv

Customer_information_Detail_2222_201010.csv
Product_information_Detail_2222_201010.csv
Relation_information_Detail_2222_201010.csv

Customer_information_Detail_3333_201010.csv
Product_information_Detail_3333_201010.csv
Relation_information_Detail_3333_201010.csv

Output file:

Finally i have to create the files like this

Code:
cat Customer_information_Detail_1111_201010.csv >> Customer_information_Detail.csv
cat Customer_information_Detail_2222_201010.csv >> Customer_information_Detail.csv
cat Customer_information_Detail_3333_201010.csv >> Customer_information_Detail.csv


cat Product_information_Detail_1111_201010.csv >> Product_information_Detail.csv
cat Product_information_Detail_2222_201010.csv >> Product_information_Detail.csv
cat Product_information_Detail_3333_201010.csv >> Product_information_Detail.csv

cat Relation_information_Detail_1111_201010.csv >Relation_information_Detail.csv
cat Relation_information_Detail_2222_201010.csv >Relation_information_Detail.csv
cat Relation_information_Detail_3333_201010.csv >Relation_information_Detail.csv

Like this i have many files.

i have created a script. it is writing into two files. but it is not working properly. Any help greatly app.

Code:
Filename.out contains 
Customer_information_
Product_information_
Relation_information_


Code :

Code:
#!/bin/sh
for file in `cat Filename.out`
do
    Var=`echo $file"Detail`
    echo $Var
    for file1 in `ls |grep $Var`
    do
      cat $file* >>$file1.in
done
done

Moderator's Comments:
Mod Comment Please use code tags

Last edited by Scott; 06-25-2010 at 05:44 AM.. Reason: Code tags
# 2  
Old 06-25-2010
Maybe:
Code:
for i in Customer_information_Detail_*csv; do cat $i >> Customer_information_Detail.csv; done
for i in Product_information_Detail_*csv; do cat $i >> Product_information_Detail.csv; done
for i in Relation_information_Detail_*csv; do cat $i >> Relation_information_Detail.csv; done



---------- Post updated at 10:33 ---------- Previous update was at 10:15 ----------

or
Code:
#!/bin/sh

while read file; do
Var=`echo "$file"Detail`
for file1 in $Var*
do
cat $file1 >> $(echo $Var | awk -F_ '{print $1"_"$2"_"$3".csv"}')
done
done <Filename.out

# 3  
Old 06-25-2010
Hi,


Thanks for sharing valuable information.


Input file name

Code:
Custo_To_Cust_Info_Rela_Detail_1212_200001.csv
Custo_To_Detail_1212_200001.csv
Prod_To_Cust_Info_Rela_Detail_1212_200001.csv
Prod_To_Detail_1212_200001.csv

Atual Output file naming convesion:

Code:
Custo_To_Cust_Info_Rela_Detail.csv
Custo_To_Detail.csv
Prod_To_Cust_Info_Rela_Detail.csv
Prod_To_Detail.csv


The below code is working fine. but output file name is not correct.
Wrong output file naming convesion. As we are passing the 3 columns

Code:
Custo_To_Cust.csv
Custo_To_Detail.csv
Prod_To_Cust.csv
Prod_To_Detail.csv

Code:
while read file; do
  Var=`echo "$file"Detail`
  for file1 in $Var*
  do
    echo $file1

    cat $file1 >> $(echo $Var | awk -F_ '{print $1"_"$2"_"$3".csv"}')
  done
done <filename.out

any help greatly app.

Regards,
Suri
~

Last edited by Scott; 06-25-2010 at 08:53 AM.. Reason: Code tags, please...
# 4  
Old 06-25-2010
Not sure if I got you right, but try this:


filename.out
Code:
Custo_To_Cust_Info_Rela_Detail
Custo_To_Detail
Prod_To_Cust_Info_Rela_Detail
Prod_To_Detail

Code:
while read file; do
 for file1 in $file*; do
 cat $file1 >> $(echo $file1 | sed 's/_[0-9].*/\.csv/')
 done
done <filename.out


Last edited by pseudocoder; 06-25-2010 at 07:54 AM..
# 5  
Old 06-25-2010
Hi

Code:
awk '{sub(/_[0-9]*_[0-9]*/, "",FILENAME);print> FILENAME;}' *.csv

Guru.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to merge two files?

Dear Frens, I have two files and need to merge into one file. Like File_1: Field1 Field2 1 4 File_2: Field1 Field2 3 5 I need one single output as File_1: Field1 Field2 1 4 3 5 This means taking header from either file. (8 Replies)
Discussion started by: manisha_singh
8 Replies

2. Shell Programming and Scripting

Merge files and generate a resume in two files

Dear Gents, Please I need your help... I need small script :) to do the following. I have a thousand of files in a folder produced daily. I need first to merge all files called. txt (0009.txt, 0010.txt, 0011.txt) and and to output a resume of all information on 2 separate files in csv... (14 Replies)
Discussion started by: jiam912
14 Replies

3. Shell Programming and Scripting

Checking in a directory how many files are present and basing on that merge all the files

Hi, My requirement is,there is a directory location like: :camp/current/ In this location there can be different flat files that are generated in a single day with same header and the data will be different, differentiated by timestamp, so i need to verify how many files are generated... (10 Replies)
Discussion started by: srikanth_sagi
10 Replies

4. UNIX for Dummies Questions & Answers

Merge files

Hi, I would like to know how can I merge files based on their coordinates, but mantaining the score of each file in the output file like: Note: 1st column is for chromosome, 2nd for start, 3rd for end of segment, 4th for score file1: 1 200 300 20 1 400 500 30 file2: 1 200 350 30 1... (1 Reply)
Discussion started by: fadista
1 Replies

5. Shell Programming and Scripting

MERGE FILES

Hi all! How could I merge all the text files (in format xml) of a single folder, after having deleted from each of them all the text from its beginning up to a specific string: "<body>" ? Thanks a lot! mjomba (4 Replies)
Discussion started by: mjomba
4 Replies

6. Shell Programming and Scripting

Merge 2 files

Hi , This might be the stupidest question I am asking, but I am struck with this problem. I am trying to merge 2 files, file1 has header and file2 has contents. while I merge them , it merges from the 1st line of file1. for ex: file1 col1|col2|col3| file2 123|234|456|... (2 Replies)
Discussion started by: rashmisb
2 Replies

7. Shell Programming and Scripting

merge files

hi i have two files file1 1234567 1234678 1234679 file2 98765|jjkskk|9393|iyutr 98765|kkooo|9393|hjjjd 98765|abcvfg|9393|sskds output should be 1234567|jjkskk|9393|iyutr 1234678|kkooo|9393|hjjjd 1234679|abcvfg|9393|sskds (5 Replies)
Discussion started by: mad_man12
5 Replies

8. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

9. Shell Programming and Scripting

how to merge these two files?

I have two files, each of them has 12 lines, fileA has 3 columns, fileB has 1 column, like the following FileA a 1 b 2 c 3 ..blabla FileB A B C ..blabla Now I am trying to put the content of fileB as column 3 of fileA, e.g. a 1 A b 2 B c 3 C (3 Replies)
Discussion started by: fedora
3 Replies

10. Shell Programming and Scripting

merge files

Hi, i have the files f1 and f2 like: files f1: c1 a1 c2 a2 c3 a3 file f2: c1 b1 c2 b2 c3 b3 i want merge the f1 and f2 file to f3 file like: c1 a1 b1 c2 a2 b3 c3 a3 b3........ .... . . please help me onthis..... (5 Replies)
Discussion started by: koti_rama
5 Replies
Login or Register to Ask a Question