Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

formatting and merging 2 data files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting formatting and merging 2 data files
# 1  
Old 02-26-2010
formatting and merging 2 data files
Hi,
I have 2 files that I got as an output from another program. They are :

File 1
Code:
((((((CtBJa:197.0,CtBTz:197.0):85.0,CtAHr:197.0):116.0,CtDUw:197.0):176.0,CtSwe:197.0):110.0,
(CtL2b:197.0,Ct4Bu:197.0):196.0):197.0,CmuNg:197.0);
((((CtL2b:200.0,Ct4Bu:200.0):177.0,((CtAHr:200.0,CtBJa:200.0):77.0,(CtBTz:200.0,
CtDUw:200.0):84.0):127.0):130.0,CtSwe:200.0):200.0,CmuNg:200.0);
(((CtBJa:200.0,CtAHr:200.0):102.0,(((Ct4Bu:200.0,(CtBTz:200.0,CtSwe:200.0):147.0):115.0,
CtDUw:200.0):73.0,CtL2b:200.0):105.0):200.0,CmuNg:200.0);
((((((CtBTz:200.0,CtAHr:200.0):70.0,CtDUw:200.0):68.0,CtBJa:200.0):172.0,(Ct4Bu:200.0,
CtL2b:200.0):178.0):135.0,CtSwe:200.0):200.0,CmuNg:200.0);
((CtSwe:200.0,(((((CtAHr:200.0,CtBJa:200.0):92.0,CtDUw:200.0):48.0,CtBTz:200.0):126.0,
Ct4Bu:200.0):81.0,CtL2b:200.0):135.0):200.0,CmuNg:200.0);
(((((CtL2b:200.0,Ct4Bu:200.0):125.0,((CtAHr:200.0,CtSwe:200.0):85.0,CtBTz:200.0):92.0):107.0,
CtDUw:200.0):173.0,CtBJa:200.0):200.0,CmuNg:200.0);
((CtSwe:200.0,(((((CtBTz:200.0,Ct4Bu:200.0):106.0,CtDUw:200.0):79.0,CtL2b:200.0):68.0,
CtAHr:200.0):78.0,CtBJa:200.0):128.0):200.0,CmuNg:200.0);
((((((Ct4Bu:200.0,(CtBTz:200.0,CtSwe:200.0):113.0):85.0,CtL2b:200.0):75.0,CtAHr:200.0):92.0,
CtBJa:200.0):140.0,CtDUw:200.0):200.0,CmuNg:200.0);

File2

Code:
10
100
101
102
103
104
105
106
107
108

For File 1 what i need is; each line should be separated by a ';'. But the file I have (file 1), it is split into 2 lines. What I want is, make them one line so that every line ends with ';'. Its like make every 2 lines in the current format to one line and each have the ';' at the end position.

Once I have file 1 in the right format, I need to merge file 2 with file 1 in the same order separated by a tab. Like this:

Code:
1st line of file 2 tab 1st newly format line of file 1
2st line of file 2 tab 2st newly format line of file 1
3st line of file 2 tab 3st newly format line of file 1
4st line of file 2 tab 4st newly format line of file 1
5st line of file 2 tab 5st newly format line of file 1
6st line of file 2 tab 6st newly format line of file 1

Please let me know the best way to do this formating using unix or awk.

LA

---------- Post updated at 06:29 PM ---------- Previous update was at 06:25 PM ----------

Like wise I have 1000's of lines in each file

LA
Last edited by Lucky Ali; 02-26-2010 at 07:28 PM.. Reason: add one more point
# 2  
Old 02-26-2010
Code:
sed '$!N;s/\n//' file1 |sed 's/;//' > tmpfile
paste tmpfile file2

# 3  
Old 02-26-2010
Try:

Code:
paste  <(awk '!/;/ {_s=$0;} /;/ { print _s,$0; }' file) <(cat file2)

# 4  
Old 02-26-2010
Just awk Smilie
Code:
awk '{getline _<x;printf "%s\t%s",_,$0;getline;print}' x=file2 file1

# 5  
Old 02-27-2010
Hi you can use the following Script ......

Script:
Code:
for file in `cat filelist`
do
 rm -r ${file}-arr 1>/dev/null 2>/dev/null
    count=1
    while read line
    do
      if [ $(($count % 2)) -eq 0 ]
      then
      echo "echo $line" >>$file-arr
      else
      echo "$line\c" >>$file-arr
      fi
     ((count=$count+1))
    done < $file
done
paste file2 ${file}-arr >$file-final

Files used here:

file2 ---- second file (while you mentioned as file2 in the post)
$file-arr ---intermediate generated file after rearrangement of lines in file1
$file-final ---- final required output file
filelist ---- a file contains a list file1 types

Hope this will help you or else atleast it will give you some idea to proceed further.
Last edited by Franklin52; 02-28-2010 at 09:09 AM.. Reason: Please use code tags
# 6  
Old 08-13-2010
Hi All,

I have the same file format as shown in file 1 but this time it is not in 2 lines i.e the ";" doesn't come every 2 lines.

How would i modify the above sed/awk code in such way that

For File 1 what i need is; each line should be separated by a ';'. But the file I have (file 1), it is split into 2 or more lines (not fixed all are above 10 lines now. What I want is, make them one line so that every line ends with ';'. Its like make every lines in the current format to one line and each have the ';' at the end position.
Let me know
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging data horizontally with newlines in files

Hi Everyone, I have two files file1 and file2 with these contents cat file1 AAAAA 01/03/2014 04:01:23 BBBB 01/03/2014 03:03:34 CCCcc 01/03/2014 03:03:34 cat file2 1 RED 1 HHHH 1 TTTT 1 BBBBB I tried the below... (2 Replies)
Discussion started by: Aditya_001
2 Replies

2. Shell Programming and Scripting

Help with merging data into single line.

Hi, My input is <message> looking for a big <message>Does fit my G74 laptop. Makes the 10 pound. <message> <message>This bag is the only one I could find to fit my awesome ASUS G74S. <message> <message> Great bag my only wish is that they had put a pocket in which to store and... (6 Replies)
Discussion started by: pamu
6 Replies

3. UNIX for Dummies Questions & Answers

Merging data in a file

Hello, Firstly I just wanted to say that I'm not a programmer at all and appreciate any help you can give. I am trying to create a shellscript that reformats the file and adding up colums 5 and 6 for those sections that are continuation of the previous line(s) (signified by beginning with '*')... (4 Replies)
Discussion started by: neilh1703
4 Replies

4. Shell Programming and Scripting

Merging data from one file into another

Hello, I have a master database of a dictionary with the following structure: a=b (b is a Unicode string) a is the English part and b is the equivalent in a foreign language I have also another file which has a database where the /b/ part of the string has been corrected by an expert. let us... (5 Replies)
Discussion started by: gimley
5 Replies

5. Shell Programming and Scripting

Merging data from 2 files of different lengths?

Hi all, Sorry if someone has answered something like this already, but I have a problem. I am not brilliant with "awk" but think it should be the command to use to get what I am after. I have 2 files: job-file (several hundred lines like): 1018003,LONG MU WAN,1113S 1018004,LONG MU... (4 Replies)
Discussion started by: sgb2301
4 Replies

6. Shell Programming and Scripting

Removing Carriage return and merging data

Hi, I am trying to remove the carriage return on the record which starts with ADD, MODIFY, or DELETE keyword as the first value in the record. If the records does not start with anyone of these keywords then combine the records with the previous record (line). Input File name xyz.txt... (6 Replies)
Discussion started by: naveed
6 Replies

7. Shell Programming and Scripting

Merging last and syslog data on time

This is on a HP-UX system. I need to merge the 2 reports, for each line in syslog I need to lookup who was logged in to the pts/# based on the time from the last.txt report. Here is what I get from sulog.log cat syslog | grep "su:" | grep "Jun 14" Jul 14 08:02:48 server1 su: - 2 ... (8 Replies)
Discussion started by: Ikon
8 Replies

8. Shell Programming and Scripting

merging CSV data using a one liner from shell?

I'm trying to merge multiple CSV (comma separated value) files into one large master file. All files have a field that is unique to act as the key for entry/merging into the master file & and all files have the same number of fields that are in the master file. I'll give an example here: ... (2 Replies)
Discussion started by: jjinca
2 Replies

9. Shell Programming and Scripting

Need help for 2 data file merging

Hello Please help me to write Shell script. I want to merge 2 data files . The data files have common columns The data file A have 3 columns Host Version Numberof Failuers The data file B have also 3 coulmns Host Version NumberofFailuers . I want to merge A and B file... (2 Replies)
Discussion started by: getdpg
2 Replies

10. Shell Programming and Scripting

Merging data

Hi, I have the following problem: Input: "num1","num2","num3",num4,num5,"num6" required output: "num1num2","num3",num4,num5,"num6" I need to join field 1 and field 2 together but I always end up getting: "num1""num2","num3",num4,num5,"num6" Note that not all fields have " at both... (8 Replies)
Discussion started by: ReV
8 Replies
Login or Register to Ask a Question