How to combine the data of files?

Login or Register to Ask a Question and Join Our Community

How to combine the data of files?

Tags

awk bash, shell scripts

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting How to combine the data of files?

06-05-2013

Registered User

1, 0

Join Date: Jun 2013

Last Activity: 25 June 2013, 10:33 PM EDT

Posts: 1

Thanks Given: 1

Thanked 0 Times in 0 Posts

How to combine the data of files?

I have a main file as follows

Code:

aaa   3/2   =  1.5  
aba   55+6  =  61
aca   67+8  =  75
hjk   3+3   =  67
ghd   66+30 =  96
ghj   99-3  =  96
ffg   67+3  =  70

I have 4 sub files named sub1, sub2, sub3, sub4

content of sub1

Code:

aaa   23+5  =  28
hjk   45+6  =  51 
ghd   40-20 =  20

content of sub2

Code:

aca   67+6  = 71
ffg   56+7  = 63

content of sub3

Code:

aca   67+6  = 71
ghd   56+7  = 63

content of sub4

Code:

aaa   3/2   =  1.5  
aba   55+6  =  61
aca   67+6  =  71
hjk   3+3   =  67
ghd   66+30 =  96
ghj   99-3  =  96
ffg   56+7  =  63

desired output

Code:

  main                        sub1                  sub2             sub3             sub4
  
aaa   3/2   =  1.5         23+5  =  28               NIL             NIL            3/2   =  1.5  
aba   55+6  =  61           NIL                      NIL             NIL            55+6  =  61
aca   67+8  =  75           NIL                     67+6  = 71      67+6  = 71      67+6  =  71       
hjk   3+3   =  67          45+6  =  51              NIL              NIL            3+3   =  67
ghd   66+30 =  96          40-20 =  20              NIL             56+7  = 63      66+30 =  96
ghj   99-3  =  96           NIL                     NIL              NIL            99-3  =  96
ffg   67+3  =  70           NIL                     56+7  = 63       NIL            56+7  =  63

I want to combine the data from sub files in to main file. What is the easy way to do this?

Thanks!!

Last edited by Franklin52; 06-05-2013 at 11:09 AM.. Reason: Please use code tags

jackevan

View Public Profile for jackevan

Find all posts by jackevan

06-05-2013

Registered User

7,747, 559

Join Date: Feb 2007

Last Activity: 20 April 2020, 11:28 AM EDT

Location: The Netherlands

Posts: 7,747

Thanks Given: 139

Thanked 559 Times in 520 Posts

Can you show us what you have tried so far?

Franklin52

View Public Profile for Franklin52

Find all posts by Franklin52

06-06-2013

Registered User

5,091, 1,931

Join Date: May 2012

Last Activity: 15 July 2020, 4:46 AM EDT

Location: Simplicity

Posts: 5,091

Thanks Given: 565

Thanked 1,931 Times in 1,668 Posts

1.
One can read the input files one by one, and store the field values in an two-dimensional array, and print it at the end.
2.
If each input file is sorted, one could go for a line-by-line merge (opening and processing all input files at the same time).
--
awk and perl have hashed arrays and can address fields by key values like aaa, aba, ... this is much more efficient than looping through the array and compare the keys.

---------- Post updated 06-06-13 at 03:11 AM ---------- Previous update was 06-05-13 at 04:19 PM ----------

Here is an implementation of 1.

Code:

awk '
NR==FNR { # first file: store key $1 in hash R
  R[$1]=++rows
}
FNR==1 { # first line of a file: increase field offset and print header
  ++fields
  printf "%-*s",width,FILENAME
}
{ # all files: store line in two-dimensional hash A; for next files skip the key $1
  A[fields,R[$1]]=(NR==FNR)?$0:substr($0,length($1)+2)
}
END { # at the end: print all fields of hash A, replace empty fields by NIL
  print ""
  for (r=1;r<=rows;r++) {
    for (f=1;f<=fields;f++) printf "%-*s",width,((f,r) in A)?A[f,r]:"NIL"
    print ""
  }
}' width=30 main sub?

BTW the A could also be a one-dimensional hash of strings, where new strings are appended.

Last edited by MadeInGermany; 06-06-2013 at 05:20 AM..

This User Gave Thanks to MadeInGermany For This Post:

MadeInGermany

View Public Profile for MadeInGermany

Find all posts by MadeInGermany

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,...

2. Shell Programming and Scripting

Combine data out of 3 files into one new file

Hi, How can I combine the data of of three files into one new file? I try to give as much informations as possible. The three existing files are called file1 file2 and file3 the new file should named output_combined. The size of the files will be around 900 words/lines each .. but always...

3. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of...

4. Shell Programming and Scripting

Combine/omit data from 2 files

i made a script on my own. this is for the inventory to all of my AWS servers, and i run it to all of my servers to get the hostname, please look at file2. Then i need some data in file3 as well,. i need to combine them #cat file1 192.10.1.41 server.age.com ###### 192.10.0.40 ssh cant...

5. Shell Programming and Scripting

How to combine data from 2 file ? How to print ?

Dear friends, I am just trying to print data from 2 file,namely file_1.txt and file_1.dat (specific column of ROW 1) file_1.txt 12 13 14 15 99 AMC 69 36 89 12 13 14 15 99 AMC 69 84 -12 12 13 14 ...

6. Shell Programming and Scripting

get data from files combine them to a file

hi people; this is my file1.txt:192.168.1.1 192.168.1.2 192.168.1.3 192.168.1.4 ... this is my file2.txt:portnames usernames maxusercap ... i want to write to file3.txt:l ./getports 192.168.1.1 'get all;l+;get . portnames;l-' l ./getports 192.168.1.1 'get all;l+;get . usernames;l-'...

7. Shell Programming and Scripting

combine data of 2 files by variable

my first post ... please be gentle. I have been working on a script to get info out of mysql. Its a support ticket system database OTRS. I can write the subject of open tickets to a text file with a unique user id. I also have a text file with the unique user id, username and email adres. I...

8. Shell Programming and Scripting

How to combine data files using for loop

Hi, I have 5 files basically;namely file1.txt situated each at folder A to E respectively. I would like to extract out third column from each of these file1.txt from folder A to folder E. Also, I wanted to extract the first and second column which are common. In other words, e.g ...

9. Shell Programming and Scripting

How to combine text data into one line?

The following input needs to be manipulated as follows: INPUT from file or results of command: ============start: Medium identifier : a45c0213:47eb5485:0aec:0321 Medium label : SQL Disk_11516 Location : Protected : None ...

10. UNIX for Dummies Questions & Answers

Recover data from 2 files then combine

Using dd or similar tools to recover data from 2 damaged cdroms, I need a way to then combine the 2 files, 1 from each cd, and make a good file: this all result from finding that certain cd's tops scratch easily even when using the "proper" cd markers, hence making the file useless, however the...

Login or Register to Ask a Question