UNIX joins : facing issue while joining three files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting UNIX joins : facing issue while joining three files
# 1  
Old 07-27-2015
UNIX joins : facing issue while joining three files

Hello ,

I have three files :


sampleoutput1.txt has columns (in the following order) :
Code:
hostname ; available patches , available packages

sampleoutput2.txt has columns (in the following order) :
Code:
hostname ; patchwave ; BSID ; Application

sampleoutput3.txt has columns (in the following order) :
Code:
boottime ; hostname


I am trying to join three files on the column hostname from the three files.
I want the output records as follows :

Code:
hostname;available patches;available packages;boot time;patchwave;BSID;Application

Below is the script. Here I am first trying to join the files : sampleoutput1.txt and sampleoutput3.txt. Then I am trying to join the resultant with sampleoutput2.txt

Code:
#!/bin/bash
awk -F";" 'FNR==NR{A[$2]=$1;next} {print $0 OFS A[$1] OFS}' OFS=";" sampleoutput3.txt sampleoutput1.txt > sampleoutput1-3.txt
awk -F";" 'FNR==NR{B[$1]=$1;next} {print $0 OFS B[$2] OFS B[$3] OFS B[$4]}' OFS=";" sampleoutput2.txt sampleoutput1-3.txt > sampleoutput1-2-3.txt
echo "Output is placed in file : sampleoutput1-2-3.txt"

The file sampleoutput1-3.txt is giving correct output but file : sampleoutput1-2-3.txt is not giving correct output.

Please help.

Thanks
Rahul

Last edited by Scrutinizer; 07-27-2015 at 10:01 AM.. Reason: Changed code tags
# 2  
Old 07-27-2015
I'd be surprised if sampleoutput1-3.txt held the correct output as the field separator is NOT ";" and the files' sequence is wrong.

Given the first file does NOT have a line break before the third field, and that spaces are used consistently around the field contents, and that the DOS/windows <CR> line terminators are removed, this might work:
Code:
join -t, -22 file1 file3 | join  -t, - file2
hostname,available patches,available packages,boottime,patchwave,BSID,Application


Last edited by RudiC; 07-27-2015 at 08:52 AM..
This User Gave Thanks to RudiC For This Post:
# 3  
Old 07-27-2015
Hello RudiC , the files are not sorted moreover there are some records for hostname which are present in sampleoutput1.txt and not present in sampleoutput2.txt and sampleoutput3.txt. Also delimiter used is ";". I think the above wont work in this scenario. Please help.
# 4  
Old 07-27-2015
You should not modify posts after an answer was given referring to its contents. And, you should do the initial specification with a bit more care.
# 5  
Old 07-27-2015
Code:
awk -F ';' '
 FILENAME == ARGV[1] {A[$1] = ($2 ";" $3); next}
 FILENAME == ARGV[2] {A[$2] = A[$2] == "" ? (";;" $1) : (A[$2] ";" $1); next}
 FILENAME == ARGV[3] {A[$1] = A[$2] == "" ? (";;;" $2 ";" $3 ";" $4) : (A[$1] ";" $2 ";" $3 ";" $4)}
 END {for(x in A) print x ";" A[x]}' sampleoutput1.txt sampleoutput3.txt sampleoutput2.txt > sampleoutput1-2-3.txt

This User Gave Thanks to SriniShoo For This Post:
# 6  
Old 07-27-2015
After your corrections, I noticed there is a comma where there should be a semicolon, also there are spaces around the semicolons. I do not know if they are there in reality, I adjusted FS to compensate. If they are not there you can just use ";". Please put more care still into your file sample specifications.

Try:
Code:
awk '
  BEGIN {
    FS=" *; *" 
    OFS=";"
  } 
  FNR==1{
    c++
  }
  c==1 { 
    A[$1]=$0 
  } 
  c==2 { 
    A[$2]=A[$2] OFS $1
  }
  c==3 { 
    $1=A[$1]
    print
  }
' sampleoutput1.txt sampleoutput3.txt sampleoutput2.txt


Last edited by Scrutinizer; 07-28-2015 at 03:06 AM..
This User Gave Thanks to Scrutinizer For This Post:
# 7  
Old 07-28-2015
Very Sorry RudiC , I will take more care next time. Thanks for your help.

---------- Post updated at 08:28 AM ---------- Previous update was at 08:24 AM ----------

Thanks a lot Scrutinizer , the code which you provided worked fine. Also I will ensure more care while posting next time. Could you please help me in explaining the script. Also Please provide some sources/books which can help me in learning to master advanced awk and sed.

---------- Post updated at 10:05 AM ---------- Previous update was at 08:28 AM ----------

Hello SriniShoo , The script which you provided went into long running and eventually i stopped it after approx 15 minutes. Thanks a lot for your efforts
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Issue with awk when joining two files when field has '-' hyphen

Dear Community; I need to join two files but I am facing issues. 1st file has multiple columns. Primary (1st) columns has unique values. There are other columns out of which some has non-ascii characters as well (other language). Example File below: 1-1001JRL,BiRecurring... (5 Replies)
Discussion started by: mystition
5 Replies

2. SuSE

Facing issue configuring network

Please let me know how to configure network in suse Linux, I have configured the network using ifup and network manager, it is not giving any error but not working, using suse Linux 11.0 sp3 I have checked network connectivity is working. (0 Replies)
Discussion started by: manoj.solaris
0 Replies

3. Solaris

Facing issue while installing weblogic on Solaris 11

Hi, i am facing issue while installing weblogic on solaris..its giving me invalid argument error. solaris is intstalled on my VM. uname -a SunOS Vishal 5.10 Generic_137138-09 i86pc i386 i86pc screenshot attached. (5 Replies)
Discussion started by: Vishal Baghla
5 Replies

4. Infrastructure Monitoring

Facing Issue in Nagios 3.3

Hi, I have installed Nagios on Redhat Linux, I have installed Nagios+Plugins+NRPE on Server A (Host Server) and Nagios Plugins + nrpe on remote linux server (red hat linux) run the command on remote linux host, it returns nrpe version usr/local/nagios/libexec/check_nrpe -H localhost ... (1 Reply)
Discussion started by: manoj.solaris
1 Replies

5. Shell Programming and Scripting

Facing Issue after configuring logrotate

Hi, I have a logrotate configuration which rotates a log every night 1 min before midnight, but somehow its not working and unfortunately not showing any error message as well. Sharing the code for the cron job as well as the conf file, I am using, if some one coule help me..whats wrong with... (2 Replies)
Discussion started by: Neeryan
2 Replies

6. Programming

SQL Developer JOINS / GROUP BY issue.

Am having a nightmare with a certain piece of code.. have tried almost everything and just cannot see what the issue is.. CREATE OR REPLACE VIEW TOP_EARNER_PER_LOCATION AS SELECT E.FIRST_NAME || ' ' || E.LAST_NAME AS EMPLOYEE_NAME, L.REGIONAL_GROUP AS REGIONAL_GROUP, ... (1 Reply)
Discussion started by: U_C_Dispatj
1 Replies

7. Shell Programming and Scripting

Issue with Joining lines from two files

Hi, I have two text files, that need their data joining/concatenation. 'Paste' works for this. But have an issue when there is mismatch in number of rows in each file. E.g. (main file) File1 - has 20 rows File2 - has 30 rows. Command 'paste file1 file2 > file3' joins all lines. I want the... (4 Replies)
Discussion started by: sharath160
4 Replies

8. Shell Programming and Scripting

Facing issue while using xsltproc tp parse XML in bash

I have written a bash script which opens a folder, reads all the *.xml files in it, and pulls the required data that i need from XML tags. I am using xsltproc (my xsl name) (my xml folder location/*.xml) and running this in a for each loop The problem is that some XML files are having special... (3 Replies)
Discussion started by: shivashankar.g
3 Replies

9. Shell Programming and Scripting

Facing issue in Solaris OS in crontab for running shell script

Hello i have a shell script. it is running fine when i manually run at command prompt using following command ./script_file but while running shell script from crontab, it is giving error in each line. (2 Replies)
Discussion started by: mabrar
2 Replies

10. HP-UX

Facing an issue related to cronjob

Dear All, I am facing an issue related to cronjob and explained below is the case study: 1. I have a java class named "DmCheckRenditionQueue.java" and placed under "/cpc/documentum/product/5.2.5/tomcat/webapps/rendition" 2. When I am using the command "/usr/openv/java/jre/bin/java -cp... (1 Reply)
Discussion started by: parindam
1 Replies
Login or Register to Ask a Question