Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Issue with awk when joining two files when field has '-' hyphen Post 303030102 by mystition on Wednesday 6th of February 2019 04:19:09 AM
Old 02-06-2019
Issue with awk when joining two files when field has '-' hyphen

Dear Community;

I need to join two files but I am facing issues.

1st file has multiple columns. Primary (1st) columns has unique values. There are other columns out of which some has non-ascii characters as well (other language).
Example File below:

Code:
 
1-1001JRL,BiRecurring
1-1001W5O,QuadRecurring
1-1002KQK,SpclRecurring
1-1002KR9,1-Recurring
1-1002KRN,2-Recurring
1-1002KS1,One-Time
1-1003BAY,Recurring

2nd file has multiple columns as well. Primary (1st) columns has repeated values.
Example File below:

Code:
 
1-1001JRL,abc-1
1-1001JRL,abc-2
1-1001JRL,abc-3
1-1001JRL,abc-4
1-1001JRL,abc-5
1-1001JRL,abc-6
1-1001JRL,abc-7
1-1001JRL,abc-8
1-1001W5O,def-1
1-1001W5O,def-2
1-1001W5O,def-3
1-1001W5O,def-4
1-1001W5O,def-5

I need to join these two files. Expected Output:

Code:
 
1-1001JRL,abc-1,1-1001JRL,BiRecurring
1-1001JRL,abc-2,1-1001JRL,BiRecurring
1-1001JRL,abc-3,1-1001JRL,BiRecurring
1-1001JRL,abc-4,1-1001JRL,BiRecurring
1-1001JRL,abc-5,1-1001JRL,BiRecurring
1-1001JRL,abc-6,1-1001JRL,BiRecurring
1-1001JRL,abc-7,1-1001JRL,BiRecurring
1-1001JRL,abc-8,1-1001JRL,BiRecurring
1-1001W5O,def-1,1-1001W5O,QuadRecurring
1-1001W5O,def-2,1-1001W5O,QuadRecurring
1-1001W5O,def-3,1-1001W5O,QuadRecurring
1-1001W5O,def-4,1-1001W5O,QuadRecurring
1-1001W5O,def-5,1-1001W5O,QuadRecurring
1-1002KQK,SpclRecurring
1-1002KR9,1-Recurring
1-1002KRN,2-Recurring
1-1002KS1,One-Time
1-1003BAY,Recurring

I used below awk commands:

Code:
awk 'NR==FNR {T[$1]=FS $1; next} {print $0 T[$1]}' FS="," File1 File2

Code:
awk -F"," 'FNR==NR{A[$1]=$1;B[$1]=$0;next} ($1 in A){print $0 FS B[$1]} !($1 in A){print $0}' File1 File2

Both the commands are only printing output of 2nd file as the condition for matching the fields are not macthing. I tried using the files after removing all non-ascii columns by just keeping the primary columns as well and it did not work.
I am suspecting that it is because of the presence of spcl character '-' hyphen.

Kindly assist and suggest.

Best Regards//
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Issue with Joining lines from two files

Hi, I have two text files, that need their data joining/concatenation. 'Paste' works for this. But have an issue when there is mismatch in number of rows in each file. E.g. (main file) File1 - has 20 rows File2 - has 30 rows. Command 'paste file1 file2 > file3' joins all lines. I want the... (4 Replies)
Discussion started by: sharath160
4 Replies

2. Shell Programming and Scripting

Joining Two Files Using Awk

Hi All, I am new to awk program. But i have got some assignment on awk. The problem is: i have two files file1 and file2. Both files have same structure. First i have to join both files on filed1,field2 and field3 and then for matching records i want to perform some calculation like:... (1 Reply)
Discussion started by: Jeetuibm
1 Replies

3. Shell Programming and Scripting

1024 field issue : awk

Hi i have a txt file in which i do a awk operation with ":" as field separator A B C D ABC::2386.13:2386.13:3248234281995::+DPY:INT:3:N::::2:200.00:0.00:2010-05-12:CA: ::2:N::N:PH:00010031:0001+DPY:BAL:3:N::::3:1601.01:0.00:2010-05-12:XT::2:N:MR ... (1 Reply)
Discussion started by: mad_man12
1 Replies

4. Shell Programming and Scripting

AWK: Pattern match between 2 files, then compare a field in file1 as > or < field in file2

First, thanks for the help in previous posts... couldn't have gotten where I am now without it! So here is what I have, I use AWK to match $1 and $2 as 1 string in file1 to $1 and $2 as 1 string in file2. Now I'm wondering if I can extend this AWK command to incorporate the following: If $1... (4 Replies)
Discussion started by: right_coaster
4 Replies

5. Shell Programming and Scripting

awk - replace first hyphen

How do I use awk to replace the first hyphen of a specific record? (1 Reply)
Discussion started by: locoroco
1 Replies

6. Shell Programming and Scripting

awk to place value at 24 field in a flat file issue

I am trying to add 0393 value at 24th feild using the below command, but its adding at all the lines including header and trailer Input file: ZHV|2657|D0217001|T|TXU|Z|PAN|20131112000552||||OPER| 754|52479| 492|489|SP40|1014570286334|20131111|20131201|14355334|CHAMELON... (1 Reply)
Discussion started by: Aditya_001
1 Replies

7. Shell Programming and Scripting

UNIX joins : facing issue while joining three files

Hello , I have three files : sampleoutput1.txt has columns (in the following order) : hostname ; available patches , available packages sampleoutput2.txt has columns (in the following order) : hostname ; patchwave ; BSID ; Application sampleoutput3.txt has columns (in the following... (10 Replies)
Discussion started by: rahul2662
10 Replies

8. Shell Programming and Scripting

awk to print unique text in field before hyphen

Trying to print the unique values in $2 before the -, currently the count is displayed. Hopefully, the below is close. Thank you :). file chr2:46603668-46603902 EPAS1-902|gc=54.3 253.1 chr2:211471445-211471675 CPS1-1205|gc=48.3 264.7 chr19:15291762-15291983 NOTCH3-1003|gc=68.8 195.8... (3 Replies)
Discussion started by: cmccabe
3 Replies

9. Shell Programming and Scripting

Joining files using awk not extracting all columns from File 2

Hello All I'm joining two files using Awk by Left outer join on the file 1 File 1 1 AA 2 BB 3 CC 4 DD File 2 1 IND 100 200 300 2 AUS 400 500 600 5 USA 700 800 900 (18 Replies)
Discussion started by: venkat_reddy
18 Replies

10. Shell Programming and Scripting

awk joining multiple lines based on field count

Hi Folks, I have a file with fields as follows which has last field in multiple lines. I would like to combine a line which has three fields with single field line for as shown in expected output. Please help. INPUT hname01 windows appnamec1eda_p1, ... (5 Replies)
Discussion started by: shunya
5 Replies
DBIx::Class::Helper::ResultSet::VirtualView(3pm)	User Contributed Perl Documentation	  DBIx::Class::Helper::ResultSet::VirtualView(3pm)

NAME
DBIx::Class::Helper::ResultSet::VirtualView - Clean up your SQL namespace (DEPRECATED) VERSION
version 2.013002 SYNOPSIS
# note that this is normally a component for a ResultSet package MySchema::ResultSet::Bar; use strict; use warnings; use parent 'DBIx::Class::ResultSet'; __PACKAGE__->load_components('Helper::ResultSet::VirtualView'); # and then in code that uses the ResultSet Join with relation x my $rs = $schema->resultset('Bar')->search({'x.name' => 'abc'},{ join => 'x' }); # 'x' now pollutes the query namespace # So the following works as expected my $ok_rs = $rs->search({'x.other' => 1}); # But this doesn't: instead of finding a 'Bar' related to two x rows (abc and # def) we look for one row with contradictory terms and join in another table # (aliased 'x_2') which we never use my $broken_rs = $rs->search({'x.name' => 'def'}); my $rs2 = $rs->as_virtual_view; # doesn't work - 'x' is no longer accessible in $rs2, having been sealed away my $not_joined_rs = $rs2->search({'x.other' => 1}); # works as expected: finds a 'table' row related to two x rows (abc and def) my $correctly_joined_rs = $rs2->search({'x.name' => 'def'}); DESCRIPTION
This component is will allow you to clean up your SQL namespace. See "NOTE" in DBIx::Class::Helper::ResultSet for a nice way to apply it to your entire schema. DEPRECATED
This component has been suplanted by DBIx::Class::ResultSet::as_subselect_rs. In the next major version(3) we will begin issuing a warning on it's use. In the major version after that(4) we will remove it entirely. METHODS
as_virtual_view Act as a barrier to SQL symbols. The resultset provided will be made into a "virtual view" by including it as a subquery within the from clause. From this point on, any joined tables are inaccessible to ->search on the resultset (as if it were simply where-filtered without joins). See "SYNOPSIS" for example. NOTE
You don't have to use this as a Component. If you prefer you can use it in the following manner: # in code using ResultSet: use DBIx::Class:Helper::VirtualView; my $new_rs = DBIx::Class::Helper::VirtualView::as_virtual_view($rs); THANKS
Thanks to ijw from #dbix-class for the idea for this helper (originally called seal), most of the code, and most of the documentation. AUTHOR
Arthur Axel "fREW" Schmidt <frioux+cpan@gmail.com> COPYRIGHT AND LICENSE
This software is copyright (c) 2012 by Arthur Axel "fREW" Schmidt. This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself. perl v5.14.2 2012-06-18 DBIx::Class::Helper::ResultSet::VirtualView(3pm)
All times are GMT -4. The time now is 03:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy