Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Merge file lines based off of keyword

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge file lines based off of keyword
# 1  
Old 11-22-2011
Merge file lines based off of keyword
Hello Everyone,
I have two files I created in a format similar to the ones found below (character position is important):

File 1:
Code:
21 Cat Y N S Y Y N N 
FOUR LEGS
TAIL
WHISKERS
30 Dog N N 1 Y Y N N 
FOUR LEGS
TAIL
33 Fish Y N 1 Y Y N N 
FINS
43 CAR Y N S Y Y N N 
WHEELS
DOORS
TRAVEL

File 2:
Code:
FLAG1 SetA
FLAG7 SetB
FLAG10 SetC
FLAG9 SetD

I'd like to somewhat merge the files so that my final output is the following:

Code:
21 Cat Y N 1 Y Y N N FLAG1 SetA
FOUR LEGS
TAIL
WHISKERS
30 Dog N N 1 Y Y N N FLAG7 SetB
FOUR LEGS
TAIL
33 Fish Y N S Y Y N N FLAG10 SetC
FINS
43 CAR Y N S Y Y N N FLAG9 SetD
WHEELS
DOORS
TRAVEL

To do this, I believe that I should search for the following keyword "Y Y N N " and iterate through file2 to paste the flag & set id, but I'm not quite sure how I can do this. Any help would be greatly appreciated.
Moderator's Comments:
Mod Comment
Please use code tags when posting data and code samples!
Last edited by vgersh99; 11-22-2011 at 11:46 AM.. Reason: code tags, please!
# 2  
Old 11-22-2011
Code:
nawk 'FNR==NR{f2[FNR]=$0;next}{print $0 (($1~/[0-9]+/)?OFS f2[++c]:"")}' file2 file1

Last edited by vgersh99; 11-22-2011 at 12:52 PM.. Reason: ooops - buggy bug!
# 3  
Old 11-22-2011
Quote:
Originally Posted by vgersh99
Code:
nawk 'FNR==NR{f2[FNR]=$0;next}{print $0 (($1~/[0-9]+/)?OFS f2[++c]:"")}' file2 file1

thanks for the edit. Quick question though, where in the code is it looking for the keyword "Y Y N N"? When I run the code, the files merge, but the (byte) position does not remain intact. For instance, in some cases, the FLAGS showed appread in column 51, when they should all start at column 80.

Not quite sure what is going on in your code? Could you please elaborate?
# 4  
Old 11-22-2011
Quote:
Originally Posted by jl487
thanks for the edit. Quick question though, where in the code is it looking for the keyword "Y Y N N"? When I run the code, the files merge, but the (byte) position does not remain intact. For instance, in some cases, the FLAGS showed appread in column 51, when they should all start at column 80.

Not quite sure what is going on in your code? Could you please elaborate?
I'm not looking for 'Y Y N N'. I'm looking for lines in file1 where the first field is numeric:
Code:
21 Cat Y N S Y Y N N
FOUR LEGS
TAIL
WHISKERS
30 Dog N N 1 Y Y N N
FOUR LEGS
TAIL
33 Fish Y N 1 Y Y N N
FINS
43 CAR Y N S Y Y N N
WHEELS
DOORS
TRAVEL

This is the piece of code that performs that check:
Code:
$1~/[0-9]+/

I should probably tighten it up a bit more:
Code:

nawk 'FNR==NR{f2[FNR]=$0;next}{print $0 (($1~/^[0-9]+$/)?OFS f2[++c]:"")}' file2 file1

# 5  
Old 11-22-2011
Quote:
Originally Posted by vgersh99
I'm not looking for 'Y Y N N'. I'm looking for lines in file1 where the first field is numeric:
Code:
21 Cat Y N S Y Y N N
FOUR LEGS
TAIL
WHISKERS
30 Dog N N 1 Y Y N N
FOUR LEGS
TAIL
33 Fish Y N 1 Y Y N N
FINS
43 CAR Y N S Y Y N N
WHEELS
DOORS
TRAVEL

This is the piece of code that performs that check:
Code:
$1~/[0-9]+/

I should probably tighten it up a bit more:
Code:
nawk 'FNR==NR{f2[FNR]=$0;next}{print $0 (($1~/^[0-9]+$/)?OFS f2[++c]:"")}' file2 file1

very clever!! But what can I do if i get the following entry?
Code:
99 Man Y N S Y Y N N
2 Hands
2 Feet

# 6  
Old 11-22-2011
Quote:
Originally Posted by jl487
very clever!! But what can I do if i get the following entry?
Code:
99 Man Y N S Y Y N N
2 Hands
2 Feet

Then my 'clever' assumption is not clever no more and we need to identify the 'line' by some other means. Did you say the pattern 'Y Y N N'? It's not going to appear in any other lines? It doesn't matter where it appears on a line?
Code:
nawk -v p='Y Y N N' 'FNR==NR{f2[FNR]=$0;next}{print $0 (($0~p)?OFS f2[++c]:"")}' file2 file1

# 7  
Old 11-22-2011
Quote:
Originally Posted by vgersh99
Then my 'clever' assumption is not clever no more and we need to identify the 'line' by some other means. Did you say the pattern 'Y Y N N'? It's not going to appear in any other lines? It doesn't matter where it appears on a line?
Code:
nawk -v p='Y Y N N' 'FNR==NR{f2[FNR]=$0;next}{print $0 (($0~p)?OFS f2[++c]:"")}' file2 file1

Yes, Y N N is shown on all of the main categories. When i tried the code you provided, it seems to have copied extra info. I think i can use notepad++ to remove some of this.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk join lines based on keyword

Hello , I will need your help once again. I have the following file: cat file02.txt PATTERN XXX.YYY.ZZZ. 500 ROW01 aaa. 300 XS 14 ROW 45 29 AS XD.FD. PATTERN 500 ZZYN002 ROW gdf gsste ALT 267 fhhfe.ddgdg. PATTERN ERE.MAY. 280 PATTERRNTH 5000 rt.rt. ROW SO a 678 PATTERN... (2 Replies)
Discussion started by: alex2005
2 Replies

2. Shell Programming and Scripting

Merge lines based on match

I am trying to merge two lines to one based on some matching condition. The file is as follows: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow ... (8 Replies)
Discussion started by: jamie_123
8 Replies

3. Shell Programming and Scripting

Merge mutiple lines into one based on if the first word is some particular value

Hi, trying to knock something together to create one line entries based on whether the first word on each line matches a particular value. eg. Link,"Name=""Something\something"","Timeout=""1800""", "Target=""\\thing\thing\thing""","State=""ONLINE""",something,... (0 Replies)
Discussion started by: adamdb
0 Replies

4. Shell Programming and Scripting

Need to merge lines based on pattern

Hi, I have a requirement to merge multiple lines based on search pattern. The search criteria is : it will search for CONSTRAINT and when it found CONSTRAINT, it will merge all lines to 1 line till it founds blank line. For Example: CREATE TABLE "AMS_DISTRIBUTOR_XREF" ( "SOURCE"... (5 Replies)
Discussion started by: satyaatcgi
5 Replies

5. Shell Programming and Scripting

How to merge lines based off of text?

Hello Everyone, I have two files, similar to the following: File 1: 8010 ITEM01 CODE1 FLAG1 filler filler 7020 OBJECT CODE2 FLAG2 filler 6010 THING1 CODE4 FLAG4 6011 ITEM20 CODE7 FLAG7 File 2 contains: 6020 ITEM01 CODEA FLAGA filler filler filler 7000 OBJECT CODEB... (2 Replies)
Discussion started by: jl487
2 Replies

6. Shell Programming and Scripting

Gawk / Awk Merge Lines based on Key

Hi Guys, After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines. I'm an SEO Consultant and provide... (9 Replies)
Discussion started by: Jamesfirst
9 Replies

7. Shell Programming and Scripting

Merge lines in text file based on pattern

Hello, I have searched forum trying to find a solution to my problem, but could not find anything or I did not understand the examples.... I should say, I am very inexperienced with text processing. I have a text file with approx 60k lines in it. I need to merge lines based on the number... (8 Replies)
Discussion started by: Bertik
8 Replies

8. Shell Programming and Scripting

Extract lines of text based on a specific keyword

I regularly extract lines of text from files based on the presence of a particular keyword; I place the extracted lines into another text file. This takes about 2 hours to complete using the "sort" command then Kate's find & highlight facility. I've been reading the forum & googling and can find... (4 Replies)
Discussion started by: DionDeVille
4 Replies

9. Shell Programming and Scripting

merge lines into single line based on symbol \t

The symbols are \t and \t\t (note: not tab) If the line starts with \t merge them into a single line upto symbol \t\t \t\t to end and start new line I able to join in a single line but not ending at \t\t and I completely confused help would be appreciated:b::D Input \ta tab XXXXXXXXXX \te... (5 Replies)
Discussion started by: repinementer
5 Replies

10. Shell Programming and Scripting

Merge lines in Flat file based on first 5 characters

Hi I have the fixed width flat file having the following data 12345aaaaaaaaaabbbbbbbbbb 12365sssssssssscccccccccc 12365sssss 12367ddddddddddvvvvvvvvvv 12367 vvvvv Here the first column is length 5 second is length 10 third is length 10 if the second or third column exceeds... (3 Replies)
Discussion started by: Brado
3 Replies
Login or Register to Ask a Question