Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Gawk / Awk Merge Lines based on Key

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Gawk / Awk Merge Lines based on Key
Prev   Next
# 1  
Old 10-27-2010
Gawk / Awk Merge Lines based on Key
Hi Guys,

After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines.

I'm an SEO Consultant and provide monthly reports to clients which I'm currently revamping. Essentially I have a csv file similar to
domain.com, domain.com/page.html
domain.com, domain.com/data/product1.html
domain2.com, domain2.com/contact.html
domain2.com, domain2.com/index.html
domain2.com, domain2.com/products/shoes.html
I'm trying to create a file like
domain.com, page.html, product1.html
domain2.com, contact.html, index.html, shoes.html
Each website will have a different number of pages from 1 to 10.

Anyone have any idea how I could do this with Gawk?

Thanks,

James
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - Merge two files based on one key

Hi, I am struggling with the an awk command to merge two files based on a common key. I want to append the value from File2 ($2) onto the end of File1 where $1 from each file matches - If no match then nothing is apended File1 COL1|COL2|COL3|COL4|COL5|COL6|COL7... (3 Replies)
Discussion started by: Ads89
3 Replies

2. UNIX for Dummies Questions & Answers

Merge selective columns from files based on common key

Hi, I am trying to selectively merge two files based on keys reported in the 1st column. File1: #file1-header1 file1-header2 111 qwe rtz uio 198 asd fgh jkl 165 yxc 789 poi uzt rew 89 lkj File2: #file2-header2 file2-header2 165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies

3. Shell Programming and Scripting

Merge lines based on match

I am trying to merge two lines to one based on some matching condition. The file is as follows: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow ... (8 Replies)
Discussion started by: jamie_123
8 Replies

4. Shell Programming and Scripting

Merge mutiple lines into one based on if the first word is some particular value

Hi, trying to knock something together to create one line entries based on whether the first word on each line matches a particular value. eg. Link,"Name=""Something\something"","Timeout=""1800""", "Target=""\\thing\thing\thing""","State=""ONLINE""",something,... (0 Replies)
Discussion started by: adamdb
0 Replies

5. Shell Programming and Scripting

Need to merge lines based on pattern

Hi, I have a requirement to merge multiple lines based on search pattern. The search criteria is : it will search for CONSTRAINT and when it found CONSTRAINT, it will merge all lines to 1 line till it founds blank line. For Example: CREATE TABLE "AMS_DISTRIBUTOR_XREF" ( "SOURCE"... (5 Replies)
Discussion started by: satyaatcgi
5 Replies

6. Shell Programming and Scripting

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! ;) I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff... (6 Replies)
Discussion started by: protosd
6 Replies

7. Shell Programming and Scripting

How to merge lines based off of text?

Hello Everyone, I have two files, similar to the following: File 1: 8010 ITEM01 CODE1 FLAG1 filler filler 7020 OBJECT CODE2 FLAG2 filler 6010 THING1 CODE4 FLAG4 6011 ITEM20 CODE7 FLAG7 File 2 contains: 6020 ITEM01 CODEA FLAGA filler filler filler 7000 OBJECT CODEB... (2 Replies)
Discussion started by: jl487
2 Replies

8. Shell Programming and Scripting

Merge file lines based off of keyword

Hello Everyone, I have two files I created in a format similar to the ones found below (character position is important): File 1: 21 Cat Y N S Y Y N N FOUR LEGS TAIL WHISKERS 30 Dog N N 1 Y Y N N FOUR LEGS TAIL 33 Fish Y N 1 Y Y N N FINS 43 CAR Y N S Y Y N N WHEELS DOORS... (7 Replies)
Discussion started by: jl487
7 Replies

9. Shell Programming and Scripting

Merge two files based on a 3rd key file

Hi, I want to merge the two files based on the key file's columns. The key file: DATE~DATE HOUSE~IN_HOUSE CUST~IN_CUST PRODUCT~PRODUCT ADDRESS~CUST_ADDR BASIS_POINTS~BASIS_POINTS ... The other 2 files are From_file & To_file - The From_file: DATE|date/time|29|9 ... (9 Replies)
Discussion started by: dips_ag
9 Replies

10. Shell Programming and Scripting

Merge files based on key

Hi Friends, Can any one help me with merging these file based on two columns : File1: A|123|99|SAMS B|456|95|GEORGE D|789|85|HOVARD File2: S|123|99|NANcY|6357 S|123|99|GREGRO|83748 A|456|95|HARRY|827|somers S|456|95|ANTONY|546841|RUDOLPH|7263 B|456|95|SMITH|827|BOISE STATE|834... (3 Replies)
Discussion started by: sbasetty
3 Replies
Login or Register to Ask a Question

LEARN ABOUT DEBIAN

excel::template::container::loop

Excel::Template::Container::Loop(3pm)			User Contributed Perl Documentation		     Excel::Template::Container::Loop(3pm)

NAME

       Excel::Template::Container::Loop - Excel::Template::Container::Loop

PURPOSE

       To provide looping

NODE NAME

       LOOP

INHERITANCE

       Excel::Template::Container

ATTRIBUTES

       o   NAME

	   This is the name of the loop. It's used to identify within the parameter set what variables to expose to the children nodes each
	   iteration.

CHILDREN

       None

EFFECTS

       None

DEPENDENCIES

       None

USAGE

	 <loop name="LOOPY">
	   ... Children here ...
	 </loop>

       In the above example, the children nodes would have access to the LOOPY array of hashes as parameters. Each iteration through the array
       would expose a different hash of parameters to the children.

       These loops work just like HTML::Template's loops. (I promise I'll give more info here!)

       There is one difference - I prefer using Perl-like scoping, so accessing of variables outside the LOOP scope from within is perfectly
       acceptable. You can also hide outside variables with inner values, if you desire, just like Perl.

AUTHOR

       Rob Kinyon (rob.kinyon@gmail.com)

SEE ALSO

perl v5.14.2							    2012-04-29				     Excel::Template::Container::Loop(3pm)