Sponsored Content
Top Forums Shell Programming and Scripting Gawk / Awk Merge Lines based on Key Post 302466934 by Jamesfirst on Wednesday 27th of October 2010 06:13:54 PM
Old 10-27-2010
Gawk / Awk Merge Lines based on Key

Hi Guys,

After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines.

I'm an SEO Consultant and provide monthly reports to clients which I'm currently revamping. Essentially I have a csv file similar to
domain.com, domain.com/page.html
domain.com, domain.com/data/product1.html
domain2.com, domain2.com/contact.html
domain2.com, domain2.com/index.html
domain2.com, domain2.com/products/shoes.html
I'm trying to create a file like
domain.com, page.html, product1.html
domain2.com, contact.html, index.html, shoes.html
Each website will have a different number of pages from 1 to 10.

Anyone have any idea how I could do this with Gawk?

Thanks,

James
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge files based on key

Hi Friends, Can any one help me with merging these file based on two columns : File1: A|123|99|SAMS B|456|95|GEORGE D|789|85|HOVARD File2: S|123|99|NANcY|6357 S|123|99|GREGRO|83748 A|456|95|HARRY|827|somers S|456|95|ANTONY|546841|RUDOLPH|7263 B|456|95|SMITH|827|BOISE STATE|834... (3 Replies)
Discussion started by: sbasetty
3 Replies

2. Shell Programming and Scripting

Merge two files based on a 3rd key file

Hi, I want to merge the two files based on the key file's columns. The key file: DATE~DATE HOUSE~IN_HOUSE CUST~IN_CUST PRODUCT~PRODUCT ADDRESS~CUST_ADDR BASIS_POINTS~BASIS_POINTS ... The other 2 files are From_file & To_file - The From_file: DATE|date/time|29|9 ... (9 Replies)
Discussion started by: dips_ag
9 Replies

3. Shell Programming and Scripting

Merge file lines based off of keyword

Hello Everyone, I have two files I created in a format similar to the ones found below (character position is important): File 1: 21 Cat Y N S Y Y N N FOUR LEGS TAIL WHISKERS 30 Dog N N 1 Y Y N N FOUR LEGS TAIL 33 Fish Y N 1 Y Y N N FINS 43 CAR Y N S Y Y N N WHEELS DOORS... (7 Replies)
Discussion started by: jl487
7 Replies

4. Shell Programming and Scripting

How to merge lines based off of text?

Hello Everyone, I have two files, similar to the following: File 1: 8010 ITEM01 CODE1 FLAG1 filler filler 7020 OBJECT CODE2 FLAG2 filler 6010 THING1 CODE4 FLAG4 6011 ITEM20 CODE7 FLAG7 File 2 contains: 6020 ITEM01 CODEA FLAGA filler filler filler 7000 OBJECT CODEB... (2 Replies)
Discussion started by: jl487
2 Replies

5. Shell Programming and Scripting

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! ;) I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff... (6 Replies)
Discussion started by: protosd
6 Replies

6. Shell Programming and Scripting

Need to merge lines based on pattern

Hi, I have a requirement to merge multiple lines based on search pattern. The search criteria is : it will search for CONSTRAINT and when it found CONSTRAINT, it will merge all lines to 1 line till it founds blank line. For Example: CREATE TABLE "AMS_DISTRIBUTOR_XREF" ( "SOURCE"... (5 Replies)
Discussion started by: satyaatcgi
5 Replies

7. Shell Programming and Scripting

Merge mutiple lines into one based on if the first word is some particular value

Hi, trying to knock something together to create one line entries based on whether the first word on each line matches a particular value. eg. Link,"Name=""Something\something"","Timeout=""1800""", "Target=""\\thing\thing\thing""","State=""ONLINE""",something,... (0 Replies)
Discussion started by: adamdb
0 Replies

8. Shell Programming and Scripting

Merge lines based on match

I am trying to merge two lines to one based on some matching condition. The file is as follows: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow ... (8 Replies)
Discussion started by: jamie_123
8 Replies

9. UNIX for Dummies Questions & Answers

Merge selective columns from files based on common key

Hi, I am trying to selectively merge two files based on keys reported in the 1st column. File1: #file1-header1 file1-header2 111 qwe rtz uio 198 asd fgh jkl 165 yxc 789 poi uzt rew 89 lkj File2: #file2-header2 file2-header2 165 ghz nko2 ... (2 Replies)
Discussion started by: dovah
2 Replies

10. Shell Programming and Scripting

awk - Merge two files based on one key

Hi, I am struggling with the an awk command to merge two files based on a common key. I want to append the value from File2 ($2) onto the end of File1 where $1 from each file matches - If no match then nothing is apended File1 COL1|COL2|COL3|COL4|COL5|COL6|COL7... (3 Replies)
Discussion started by: Ads89
3 Replies
runacct(1M)															       runacct(1M)

NAME
runacct - run daily accounting SYNOPSIS
[mmdd[state]] DESCRIPTION
runacct is the main daily accounting shell procedure. It is normally initiated via cron(1M). runacct processes connect, fee, disk, and process accounting files. It also prepares summary files for prdaily or billing purposes. runacct takes care not to damage active accounting files or summary files in the event of errors. It records its progress by writing descriptive diagnostic messages into When an error is detected, a message is written to mail (see mail(1), mailx(1), or elm(1)) is sent to and and runacct terminates. runacct uses a series of lock files to protect against re-invocation. The files and are used to prevent simultaneous invocation, and is used to prevent more than one invocation per day. runacct breaks its processing into separate, restartable states using to remember the last state completed. It accomplishes this by writ- ing the state name into runacct then looks in to see what it has done and to determine what to process next. states are executed in the following order: Move active accounting files into working files. Verify integrity of file, correcting date changes if necessary. Produce connect session records in format. Convert process accounting records into format. Merge the connect and process accounting records. Convert output of chargefee into format and merge with connect and process accounting records. Merge disk accounting records with connect, process, and fee accounting records. Merge the daily total accounting records in with the summary total accounting records in Produce command summaries. Any installation-dependent accounting programs can be included here. Cleanup temporary files and exit. To restart runacct after a failure, first check the file for diagnostics, then fix up any corrupted data files such as or The files and file must be removed before runacct can be restarted. The argument mmdd is necessary if runacct is being restarted, and specifies the month and day for which runacct will rerun the accounting. Entry point for processing is based on the contents of to override this, include the desired state on the command line to designate where processing should begin. EXAMPLES
To start runacct. To restart runacct. To restart runacct at a specific state. WARNINGS
Normally it is not a good idea to restart runacct in its state. Run manually, then restart via: If runacct failed in its state, remove the last file because it will not be complete. FILES
SEE ALSO
mail(1), acct(1M), acctcms(1M), acctcom(1M), acctcon(1M), acctmerg(1M), acctprc(1M), acctsh(1M), cron(1M), fwtmp(1M), acct(2), acct(4), utmp(4). STANDARDS CONFORMANCE
runacct(1M)
All times are GMT -4. The time now is 04:17 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy