Merge multiple lines in same file with common key using awk Post: 302735689

Sponsored Content

Top Forums Shell Programming and Scripting Merge multiple lines in same file with common key using awk Post 302735689 by protosd on Monday 26th of November 2012 03:22:23 AM

11-26-2012

Registered User

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! Smilie

I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff after it which I want to merge onto a single line. Sometimes there are just a couple of lines to be merge, sometimes there are 4 or 5. The input file is already sorted.
Here's an example:

Code:

2,"URL","website","aaa.com"
2,"abc","text","some text"
2,"Password","password","12345678"
3,"URL","website","10.0.10.75"
3,"uname","text","some text"
3,"Password","password","password"
3,"ulang","text","some text"
4,"URL","website","192.168.2.110"
4,"Login","login","admin"
4,"Password","password","blah-blah"

Output should look like:

Code:

2,"URL","website","aaa.com","abc","text","some text",Password","password","12345678"
3,"URL","website","10.0.10.75","uname","text","some text""Password","password","password","ulang","text","some text"

---------- Post updated at 12:22 AM ---------- Previous update was at 12:11 AM ----------

I forgot to mention, each line in the input file only has 4 fields.

protosd

View Public Profile for protosd

Find all posts by protosd

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge multiple lines from flat file

Hi, I have a tab delimited flat file like this: 189 Guide de lutilisateur sur lappel conf�rence � trois au moyen d'adaptateurs t�l�phoniques <TABLE><TBODY><TR><TD><DIV class=subheader>La fonction Appel conf�rence � trois </DIV></TD> \ <TD><?php print $navTree;?> vous permet de tenir un appel...

2. Shell Programming and Scripting

merge based on common, awk help

All, $ cat x.txt z 11 az x 12 ax y 13 ay $ cat y.txt ay TT ax NN Output required: y 13 ay TT x 12 ax NN

3. Shell Programming and Scripting

Merge lines in a file with Awk - incorrect output

Hi, I would like: FastEthernet0/0 is up, line protocol is up 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored 0 output errors, 0 collisions, 0 interface resets Serial1/0:0 is up, line protocol is up 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 0...

4. Shell Programming and Scripting

Using Perl to Merge Multiple Lines in a File

I've hunted and hunted but nothing seems to apply to what I need. Any help will be much appreciated! My input file looks like (Unix): marker,allele1,allele2 RS1002244,1,1 RS1002244,1,3 RS1002244,3,3 RS1003719,2,2 RS1003719,2,4 RS1003719,4,4 Most markers are listed 3 times but a few...

5. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and...

6. Shell Programming and Scripting

Gawk / Awk Merge Lines based on Key

Hi Guys, After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines. I'm an SEO Consultant and provide...

7. UNIX for Dummies Questions & Answers

Writing a loop to merge multiple files by common column

I have 100 data files labelled 250.1.txt through 250.100.txt. The second column of the data files partially match (there is about %90 overlap). Each data file has 4 columns. I want the merge all these text files by the matching values in the second column. In the output, the first column should...

8. Shell Programming and Scripting

Merge multiple files with common header

Hi all, Say i have multiple files x1 x2 x3 x4, all with common header (date, time, year, age),, How can I merge them to one singe file "X" in shell scripting Thanks for your suggestions.

9. UNIX for Dummies Questions & Answers

Merge selective columns from files based on common key

Hi, I am trying to selectively merge two files based on keys reported in the 1st column. File1: #file1-header1 file1-header2 111 qwe rtz uio 198 asd fgh jkl 165 yxc 789 poi uzt rew 89 lkj File2: #file2-header2 file2-header2 165 ghz nko2 ...

10. Shell Programming and Scripting

Join and merge multiple files with duplicate key and fill void columns

Join and merge multiple files with duplicate key and fill void columns Hi guys, I have many files that I want to merge: file1.csv: 1|abc 1|def 2|ghi 2|jkl 3|mno 3|pqr file2.csv:

LEARN ABOUT DEBIAN

password-gorilla

PASSWORD-GORILLA(1)					      General Commands Manual					       PASSWORD-GORILLA(1)

NAME

       password-gorilla -- a password manager

SYNOPSIS

       password-gorilla

DESCRIPTION

       The  Password  Gorilla  helps  you  manage your logins. It stores all your user names and passwords, along with login information and other
       notes, in a securely encrypted file. A single "master password" is used to protect the file. This way, you only need to remember the single
       master password, instead of the many logins that you use.

       If  you	want to log in to a service or Web site, the Password Gorilla copies your user name and password to the clipboard, so that you can
       easily paste it into your Web browser or other application. Because the password does not appear on the screen, Password Gorilla is safe to
       use in the presence of others.

       The convenience of Password Gorilla allows you to choose different, non-intuitive passwords for each service. An integrated random password
       generator can provide one-time passwords, tunable to various services' policies.

FILES

       The program creates a configuration file .gorillarc in the homedirectory of the user.

REPORTING BUGS

       Bugs can be reported through the Debian Bug Tracking System at http://bugs.debian.org.

AUTHOR

       Author of password-gorilla is Frank Pilhofer <fp@fpx.de>

       This manual page was written by Patrick Schoenfeld <schoenfeld@in-medias-res.com> for the Debian system (but may be used by others).   Per-
       mission	is  granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later
       version published by the Free Software Foundation.

       On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL.

															       PASSWORD-GORILLA(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge multiple lines from flat file

Discussion started by: hnhegde

2. Shell Programming and Scripting

merge based on common, awk help

Discussion started by: jkl_jkl

3. Shell Programming and Scripting

Merge lines in a file with Awk - incorrect output

Discussion started by: mv652

4. Shell Programming and Scripting

Using Perl to Merge Multiple Lines in a File

Discussion started by: Peggy White