Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Merge data in lines from same file
Top Forums Shell Programming and Scripting Merge data in lines from same file Post 302993667 by Scrutinizer on Monday 13th of March 2017 10:21:17 AM
Old 03-13-2017
Hi, try:
Code:
awk '{i=$1 FS $2} !B[$0]++{A[i]=A[i] (A[i]?",":x) $3; C[i]=$4} END{for(i in A) print i,A[i],C[i]}' FS=\| OFS=\| file


--
Code:
awk '
  {
    i=$1 FS $2                     # Set index to $1 "|" $2
  }
  !B[$0]++ {                       # Only execute if line has not occurred before. Discard duplicate lines, using independent array B 
    A[i]=A[i] (A[i]?",":x) $3      # Add $3 to the string if A[i] is not empty, the prepend a comma else prepend nothing (x) 
    C[i]=$4                        # Add the fourth field to a entry with index i in separate array C
  }
  END {
    for(i in A) print i,A[i],C[i]  # For all found indices where the entire line is no duplicate, print the index "|" concatenated field 3 "|" C[i]
  }
' FS=\| OFS=\| file

Output:
Code:
Eric.W|2/17/1999|817-555-5001,902-555-5999,718-444-5111|Seattle
Doug.G|3/12/2011|817-555-5555,817-555-5522|Portland
Steve.F|1/11/2007|817-555-5111|Portland

Last edited by Scrutinizer; 03-13-2017 at 11:33 AM..
This User Gave Thanks to Scrutinizer For This Post:
cdubu2
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge multiple lines from flat file

Hi, I have a tab delimited flat file like this: 189 Guide de lutilisateur sur lappel conférence à trois au moyen d'adaptateurs téléphoniques <TABLE><TBODY><TR><TD><DIV class=subheader>La fonction Appel conférence à trois </DIV></TD> \ <TD><?php print $navTree;?> vous permet de tenir un appel... (4 Replies)
Discussion started by: hnhegde
4 Replies

2. Shell Programming and Scripting

Merge 2 lines in file

Hi All, I have a data in flat file like below. Some of the information are in second row. 111_ABCProcess ----- ----- IN 0/0 111_PQRTrimPRocess ----- ----- OI 0/0 111_ZigZagTrimProcess ----- ----- ... (1 Reply)
Discussion started by: Amit.Sagpariya
1 Replies

3. Shell Programming and Scripting

Using Perl to Merge Multiple Lines in a File

I've hunted and hunted but nothing seems to apply to what I need. Any help will be much appreciated! My input file looks like (Unix): marker,allele1,allele2 RS1002244,1,1 RS1002244,1,3 RS1002244,3,3 RS1003719,2,2 RS1003719,2,4 RS1003719,4,4 Most markers are listed 3 times but a few... (2 Replies)
Discussion started by: Peggy White
2 Replies

4. Shell Programming and Scripting

Merge lines from one file if pattern matches

I have one comma separated file (a.txt) with two or more records all matching except for the last column. I would like to merge all matching lines into one and consolidate the last column, separated by ":". Does anyone know of a way to do this easily? I've searched the forum but most talked... (6 Replies)
Discussion started by: giannicello
6 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

Merge file lines based off of keyword

Hello Everyone, I have two files I created in a format similar to the ones found below (character position is important): File 1: 21 Cat Y N S Y Y N N FOUR LEGS TAIL WHISKERS 30 Dog N N 1 Y Y N N FOUR LEGS TAIL 33 Fish Y N 1 Y Y N N FINS 43 CAR Y N S Y Y N N WHEELS DOORS... (7 Replies)
Discussion started by: jl487
7 Replies

7. UNIX for Dummies Questions & Answers

Merge rows in bid data file

Dear all, Please help me ,,,, if I have input file like this A_AA960715 leucine-rich repeat-containing protein GO:0006952 defense response P A_AA960715 leucine-rich repeat-containing protein GO:0008152 metabolic process P A_AA960715 leucine-rich... (5 Replies)
Discussion started by: AAWT
5 Replies

8. Shell Programming and Scripting

Merge the data from two servers into a single file

Hi All, Need your inputs for the below. I have 2 different servers 611 & 610, where i would be running two scripts. And would would be running one script from 611 at every 4 hours to merge the data from the 2 servers into 2 files and send a mail. so below is the code snippet for 611: ... (3 Replies)
Discussion started by: ss_ss
3 Replies

9. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

10. Shell Programming and Scripting

How to merge the multiple data files as a single file?

Hi Experts, I have created multiple scripts and send the output to new file, getting this output to my mailbox on daily basis. I would like to send the all outputs to a single file, need to merge all file outputs on a single file. For example, Created script for df -h > df.doc grep... (7 Replies)
Discussion started by: seenuvasan1985
7 Replies

LEARN ABOUT LINUX

git-fmt-merge-msg

GIT-FMT-MERGE-MSG(1)						    Git Manual						      GIT-FMT-MERGE-MSG(1)

NAME

       git-fmt-merge-msg - Produce a merge commit message

SYNOPSIS

       git fmt-merge-msg [-m <message>] [--log[=<n>] | --no-log] <$GIT_DIR/FETCH_HEAD
       git fmt-merge-msg [-m <message>] [--log[=<n>] | --no-log] -F <file>

DESCRIPTION

       Takes the list of merged objects on stdin and produces a suitable commit message to be used for the merge commit, usually to be passed as
       the <merge-message> argument of git merge.

       This command is intended mostly for internal use by scripts automatically invoking git merge.

OPTIONS

       --log[=<n>]
	   In addition to branch names, populate the log message with one-line descriptions from the actual commits that are being merged. At most
	   <n> commits from each merge parent will be used (20 if <n> is omitted). This overrides the merge.log configuration variable.

       --no-log
	   Do not list one-line descriptions from the actual commits being merged.

       --[no-]summary
	   Synonyms to --log and --no-log; these are deprecated and will be removed in the future.

       -m <message>, --message <message>
	   Use <message> instead of the branch names for the first line of the log message. For use with --log.

       -F <file>, --file <file>
	   Take the list of merged objects from <file> instead of stdin.

CONFIGURATION

       merge.branchdesc
	   In addition to branch names, populate the log message with the branch description text associated with them. Defaults to false.

       merge.log
	   In addition to branch names, populate the log message with at most the specified number of one-line descriptions from the actual
	   commits that are being merged. Defaults to false, and true is a synonym for 20.

       merge.summary
	   Synonym to merge.log; this is deprecated and will be removed in the future.

SEE ALSO

       git-merge(1)

GIT

       Part of the git(1) suite

Git 1.8.5.3							    01/14/2014						      GIT-FMT-MERGE-MSG(1)
All times are GMT -4. The time now is 01:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy