Visit Our UNIX and Linux User Community


Using Perl to Merge Multiple Lines in a File


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using Perl to Merge Multiple Lines in a File
# 1  
Old 10-23-2009
Using Perl to Merge Multiple Lines in a File

I've hunted and hunted but nothing seems to apply to what I need. Any help will be much appreciated!

My input file looks like (Unix):
marker,allele1,allele2
RS1002244,1,1
RS1002244,1,3
RS1002244,3,3
RS1003719,2,2
RS1003719,2,4
RS1003719,4,4

Most markers are listed 3 times but a few have 3 alleles and are listed more.

An example of a marker with 3 alleles is:
marker,allele1,allele2
RS757210,2,2
RS757210,2,3
RS757210,2,4
RS757210,3,3
RS757210,3,4
RS757210,4,4

I would like to get output like:
marker,allele1,allele2,allele3
RS1002244,1,3,.
RS1003719,2,4,,
RS757210,2,3,4

Thanks very much in advance, Peggy 10/23 (happy to have this in formats other than Perl, too)
# 2  
Old 10-23-2009
Not sure about your requirements, but this awk may help:
Code:
NR == 1 { print "marker,allele1,allele2,allele3" }
NR > 1 {
  if ($1 != lm) {
    if (lm != "") print substr(",,,", 1, 3-c)
    printf("%s,%s", $1, $2)
    c = 1
    split("", seen)
    seen[$2]
    lm = $1
  }
  if (!($2 in seen)) { printf(",%s", $2); ++c; seen[$2] }
  #if (!($3 in seen)) { printf(",%s", $3); ++c; seen[$3] }
}
END { print substr(",,,", 1, 3-c) }

# 3  
Old 10-24-2009
Using Perl to Merge Multiple Lines in a File

Thanks very much! Very close and give me new stuff to learn and play with.

Previous Thread | Next Thread
Test Your Knowledge in Computers #5
Difficulty: Easy
The UNIX trademark is a "certification mark".
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge multiple lines into a single line

Hi all, I'm relatively new to scripting, I can do pretty basic things. I have a daily log file that looks like: timestamp=2017-06-28-01.01.35.080576; event status=0; userid=user1; authid=user1; application id=10.10.10.10.11111.12345678901; application name=GUI; ... (29 Replies)
Discussion started by: dwdnet
29 Replies

2. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

3. Shell Programming and Scripting

Merge multiple lines in one line

Hi guys, So i have a input file with several sequences aligned (fasta) >NC_005930 241 bp MNMINIFIINNIFDQFIPVKLSIFSLTSVGSIIA LSWVWINTKTHWAISRSNTP-SLLLNSL WTLLITNL-NEKTNPWAPWLFSLFLLCFSFNIMSLI-PYTF-SQ TSHLSFTFGLSLPIWIMVNIAGFKNNWKKKISHLLPQGTPIYLVPVMII IETISLFIQPLTLGFRLGANLLAGHLLIFLCSCTIWE... (6 Replies)
Discussion started by: andreia
6 Replies

4. Shell Programming and Scripting

Merge multiple lines in same file with common key using awk

I've been a Unix admin for nearly 30 years and never learned AWK. I've seen several similar posts here, but haven't been able to adapt the answers to my situation. AWK is so damn cryptic! ;) I have a single file with ~900 lines (CSV list). Each line starts with an ID, but with different stuff... (6 Replies)
Discussion started by: protosd
6 Replies

5. Shell Programming and Scripting

Combine multiple unique lines from event log text file into one line, use PERL or AWK?

I can't decide if I should use AWK or PERL after pouring over these forums for hours today I decided I'd post something and see if I couldn't get some advice. I've got a text file full of hundreds of events in this format: Record Number : 1 Records in Seq : ... (3 Replies)
Discussion started by: Mayday22
3 Replies

6. Shell Programming and Scripting

Search and swap multiple lines in file using Perl

Hi all, I have a vcd file with a bunch of lines containing an array, like this $var wire 1 b a $end $var wire 1 c a $end $var wire 1 d a $end $var wire 1 e a $end $var wire 1 f b $end $var wire 1 g b $end $var wire 1 h b $end $var wire 1 i b $end I want it like this: $var wire 1 e a... (12 Replies)
Discussion started by: veerabahu
12 Replies

7. Shell Programming and Scripting

merge multiple tables with perl

Hi everyone, I once again got stuck with merging tables and was wondering if someone could help me out on that problem. I have a number of tab delimited tables which I need to merge into one big one. All tables have the same header but a different number of rows (this could be changed if... (6 Replies)
Discussion started by: TuAd
6 Replies

8. Shell Programming and Scripting

Removing end of line to merge multiple lines

I'm sure this will be an easy question for you experts out there, but I have been searching the forum and working on this for a couple hours now and can't get it right. I have a very messy data file that I am trying to tidy up - one of the issues is some records are split into multiple lines: ... (4 Replies)
Discussion started by: tink
4 Replies

9. Shell Programming and Scripting

merge multiple lines from flat file

Hi, I have a tab delimited flat file like this: 189 Guide de lutilisateur sur lappel conférence à trois au moyen d'adaptateurs téléphoniques <TABLE><TBODY><TR><TD><DIV class=subheader>La fonction Appel conférence à trois </DIV></TD> \ <TD><?php print $navTree;?> vous permet de tenir un appel... (4 Replies)
Discussion started by: hnhegde
4 Replies

10. Shell Programming and Scripting

Use sed to merge multiple lines

Hi all: I have a file in which the contents are as following: ... This is a test ONE TWO Hello, world! XXX YYY CCC test again three, four five six seven world AAA BBB QQQ test eight, nine world (3 Replies)
Discussion started by: xb88
3 Replies

Featured Tech Videos