Merge lines in Flat file based on first 5 characters


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge lines in Flat file based on first 5 characters
# 1  
Old 04-06-2009
Merge lines in Flat file based on first 5 characters

Hi

I have the fixed width flat file having the following data

12345aaaaaaaaaabbbbbbbbbb
12365sssssssssscccccccccc
12365sssss
12367ddddddddddvvvvvvvvvv
12367 vvvvv

Here the first column is length 5 second is length 10 third is length 10

if the second or third column exceeds length it is repeated in next line with the first column repeated

Now I need the file to be modified in such a way that there is only one row for the data like

12345aaaaaaaaaa bbbbbbbbbb
12365sssssssssssssss cccccccccc
12367dddddddddd vvvvvvvvvvvvvvv

changing the lenght of the second and third column to 20 instead of 10.
I am trying to write this in perl or shell.
Kindly help me..
# 2  
Old 04-06-2009
If awk is acceptable (use nawk or /usr/xpg4/bin/awk on Solaris):

Code:
awk 'END {
  print fmt(k, _[k])
  }
NR > 1 && substr($0, 1, 5) != k {
  print fmt(k, _[k])
  }
{
  k = substr($0, 1, 5)
  v = substr($0, 6)
  _[k] = _[k] ? _[k] v : v
  }
func fmt(k, v) { 
  return k  sprintf("%-20s%-20s",\
  substr(v, 1, 10), substr(v, 11))
  }' infile


Last edited by radoulov; 04-06-2009 at 12:18 PM.. Reason: sorry for the multiple edits :)
# 3  
Old 04-06-2009
Thank You for the code.

Is there any way that i could do this in perl.
# 4  
Old 04-06-2009
Something like this:

Code:
perl -ne'
  /(.{5})(.*)/ and $h{$1} .= $2 ;
  print map { 
    $_, sprintf( "%-20s%-20s", $h{$_} =~ /(.{10})(.*)/ ), "\n" 
    } sort { $a <=> $b } keys %h if eof
    ' infile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merge lines with varying characters

Hi, I have a large set of data (firewall logs) that I'm trying to summarize. I've been able to write a script to consolidate the ports, now am looking to conslidate even further, based on IP. Source Destination Type Port 192.168.5.108 192.168.11.12 TCP 1, 2, 3, 4, 5, 15 192.168.5.109... (6 Replies)
Discussion started by: umang2382
6 Replies

2. Shell Programming and Scripting

Merge lines based on match

I am trying to merge two lines to one based on some matching condition. The file is as follows: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow request=<GET: Matches filter: 'request ', timestamp, <HTTPFlow ... (8 Replies)
Discussion started by: jamie_123
8 Replies

3. Shell Programming and Scripting

Merge mutiple lines into one based on if the first word is some particular value

Hi, trying to knock something together to create one line entries based on whether the first word on each line matches a particular value. eg. Link,"Name=""Something\something"","Timeout=""1800""", "Target=""\\thing\thing\thing""","State=""ONLINE""",something,... (0 Replies)
Discussion started by: adamdb
0 Replies

4. Shell Programming and Scripting

Need to merge lines based on pattern

Hi, I have a requirement to merge multiple lines based on search pattern. The search criteria is : it will search for CONSTRAINT and when it found CONSTRAINT, it will merge all lines to 1 line till it founds blank line. For Example: CREATE TABLE "AMS_DISTRIBUTOR_XREF" ( "SOURCE"... (5 Replies)
Discussion started by: satyaatcgi
5 Replies

5. Shell Programming and Scripting

How to merge lines based off of text?

Hello Everyone, I have two files, similar to the following: File 1: 8010 ITEM01 CODE1 FLAG1 filler filler 7020 OBJECT CODE2 FLAG2 filler 6010 THING1 CODE4 FLAG4 6011 ITEM20 CODE7 FLAG7 File 2 contains: 6020 ITEM01 CODEA FLAGA filler filler filler 7000 OBJECT CODEB... (2 Replies)
Discussion started by: jl487
2 Replies

6. Shell Programming and Scripting

Merge file lines based off of keyword

Hello Everyone, I have two files I created in a format similar to the ones found below (character position is important): File 1: 21 Cat Y N S Y Y N N FOUR LEGS TAIL WHISKERS 30 Dog N N 1 Y Y N N FOUR LEGS TAIL 33 Fish Y N 1 Y Y N N FINS 43 CAR Y N S Y Y N N WHEELS DOORS... (7 Replies)
Discussion started by: jl487
7 Replies

7. Shell Programming and Scripting

Sed - merge lines bw 2 specific characters

Hi, I have a bash script and I am looking for a command that will merge specific lines together. Sample Data: registration time = 1300890272 Id = 1 setd = 0 tagunt = 26 tagId=6, length=8, value= tagId=9, length=5, value= tagId=7, length=2, value= tagId=16, length=2, value= tagId=32,... (8 Replies)
Discussion started by: Winsarc
8 Replies

8. Shell Programming and Scripting

Merge lines in text file based on pattern

Hello, I have searched forum trying to find a solution to my problem, but could not find anything or I did not understand the examples.... I should say, I am very inexperienced with text processing. I have a text file with approx 60k lines in it. I need to merge lines based on the number... (8 Replies)
Discussion started by: Bertik
8 Replies

9. Shell Programming and Scripting

merge lines into single line based on symbol \t

The symbols are \t and \t\t (note: not tab) If the line starts with \t merge them into a single line upto symbol \t\t \t\t to end and start new line I able to join in a single line but not ending at \t\t and I completely confused help would be appreciated:b::D Input \ta tab XXXXXXXXXX \te... (5 Replies)
Discussion started by: repinementer
5 Replies

10. Shell Programming and Scripting

merge multiple lines from flat file

Hi, I have a tab delimited flat file like this: 189 Guide de lutilisateur sur lappel conférence à trois au moyen d'adaptateurs téléphoniques <TABLE><TBODY><TR><TD><DIV class=subheader>La fonction Appel conférence à trois </DIV></TD> \ <TD><?php print $navTree;?> vous permet de tenir un appel... (4 Replies)
Discussion started by: hnhegde
4 Replies
Login or Register to Ask a Question