Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Joining broken lines and removing empty lines

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Joining broken lines and removing empty lines
# 1  
Old 11-17-2014
Question Joining broken lines and removing empty lines
Hi - I have req to join broken lines and remove empty lines but should NOT be in one line. It has to be as is line by line. The challenge here is there is no end of line/start of line char.
thanks in advance

Source:-
Code:
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver; |Athens|Georgia|null
2003-04-34025|04-10-2003|Product Liability|Lawsuit|12-06-1999|Rollover:severe head injury / comatose
 
 Plaintiff |Austin|Texas|Yes

Target:-
Code:
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver; |Athens|Georgia|null
2003-04-34025|04-10-2003|Product Liability|Lawsuit|12-06-1999|Rollover:severe head injury / comatose Plaintiff |Austin|Texas|Yes

Last edited by Scrutinizer; 11-18-2014 at 02:18 AM.. Reason: CODE TAGS
# 2  
Old 11-17-2014
Is there a fixed number of fields per record?
Will there always be a newline at the end of the last field?
# 3  
Old 11-18-2014
@derekludwig - Yes , it is fixed number of fields & new line at end of line

---------- Post updated at 11:24 PM ---------- Previous update was at 11:23 PM ----------

Source:-
Code:
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bagriver; |Athens|Georgia|null
2003-04-34025|04-10-2003|Product Liability|Lawsuit|12-06-1999|Rollover:severe head injury / comatose

Plaintiff |Austin|Texas|Yes

Target:-
Code:
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bagriver; |Athens|Georgia|null
2003-04-34025|04-10-2003|Product Liability|Lawsuit|12-06-1999|Rollover:severe head injury / comatose Plaintiff |Austin|Texas|Yes

Last edited by Scrutinizer; 11-18-2014 at 02:19 AM.. Reason: CODE tags
# 4  
Old 11-18-2014
Try:
Code:
awk -v f=9 '{while(NF<f && (getline n)>0) $(NF+1)=n}1' FS=\| OFS=\| file

# 5  
Old 11-18-2014
Question
Thanks for the code. But I have other valid records with NF =9, that should not be joined with this. So when I ran your cmd, lines after the bottom most line in my prev note gets appended with next line. sorry f I confused. pasting extension of source lines.

Code:
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver; |Athens|Georgia|null
2003-04-34025|04-10-2003|Product Liability|Lawsuit|12-06-1999|Rollover:severe head injury / comatose
 
 Plaintiff |Austin|Texas|Yes
2003-04-34099|04-20-2013|Claims|Claim|01-13-2003|Air Bag:Driver; |Trenton|Jersey|null

thnks again..
Last edited by Don Cragun; 11-18-2014 at 03:31 AM.. Reason: Add CODE tags.
# 6  
Old 11-18-2014
While it's not a one-liner, here's a perl solution:
Code:
use strict;
use warnings;

my $N = 9;

$, = '|';
$\ = "\n";

my $L = ''; # concatenated line
my @F = (); # fields

while (<>) {
    chomp;
    $L .= $_;
    @F = split '\|', $L;
    next if @F < $N;
    print @F;
    $L = '';
}

$, = ': ';
print '?incomplete', $L if $L ne '';

which produces:
Code:
2003-04-34024|04-10-2003|Claims|Claim|01-13-2003|Air Bag:Driver; |Athens|Georgia|null
2003-04-34025|04-10-2003|Product Liability|Lawsuit|12-06-1999|Rollover:severe head injury / comatose  Plaintiff |Austin|Texas|Yes
2003-04-34099|04-20-2013|Claims|Claim|01-13-2003|Air Bag:Driver; |Trenton|Jersey|null

# 7  
Old 11-18-2014
Yes that was not OK. This should work better:
Code:
awk -v f=9 '{while(NF<f && (getline n)>0) $0=$0 n}1' FS=\| OFS=\| file
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Joining especific lines in "2n" lines file

Hi to everybody. I have a "2n" lines file. I would like to create a new file with only "n" lines, each line in the new file formed by the proper odd line of the old file joined with the following even line (separated by a space) of the old file. I'd prefer using sed or bash. -example-... (5 Replies)
Discussion started by: felino
5 Replies

2. Shell Programming and Scripting

Joining lines in different way

Hi all, I'm excited to the part of unix.com forum, and noob to it. I have an query, where I have an file and it contains data like this use thread when posting do no I was expecting the result as use thread thread when when posting posting do do no use thread when thread when... (6 Replies)
Discussion started by: Jose Nirmal
6 Replies

3. Shell Programming and Scripting

Joining broken lines with awk or perl

Hi, I have a huge file with sql broken statements like: PP3697HB @@@@0 <<<<<<Record has been deleted as per PP3697HB>>>>>> FROM sys.xtab_ref rc,sys.xtab_sys f,sys.domp ur WHE RE rc.milf = ur.milf AND rc.molf = f.molf AND ur.dept = 'SWIT'AND ur .department = 'IND' AND share = '2' AND... (4 Replies)
Discussion started by: som.nitk
4 Replies

4. UNIX for Dummies Questions & Answers

Removing empty lines at the end of a Tab-delimited file

I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs. I've tried may things, here are a couple: sed /^\t.\t/d File1 > File2 sed /^\t{44}/d File1 > File2 What am I missing? (9 Replies)
Discussion started by: SirHenry1
9 Replies

5. Shell Programming and Scripting

Joining broken lines

I have a plain test file with a delimeter ''. In this file some lines are broken into two. The first part of these broken line will have 6 columns and the second part will have 4. These broken lines will be consicutive. I want to join the two consicutive lines which are having 6 fields and 4... (8 Replies)
Discussion started by: ratheeshjulk
8 Replies

6. Shell Programming and Scripting

pattern matching lines using the date, and then joining the lines

Hi Guys, Was trying to attempt the below using awk and sed, have no luck so far, so any help would be appreciated. Current Text File: The first line has got an "\n", and the second line has got spaces/tabs then the word and "\n" TIME SERVER/CLIENT TEXT... (6 Replies)
Discussion started by: eo29
6 Replies

7. Shell Programming and Scripting

Need help joining lines

Hi All, I need the command to join 2 lines into one. I found lots of threads but none give me the sollution. Probably because unix scripting is one of my best features ;) I got a logfile where line 2 needs to be joined with line 1, lines 4 needs to be joined with line 3 etc If you need... (16 Replies)
Discussion started by: rene21976
16 Replies

8. Shell Programming and Scripting

joining two lines

Hi , I want to join two lines in a file, where the second line contain query string. if it doesn't contain that string i don't want to join e.g. Input file is as following: name fame game none none none name fame game cat eat mice I need output file as name fame game none none... (2 Replies)
Discussion started by: ashrafonics
2 Replies

9. Shell Programming and Scripting

Removing empty lines(space) between two lines containing strings

Hi, Please provide shell script to Remove empty lines(space) between two lines containing strings in a file. Input File : A1/EXT "BAP_BSC6/07B/00" 844 090602 1605 RXOCF-465 PDTR11 1 SITE ON BATTERY A2/EXT... (3 Replies)
Discussion started by: sudhakaryadav
3 Replies

10. Shell Programming and Scripting

Joining lines from two files - please help

Hello, I have 2 files say File 1 has ABC DEF GHI File 2 has 123 456 789 I need output as ABC 123 DEF 456 GHI 789 I tried awk and sed but not able to get it in the right way. :confused: Please help. Thanks (25 Replies)
Discussion started by: chandra004
25 Replies
Login or Register to Ask a Question