Need to concatenate spuriously separated lines


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need to concatenate spuriously separated lines
# 15  
Old 12-27-2011
Quote:
Originally Posted by lemele
gersh99,

upon further examination of the input file I have encountered spurious new lines that were not anchored to pattern "^|". In such cases, your "nawk" one-liner choked with the following message:

awk: 0602-565 There are not enough parameters in printf statement Both Eyes VA TA Dilate: 1%M Dilate: 2.5%N Contacts: previous Indication: Exam Eye [V72.0]|N||15589218|2011.11.1611:01:16:436||||0.
The input line number is 1742.
The source line number is 1.

The task at hand is to find (and replace) any spurious new lines NOT IMMEDIATELY preceding the integer 10-digit primary key

Please help.

lemele
Interesting - works just fine with AiX's awk.
Try:
Code:
nawk -F'|' '$1~"^[0-9][0-9]*$" && length($1)==10 {printf("%c%s", (FNR==1)?"":ORS,$0);next}{printf("%s", $0)}' myFile

# 16  
Old 12-27-2011
need to concatenate spuriously separated lines

gersh99,

I tried the modified version of your "nawk" one-liner and it worked. Interestingly, the two files that are needed for comparison come from 2 RDBMS's (DB2 & Sybase), the resulting data files have different "number" of lines in them (even though the number of primary keys in the original SQL files is the same); this signifies that the spuriousness of the new line characters is severe and unpredictable.

Thank you so much for your continued help.

Lemele
# 17  
Old 12-28-2011
My thought:

Code:
 
#!/usr/bin/perl
my ($dataline, $record, $buildline);
$buildline = "";
$dataline = 0;
open(INFILE, "<", "datafile.txt") or die "Cannot open datafile.txt: $!";
while( $record = <INFILE>) {
       chomp($record);
       $dataline++;
       if ($record =~ m/^\d.*0$/) {
          # full line
          if ($buildline ne "") {         # Was there partial previous line?
             print "$buildline\n";
          }
          print "$record\n";
          $buildline = "";            # Reset line buffer for next fetch
       }
       else {
            $buildline = $buildline . $record;
       }
}
close(INFILE);

dc++
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Concatenate three lines into one

Hi. I'm new to this forum. I am attempting to parse an Audit Log from Cognos/TM1, selecting only Event IDs of "client" which are found on the "start-tag" record. These Logs are in a pseudo-XML format but not a true XML format. I want to FTP an Audit Log File from the Cognos server to our UNIX... (7 Replies)
Discussion started by: FredAtArrow
7 Replies

2. Shell Programming and Scripting

Count lines separated by new line

Hi guys, I have a file which has random records like mentioned below emcpower28a pci@3,03 (disk physical name) pci@3,04 emcpower9a pci@1,03 pci@2,03 pci@3,01 pci@4,03 there could be any number of disk names for any LUN (emc...) So, I want a solution to count disk names for its... (12 Replies)
Discussion started by: prashant2507198
12 Replies

3. Shell Programming and Scripting

grep lines separated with semicolon

Hello, I would like to kindly ask you for help. I have a file with some lines in one row separated by semicolon. I need to find out, if the line I have in different variable is included in this file. e.g I have a file foo.txt with lines A=hello there;hello world;hello there world In... (6 Replies)
Discussion started by: satin1321
6 Replies

4. Programming

Concatenate two lines in a fIle

Hi All, Can any one help me in finding the solution for concatenating two or more lines in a file and writing them to a temporary file. for Example: He is a wise student. So he got first rank. This is in a file i want the output as He is a wise student so he got first rank. into a file... (3 Replies)
Discussion started by: uday.sena.m
3 Replies

5. Shell Programming and Scripting

Grep and print next 10 Lines separated by ,

Hi All, I need to grep through a file for a string and print the next ten lines to a file separating the lines with a , and save it as a csv file to open it as a XL file. The 10 lines should be on a sigle row in xl. Any suggesstions please. Note; I dont have a GNU Grep to use -A flag. ... (6 Replies)
Discussion started by: Nani369
6 Replies

6. Shell Programming and Scripting

Concatenate lines between lines starting with a specific pattern

Hi, I have a file such as: --- >contig00001 length=35524 numreads=2944 gACGCCGCGCGCCGCGGCCAGGGCTGGCCCA CAGGCCGCGCGGCGTCGGCTGGCTGAG >contig00002 length=4242 numreads=43423 ATGCCGAAGGTCCGCCTGGGGCTGG CGCCGGGAGCATGTAGCG --- I would like to concatenate the lines not starting with ">"... (9 Replies)
Discussion started by: s052866
9 Replies

7. Shell Programming and Scripting

concatenate lines in pairs

Hi, I have a text file with the following contents /C=IT/O=INFN/OU=Personal Certificate/L=Napoli/CN=Some guy /C=IT/O=INFN/CN=INFN CA /O=Grid/O=NorduGrid/OU=uninett.no/CN=Another guy /O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority /C=TW/O=AP/OU=GRID/CN=Someone else... (5 Replies)
Discussion started by: kerl
5 Replies

8. Shell Programming and Scripting

Concatenate two lines in one

Does anyone know a way I can fix my file below where it has lines that are cut? Just like the (notice the space before ms,c=PH).. dn: cn=english,ou=Messaging,ou=lang,o=subject,t=j ms,c=PH which should be dn: cn=english,ou=Messaging,ou=lang,o=subject,t=jms,c=PH The whole text file... (5 Replies)
Discussion started by: Orbix
5 Replies

9. UNIX Desktop Questions & Answers

How to concatenate consecutive lines

I have a few lines like -- feature 1, subfeat 0, type 3, subtype 1, value 0, -- feature 1, subfeat 0, type 1, subtype 1, value 0, I would like to concatenate the... (1 Reply)
Discussion started by: shivi707
1 Replies

10. Shell Programming and Scripting

two lines into one colon separated line...

Does anyone know how to get these two output lines into one colon ':' separated line with some unix command? Maybe nawk. I've tried to read the nawk and awk man pages but I don't get it right. Are these commands the one to use? Output from find command: # /sw/tools/matlab/7.0.1/man... (2 Replies)
Discussion started by: tonlu
2 Replies
Login or Register to Ask a Question