Shell Programming and Scripting

Above works well for the one record sample in post#1 as indicated. If you want to apply it to multiple records, try a small adaption (still far from bullet proof):

Code:

awk -F\| '{while (NF < 15 && getline s>0) $0=$0 RS s}1'  file

And, as stated before in this thread, a more precise and detailed specification would help to taylor a solution for you

Yes indeed that is just silly. Thanks RudiC. That is of course how it should be (but then RS should not be there)

Code:

awk -F\| '{while (NF < 15 && getline s>0) $0=$0 s}1' file

or the 2nd example with RS maintained:

Code:

awk -F\| '{while(NF<15 && getline s>0) $0=$0 RS s}{gsub(/^"|"$/,x,$10); print $10}'

If the "records" are not only defined as an amount of a fixed number of fields but also being of fixed length (each record has the same number of characters) and, of course, if the line breaks are real UNIX-linefeeds instead of may DOS-CR/LFs or whatever - then this is the standard textbook application for the fmt utility, no?

Code:

fmt -<number of characters the record is supposed to have> /path/to/file

I hope this helps.

bakunin

Hi All

Sorry for not providing the correct information.Please find the details below of my input file.I am getting encrypted file where I used to decrypt and do a DOS2UNIX conversion.Once I did that I will load it using my job

File Details
-------------
1. Line Delimiter is \n
2. Record Delimiter is |
3. Line Delimiter on the split lines is same as in a correctly formatted line.
4. In my output file I need the same line delimiter as comes in the input file which is \n
5. The number of fields are same for all the records which is 15 and all the column values are coming with double quotes.
6. Is an invalid input line ALWAYS split between a pair of double quotes? Can we assume that a line needs to be combined with the next line from the input file if and only if an input line does not end with a double quote character?--We cant do like that as all the records are coming with double quotes.
7. Are invalid records always split by converting a space to a newline? How do we know whether or not a character needs to be added when lines are joined? Is a space character ALWAYS supposed to be added when lines are joined?--Space dose not matter, as I just want the split lines to append to a single line

Hope this information will help.I am pretty new to this process,so please let me know if you need more details

If the broken records always break inside a double quoted string as in your sample input, an easy fix is just to use:

Code:

awk '{printf("%s%s", $0, (substr($0, length, 1) == "\"") ? "\n" : "")}' file

If the broken records sometimes break immediately before or after a pipe symbol (as long as it isn't after the 14th pipe symbol, you can use most of the other suggestions in this thread.

You didn't answer my question about operating system and shell. So, if you're using a Solaris/SunOS operating system, you'll need to change the above suggestion to use /usr/xpg4/bin/awk or nawk instead of awk.

Save as cleaner.pl
Run as perl cleaner.pl ginrkf.file > ginrkf.filtered

Code:

#!/usr/bin/env perl

use strict;
use warnings;

my $len = 15;
{
    local $/ = '|';
    my $field;
    while(<>) {
        /\|/ and ++$field;
        s/\n//g unless $field == $len;
        print;
        $field = 0 if $field == $len;
    }
}

Thanks a lot for all the help.Below code is working for my issue

Code:

awk '{printf("%s%s", $0, (substr($0, length, 1) == "\"") ? "\n" : "")}' file