Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Questions on removing unexpected line breaks
Top Forums Shell Programming and Scripting Questions on removing unexpected line breaks Post 302697735 by bakunin on Friday 7th of September 2012 09:29:18 AM
Old 09-07-2012
If i have counted correctly you have 15 fields in each line. This means there have to be 14 delimiters - if there are fewer, merge the next line to this, otherwise leave it alone.

The following should do what you want:

Code:
sed -n ':start
        /[^|]*\(|[^|]*\)\{14\}/ !{
              N
              s/\n//
              b start
        }
        p' /path/to/infile > /path/to/outfile

This will even connect lines broken into several pieces, but consecutive lines will have to add up to correct ones, otherwise the script will fail to produce correct results.

That is, if a line with 14 fields is followed by a line with 16 fields, it will produce one line with 30 fields, not two with 15 fields each.

If i have miscounted the fields or your file format changes, you can correct this in the counter "\{n\}", which repeats the previous expression "\(|[^|]*\)" (delimiter, followed by optional non-delimiter) n times.

I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing line breaks from a shell variable

Here is my snippet of code... getDescription() { DESCRIPTION=$(dbaccess dncsdb - << ! 2>/dev/null|sed -e 's/hctt_description//' -e '/^$/ d'|tr -d '\r' select hct_type.hctt_description from hct_type,hct_profile where hct_type.hctt_id=hct_profile.hctt_id and... (5 Replies)
Discussion started by: lyonsd
5 Replies

2. Shell Programming and Scripting

any better way to remove line breaks

Hi, I got some log files which print the whole xml message in separate lines: e.g. 2008-10-01 14:21:44,561 INFO do something 2008-10-01 14:21:44,561 INFO print xml : <?xml version="1.0" encoding="UTF-8"?> <a> <b>my data</b> </a> 2008-10-01 14:21:44,563 INFO do something again I want... (3 Replies)
Discussion started by: csmklee
3 Replies

3. Shell Programming and Scripting

Help with wc and line breaks

Hi everyone, I have gone through the forum trying to find an answer to this question but was unsuccessful. I am hoping that someone can help me with this please. I am trying to get my script to recognise line breaks from a file and to give me a result for wc of each line. So basically, if you... (7 Replies)
Discussion started by: stargazerr
7 Replies

4. Shell Programming and Scripting

Remove line breaks after a match

I need to remove all line breaks in a document after a match, until there is a blank line. Example below, after the match "THE GREEN TABLE" remove line breaks until a blank line. Then, after the match "THE BLUE TABLE" do the same. Before: THE GREEN TABLE Lorem ipsum dolor sit amet,... (14 Replies)
Discussion started by: dockline
14 Replies

5. Programming

Clean and keep line breaks

Hello, I want to keep line spaces in comments but clean more then 2 after each. Example: $sentence="This is my first sentence This will be in a new row This will be too in a new row but not separated with 3line breaks just with one "; And i want to... (1 Reply)
Discussion started by: AimyThomas
1 Replies

6. UNIX for Dummies Questions & Answers

Page breaks and line breaks

Hi All, Need an urgent solution to an issue . We have created a ksh file or shell script which generates 1 DAT file. the DAT file contains extract of a select statement . Now the issue is , when we are executing the ksh file , the output is coimng with page breaks and line breaks . We have... (4 Replies)
Discussion started by: Ayaskant
4 Replies

7. HP-UX

After using @, line breaks for a particular user in shell

Dear Concern, When we using @ sign, line breaks for a particular user in shell. Please advise how to resolve from the problem in HP UX. tabs@tabsdb02:/ccbs/users/tabs$ cat /etc/passwd|grep tabs tabs:RdCgOsmKee7Ps:221:201::/ccbs/users/tabs:/usr/bin/ksh... (3 Replies)
Discussion started by: makauser
3 Replies

8. Shell Programming and Scripting

[BASH] read 'line' issue with leading tabs and virtual line breaks

Heyas I'm trying to read/display a file its content and put borders around it (tui-cat / tui-cat -t(ypwriter). The typewriter-part is a 'bonus' but still has its own flaws, but thats for later. So in some way, i'm trying to rewrite cat using bash and other commands. But sadly it fails on... (2 Replies)
Discussion started by: sea
2 Replies

9. Linux

Line breaks in mail spool

Hi, I have an issue with the line breaks in the mail spool- /var/mail/user1. I have set up a script to go through the mail spool on one of the users and parse some parts of the mail, however there doesn't seem to exist the regular line endings CR, LF or both in the lines and this is breaking my... (4 Replies)
Discussion started by: night_watcher
4 Replies

10. Shell Programming and Scripting

Removing line breaks inside a field

Hi all, I have a csv input file with total 60 fields and the fields are not enclosed with double quotes.One of the field(50th field) in this file has line breaks in it which results in the row getting split into multiple lines.This is causing my load(to table) to fail.I tried to enforce double... (3 Replies)
Discussion started by: Bobby_2000
3 Replies

LEARN ABOUT X11R4

cut

CUT(1)                                                             User Commands                                                            CUT(1)

NAME

       cut - remove sections from each line of files

SYNOPSIS

       cut OPTION... [FILE]...

DESCRIPTION

       Print selected parts of lines from each FILE to standard output.

       With no FILE, or when FILE is -, read standard input.

       Mandatory arguments to long options are mandatory for short options too.

       -b, --bytes=LIST
              select only these bytes

       -c, --characters=LIST
              select only these characters

       -d, --delimiter=DELIM
              use DELIM instead of TAB for field delimiter

       -f, --fields=LIST
              select only these fields;  also print any line that contains no delimiter character, unless the -s option is specified

       -n     (ignored)

       --complement
              complement the set of selected bytes, characters or fields

       -s, --only-delimited
              do not print lines not containing delimiters

       --output-delimiter=STRING
              use STRING as the output delimiter the default is to use the input delimiter

       -z, --zero-terminated
              line delimiter is NUL, not newline

       --help display this help and exit

       --version
              output version information and exit

       Use  one,  and only one of -b, -c or -f.  Each LIST is made up of one range, or many ranges separated by commas.  Selected input is written
       in the same order that it is read, and is written exactly once.  Each range is one of:

       N      N'th byte, character or field, counted from 1

       N-     from N'th byte, character or field, to end of line

       N-M    from N'th to M'th (included) byte, character or field

       -M     from first to M'th (included) byte, character or field

AUTHOR

       Written by David M. Ihnat, David MacKenzie, and Jim Meyering.

REPORTING BUGS

       GNU coreutils online help: <http://www.gnu.org/software/coreutils/>
       Report cut translation bugs to <http://translationproject.org/team/>

COPYRIGHT

       Copyright (C) 2017 Free Software Foundation, Inc.  License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
       This is free software: you are free to change and redistribute it.  There is NO WARRANTY, to the extent permitted by law.

SEE ALSO

       Full documentation at: <http://www.gnu.org/software/coreutils/cut>
       or available locally via: info '(coreutils) cut invocation'

GNU coreutils 8.28                                                 January 2018                                                             CUT(1)
All times are GMT -4. The time now is 11:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy