Replace \n char in Data


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Replace \n char in Data
# 1  
Old 11-20-2018
Replace \n char in Data

File is pipe delimited with 17 fields. We may get \n char (1 or more \n in one field or multi fileds) in data in any field.
Need to replace \n in data with space and not the Ture \n that is line separator.

I tried below awk command it did not work as expected.
Code:
awk '(NR-1)%2{$1=$1}1' RS=\| ORS=\| TestInput |egrep -v "^\|$">TestOUT

Input:
Code:
455000|TTTT|97233|UUUUUU REP||Juli||EEEE||P.O. Box 550 MMMMMMMMMM JJ 55555|||||||
333333|DDD|97233|UUUUUU REP||AMAR||AJAY||P.O. Box 69 MMMMMMMMMM JJ 6666 
JJJ BBBB P.O. Box 4 MMMMMMMMMM JJ 44444
delmer Speidel P.O. Box 242 MMMMMMMMMM JJ 99456, See File For More.....|||||||
888888|Director|97382|UUUUUU REP||ANTHONY|K|JOSHI||1144 JNM ROAD LLLLLLLLLLLLL JJ 82513, Laurie Ideker, Leon Sanderson
coralie Emmons P.O. Box 34 LLLLLLLLLLLLL JJ 82513
wanda Knowles P.O. Box 958 LLLLLLLLLLLLL JJ 82513, See File For More...|||||||
999999|President|97692|UUUUUU REP||See||File|||||||||

Expected Output:
Code:
455000|TTTT|97233|UUUUUU REP||Juli||Surwald||P.O. Box 550 MMMMMMMMMM JJ 55555|||||||
333333|DDD|97233|UUUUUU REP||AMAR||AJAY||P.O. Box 69 MMMMMMMMMM JJ 6666 JJJ BBBB P.O. Box 4 MMMMMMMMMM JJ 99456 delmer Speidel P.O. Box 242 MMMMMMMMMM JJ 99456, See File For More.....|||||||
888888|Director|97382|UUUUUU REP||ANTHONY|K|JOSHI||1144 JNM ROAD LLLLLLLLLLLLL JJ 82513, Laurie Ideker, Leon Sanderson coralie Emmons P.O. Box 34 LLLLLLLLLLLLL JJ 82513 wanda Knowles P.O. Box 958 LLLLLLLLLLLLL JJ 82513, See File For More...|||||||
999999|President|97692|UUUUUU REP||See||File|||||||||


Last edited by Scrutinizer; 11-20-2018 at 12:07 PM.. Reason: code tags
# 2  
Old 11-20-2018
Code:
awk -F'|' 'NF==17 {print;next}
             {s=(s)?s $0:$0
               if (split(s,a)==17) {print s;s=""}
             } 
             END {if (s) print s}' myFile

# 3  
Old 11-20-2018
Hmmm - how (by which algorithm / rule) has the EEEE in the first input line turned into Surwald in the expected Output? And how the third line's 44444 into 99456?


Try also
Code:
awk -F'|' '
        {while (NF<17)  {getline X
                         $0 = $0 " " X
                        }
        }
1
' file

# 4  
Old 11-21-2018
Replace \n char in Data

Thanks Rudi C,

Its my bad.

Expected output:

Code:
455000|TTTT|97233|UUUUUU REP||Juli||EEEE||P.O. Box 550 MMMMMMMMMM JJ 55555|||||||
333333|DDD|97233|UUUUUU REP||AMAR||AJAY||P.O. Box 69 MMMMMMMMMM JJ 6666 JJJ BBBB P.O. Box 4 MMMMMMMMMM JJ 44444 delmer Speidel P.O. Box 242 MMMMMMMMMM JJ 99456, See File For More.....|||||||
888888|Director|97382|UUUUUU REP||ANTHONY|K|JOSHI||1144 JNM ROAD LLLLLLLLLLLLL JJ 82513, Laurie Ideker, Leon Sanderson coralie Emmons P.O. Box 34 LLLLLLLLLLLLL JJ 82513 wanda Knowles P.O. Box 958 LLLLLLLLLLLLL JJ 82513, See File For More...|||||||
999999|President|97692|UUUUUU REP||See||File|||||||||

------ Post updated 11-21-18 at 11:24 AM ------

Thanks a lot Rudi C,

1)One more thing is in my file I may get Extra pipes(|) also( other than Field delimeter | ) then how to handle
2)Can u please explain me your command

Last edited by vgersh99; 11-20-2018 at 01:20 PM.. Reason: Code tags, please!
# 5  
Old 11-21-2018
1) If you tell us how to tell separator pipes from in-field-pipes, then someone could come up with some smart algorithm to handle that.
2) That little command keeps reading / appending new lines until the field count is 17; then: print (default action after "1" (= TRUE)).
# 6  
Old 11-22-2018
1)I'm not sure how to identify data pipes, but if we get | in 2 or 3 specific fields like Address and other.
So can we handle ?
# 7  
Old 11-22-2018
If your field delimiter is sometimes a field delimiter and sometimes data, you need to be able to very clearly identify each occurrence of that character as either data or delimiter. If you can't specify a clear rule that unambiguously determines whether a given character is a delimiter or data, there is no way to identify field boundaries.

And when you have field delimiters that are sometimes data AND record delimiters that are sometimes data, you have a real mess.

Your best choice would be to choose a different field delimiter that cannot ever appear as data.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Replace \n char true line Separator

Unix File is pipe delimited with 17 fields. We may get extra pipes in data also. We may get \n char (1 or more \n in one field or multi fileds) in data in any field. Need to replace \n true ( line separator) with 'space and bell char space' chars (' \a ') Not data \n. Input:... (1 Reply)
Discussion started by: rajeshkumare
1 Replies

2. Shell Programming and Scripting

Match a char with duplicates in a line and replace one of them

Hi, i have a huge file that need to check for a pattern that occur more than once in a line like below:- #lkk>cd-m>A0DV0>192.134.1.1 blablabladsdjsk jshdfskfslfs #lqk>cd-m>A1SV0>192.14.11.1 blalalbnalablab balablablajakjakjakja #pldqw>sf-w>PH67FR>168.55.1.1 balablabala... (5 Replies)
Discussion started by: redse171
5 Replies

3. Shell Programming and Scripting

Replace Char in XML Base on Condition

HI All Here is my Input file A. I want to add extra char in the line base on below condition. 1> if Below second line is <xn:vsDataType>vsDataEUtranCellFDD</xn:vsDataType> then <xn:VsDataContainer id= will be <xn:VsDataContainer_id_1= 2 > if Below second line is... (4 Replies)
Discussion started by: asavaliya
4 Replies

4. Shell Programming and Scripting

To find char field and replace null

hi, i having a file with | seperated in which i need to search char in 3rd column and replace with null. i need to replace only the coulmn where character occurs in 3rd field for eg: file1.txt xx|yy|xx|12 output file: xx|yy||12 (5 Replies)
Discussion started by: rohit_shinez
5 Replies

5. Shell Programming and Scripting

Replace char between chars - help needed

Hello, I have a csv file with "^" as text delimiters and "|" as field delimiters. It's converted from a xls file. One record looks like this: ^Tablete Internet^|Archos|501838|^Tableta Internet ARCHOS 80 G9 ...| ... (more lines) ... "501|838"^|330.00|USD|sl|12|0|Link|^router wireless 150... (10 Replies)
Discussion started by: go0ogl3
10 Replies

6. Shell Programming and Scripting

Find and replace all extended char.

Hi Guys, I wand find and replace all Extended ASCII Codes from all my log files. My Log files: /home/Kalr/PPool/Output i have logs file in sub dir. /home/Kalr/PPool/Output/X /home/Kalr/PPool/Output/Y /home/Kalr/PPool/Output/Z My Abc.log file input: Extended ASCII Codes :– ... (4 Replies)
Discussion started by: asavaliya
4 Replies

7. Programming

I don't know how to replace input char with appropriate integer

Hi guys, I asked for help on programming forums and no one didn't helped me so I ask for help here. I am playing with some tasks from my book and I can't figure where did I get wrong. From the first program I get a blank screen, program won't generate 10*10 matrix. And second problem is I... (6 Replies)
Discussion started by: solaris_user
6 Replies

8. Shell Programming and Scripting

Replace char on text file

Hi , I have problem on replace char on text file #!/bin/bash echo "Enter Third value : " read a sed '2 s/192.160.1.1/cut -d"." -f3/$a/g' tcpip.txt > a.txt I want replace line 2 on tcpip.txt with value a that I input but it's not run Please help me (2 Replies)
Discussion started by: phillipss
2 Replies

9. Shell Programming and Scripting

In vi editor I want to replace next line char by space

in vi editor I want to replace next line char by space help me eg: input: 123 123 123 output: 123 123 123 (5 Replies)
Discussion started by: RahulJoshi
5 Replies

10. Shell Programming and Scripting

How to replace any char with newline char.

Hi, How to replace any character in a file with a newline character using sed .. Ex: To replace ',' with newline Input: abcd,efgh,ijkl,mnop Output: abcd efgh ijkl mnop Thnx in advance. Regards, Sasidhar (5 Replies)
Discussion started by: mightysam
5 Replies
Login or Register to Ask a Question