Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Remove newline character from column spread over multiple lines in a file Post 303038972 by Prathmesh on Wednesday 18th of September 2019 01:16:26 PM
Old 09-18-2019
Remove newline character from column spread over multiple lines in a file

Hi,

I came across one issue recently where output from one of the columns of the table from where i am creating input file has newline characters hence, record in the file is spread over multiple lines. Fields in the file are separated by pipe (|) delimiter. As header will never have newline character, I am trying to compare if other rows have same number of fields as that of header and if number of fields in particular row is less than number of fields in header line then I am removing newline character at the end of the line. I was able to do this for row spread over two lines but, I am not getting correct output for lines spread over multiple lines.

Below is test input file and expected output file -

Input file -
Code:
$ cat input
id|country|desscription|Language
1|UNITED STATES|WASHINGTON, D.C.|English
2|UNITED KINGDOM|Capital of UK is LONDON|English
3|NEPAL|Capital of NEPAL is
KATHMANDU|Nepali
4|QATAR|DOHA
is capital of
QATAR|Urdu
5|INDIA|capital
of
INDIA
is DELHI|Hindi
$

Expected output file -
Code:
id|country|desscription|Language
1|UNITED STATES|WASHINGTON, D.C.|English
2|UNITED KINGDOM|Capital of UK is LONDON|English
3|NEPAL|Capital of NEPAL is KATHMANDU|Nepali
4|QATAR|DOHA is capital of QATAR|Urdu
5|INDIA|capital of INDIA is DELHI|Hindi

Below code worked for row spread over two lines -
Code:
$ awk -F"|" '{if(NR==1){COL=NF}}{if(NF < COL){ sub(/\n/, ""); T=$0; getline; print T $0; next}}1' input
id|country|desscription|Language
1|UNITED STATES|WASHINGTON, D.C.|English
2|UNITED KINGDOM|Capital of UK is LONDON|English
3|NEPAL|Capital of NEPAL is KATHMANDU|Nepali
4|QATAR|DOHA is capital of
QATAR|Urdu5|INDIA|capital
of INDIA
is DELHI|Hindiis DELHI|Hindi
$

I also tried below code but it is not giving expected output -
Code:
 $ awk -F"|" '{if(NR==1){COL=NF}}{
> L_NF=NF
> C_NR=NR
> NL=$0
> CNT=0
> while(L_NF != COL)
> {
> C_NF=NF
> sub(/\n/, "");
> getline;
> NL=NL" "$0;
> CNT=+1
> L_NF=C_NF+NF
> }
> print NL
> }
> {
> for(i=0;i<=CNT;i++)
> {
> next
> }
> {
> print $0
> }}' input
id|country|desscription|Language
1|UNITED STATES|WASHINGTON, D.C.|English
2|UNITED KINGDOM|Capital of UK is LONDON|English
3|NEPAL|Capital of NEPAL is  KATHMANDU|Nepali 4|QATAR|DOHA  is capital of
QATAR|Urdu 5|INDIA|capital  of
INDIA  is DELHI|Hindi is DELHI|Hindi
$

Can someone please help me in this?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Can I spread commands over multiple lines?

Below an example of what I mean. The first attempt does what I want; the second doesn't, because bash assumes a line break means the end of an individual "command unix". Is there some way that I can convince bash to parse out, eg, to the closing parenthesis? I'm thinking this would allow for... (1 Reply)
Discussion started by: tphyahoo
1 Replies

2. Shell Programming and Scripting

removing pattern which is spread in multiple lines

I have several huge files wich contains oracle table creation scripts as follows: I would need to remove the pattern colored in red above. Any sed/awk/pearl code will be of much help. Thanks (2 Replies)
Discussion started by: sabyasm
2 Replies

3. Shell Programming and Scripting

How to remove a newline character at the end of filename

Hi All, I have named a file with current date,time and year as follows: month=`date | awk '{print $2}'` date=`date | awk '{print $3}'` year=`date | awk '{print $6}'` time=`date +%Hh_%Mm_%Ss'` filename="test_"$month"_"$date"_"$year"_"$time".txt" > $filename The file is created with a... (2 Replies)
Discussion started by: amio
2 Replies

4. Shell Programming and Scripting

To remove the newline character while appending into a file

Hi All, We append the output of a file's size in a file. But a newline character is appended after the variable. Pls help how to clear this. filesize=`ls -l test.txt | awk `{print $5}'` echo File size of test.txt is $filesize bytes >> logfile.txt The output we got is, File size of... (4 Replies)
Discussion started by: amio
4 Replies

5. Shell Programming and Scripting

Remove newline character conditionally

Hi All, I have 5000 records like this Request_id|Type|Status|Priority|Ticket Submitted Date and Time|Actual Resolved Date and Time|Current Ticket Owner Group|Case final Ticket Owner Group|Customer Severity|Reported Symptom/Request|Component|Hot Topic|Reason for Missed SLA|Current Ticket... (2 Replies)
Discussion started by: j_53933
2 Replies

6. Shell Programming and Scripting

[AWK] handeling data spread on multiple lines

Hello all, first off great forum. Now for my little problem. Using RHEL 5.4 and awk. Been doing code since a few month. So just starting. My problem is handeling data on multiple lines. { if ($1 != LASTKEY && h ~ /.*\/s_fr_/) { checkgecos( h, h ) h="" ... (2 Replies)
Discussion started by: maverick72
2 Replies

7. Shell Programming and Scripting

Remove \n <newline> character inside the records.

Hi, In my file, I have '\n' characters inside a single record. Because of this, a single records appears in many lines and looks like multiple records. In the below file. File 1 ==== 1,nmae,lctn,da\n t 2,ghjik,o\n ut,de\n fk Expected output after the \n removed File 2 =====... (5 Replies)
Discussion started by: machomaddy
5 Replies

8. Shell Programming and Scripting

Remove newline character between two delimiters

hi i am having delimited .dat file having content like below. test.dat(5 line of records) ====== PT2~Stag~Pt2 Stag Test. Updated~PT2 S T~Area~~UNCEF R20~~2012-05-24 ~2014-05-24~~ PT2~Stag y~Pt2 Stag Test. Updated~PT2 S T~Area~METR~~~2012-05-24~2014-05-24~~test PT2~Pt2 Stag Test~~PT2 S... (4 Replies)
Discussion started by: sushine11
4 Replies

9. Shell Programming and Scripting

Remove last newline character..

Hi all.. I have a text file which looks like below: abcd efgh ijkl (blank space) I need to remove only the last (blank space) from the file. When I try wc -l the file name,the number of lines coming is 3 only, however blank space is there in the file. I have tried options like... (14 Replies)
Discussion started by: Sathya83aa
14 Replies

10. Shell Programming and Scripting

How to remove newline character if it is the only character in the entire file.?

I have a file which comes every day and the file data look's as below. Vi abc.txt a|b|c|d\n a|g|h|j\n Some times we receive the file with only a new line character in the file like vi abc.txt \n (8 Replies)
Discussion started by: rak Kundra
8 Replies
mkmanifest(1)						      General Commands Manual						     mkmanifest(1)

Name
       mkmanifest - makes list of file names and their DOS 8+3 equivalent

Note of warning
       This  manpage  has been automatically generated from mtools's texinfo documentation, and may not be entirely accurate or complete.  See the
       end of this man page for details.

Description
       The mkmanifest command is used to create a shell script (packing list) to restore Unix filenames. Its syntax is:

       mkmanifest [ files ]

       Mkmanifest creates a shell script that aids in the restoration of Unix filenames that got clobbered by the  MS-DOS  filename  restrictions.
       MS-DOS filenames are restricted to 8 character names, 3 character extensions, upper case only, no device names, and no illegal characters.

       The mkmanifest program is compatible with the methods used in pcomm, arc, and mtools to change perfectly good Unix filenames to fit the MS-
       DOS restrictions. This command is only useful if the target system which will read the diskette cannot handle vfat long names.

Example
       You want to copy the following Unix files to a MS-DOS diskette (using the mcopy command).

	    very_long_name
	    2.many.dots
	    illegal:
	    good.c
	    prn.dev
	    Capital

       Mcopy converts the names to:

	    very_lon
	    2xmany.dot
	    illegalx
	    good.c
	    xprn.dev
	    capital

       The command:

	  mkmanifest very_long_name 2.many.dots illegal: good.c prn.dev Capital >manifest

       would produce the following:

	    mv very_lon very_long_name
	    mv 2xmany.dot 2.many.dots
	    mv illegalx illegal:
	    mv xprn.dev prn.dev
	    mv capital Capital

       Notice that "good.c" did not require any conversion, so it did not appear in the output.

       Suppose I've copied these files from the diskette to another Unix system, and I now want the files back to their original  names.   If  the
       file "manifest" (the output captured above) was sent along with those files, it could be used to convert the filenames.

Bugs
       The short names generated by mkmanifest follow the old convention (from mtools-2.0.7) and not the one from Windows 95 and mtools-3.0.

See Also
       Mtools' texinfo doc

Viewing the texi doc
       This  manpage  has  been automatically generated from mtools's texinfo documentation. However, this process is only approximative, and some
       items, such as crossreferences, footnotes and indices are lost in this translation process.  Indeed, these items have no appropriate repre-
       sentation  in  the manpage format.  Moreover, not all information has been translated into the manpage version.	Thus I strongly advise you
       to use the original texinfo doc.  See the end of this manpage for instructions how to view the texinfo doc.

       *      To generate a printable copy from the texinfo doc, run the following commands:

		     ./configure; make dvi; dvips mtools.dvi

       *      To generate a html copy,	run:

		     ./configure; make html

	      A premade html can be found at: `http://mtools.linux.lu' and also at: `http://www.tux.org/pub/knaff/mtools'

       *      To generate an info copy (browsable using emacs' info mode), run:

		     ./configure; make info

       The texinfo doc looks most pretty when printed or as html.  Indeed, in the info version certain examples are difficult to read due  to  the
       quoting conventions used in info.

mtools-3.9.8							      02Jun01							     mkmanifest(1)
All times are GMT -4. The time now is 02:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy