Removing Embedded Newline from Delimited File

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one...

2. Shell Programming and Scripting

Script for removing newline character from file

Hi below is my file. cat input.dat 101,abhilash,1000 102,prave en,2000 103,partha,4 000 10 4,naresh,5000 (its just a example file) and my output should be: 101,abhilash,1000 102,praveen,2000 103,partha,4000 104,naresh,5000 below is my code cat input.dat |tr -d '\n' >...

3. Shell Programming and Scripting

Removing dupes within 2 delimited areas in a large dictionary file

Hello, I have a very large dictionary file which is in text format and which contains a large number of sub-sections. Each sub-section starts with the following header : #DATA #VALID 1 and ends with a footer as shown below #END The data between the Header and the Footer consists of...

4. UNIX for Dummies Questions & Answers

Removing empty lines at the end of a Tab-delimited file

I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs. I've tried may things, here are a couple: sed /^\t.\t/d File1 > File2 sed /^\t{44}/d File1 > File2 What am I missing?

5. Shell Programming and Scripting

Help with removing embedded linefeeds

6. Shell Programming and Scripting

Removing ^M and the newline that follows it.

Hi Gurus, Apologies as I feel like this must be answered already on here somewhere but I just can't find it. I find many people looking to remove all \n and \r (CR and LF) or one or the other but the only times I've found someone trying to remove them only when both are together they've found...

7. Shell Programming and Scripting

Read Embedded Newline characters with read (builtin) in KSH93

Hi Guys, Happy New Year to you all! I have a requirement to read an embedded new-line using KSH's read builtin. Here is what I am trying to do: run_sql "select guestid, address, email from guest" | while read id addr email do ## Biz logic goes here done I can take care of any...

8. Shell Programming and Scripting

Removing blanks in a text tab delimited file

Hi Experts I am very new to perl and need to make a script using perl. I would like to remove blanks in a text tab delimited file in in a specfic column range ( colum 21 to column 43) sample input and output shown below : Input: 117 102 650 652 654 656 117 93 95...

9. UNIX for Advanced & Expert Users

Issue with Removing Carriage Return (^M) in delimited file

Hi - I tried to remove ^M in a delimited file using "tr -d "\r" and "sed 's/^M//g'", but it does not work quite well. While the ^M is removed, the format of the record is still cut in half, like a,b, c c,d,e The delimited file is generated using sh script by outputing a SQL query result to...

10. Shell Programming and Scripting

Help needed in removing intermediate segments from a pipe delimited segment file

Hi, I just stuckup in doing some regular expressions on a file. I have data which has multiple FHS and BTS segments like: FHS|12121|LOCAL|2323 MSH|10101|POTAMAS|2323 PID|121221|THOMAS|DAVID|23432 OBX|2342|H1211|3232 BTS|0000|MERSTO|LIABLE FHS|12121|LOCAL|2323 MSH|10101|POTAMAS|2323...

LEARN ABOUT BSD

syserrlst

SYSERRLST(5)							File Formats Manual						      SYSERRLST(5)

NAME

       syserrlst - error message file format

DESCRIPTION

       mkerrlst(1), creates error message files in the format described below.

       An  ``error  message file'' consists of a header, an array of structures specifying the offset and length of each message, and the array of
       message strings separated by newlines.

       The message strings are separated by newlines but the newline characters are not included in the size of the message.  These newline  char-
       acters serve only to make the file editable or printable (after stripping off the header).

       The file format is:

       /*
	* Definitions used by the 'mkerrlst' program which creates error message
	* files.
	*
	* The format of the file created is:
	*
	*   struct    ERRLSTHDR ehdr;
	*   struct    ERRLST  emsg[num_of_messages];
	*   struct    {
	*	 char msg[] = "error message string";
	*	 char lf = '0;
	*	 } [num_of_messages];
	*
	* Note:  the newlines are NOT included in the message lengths, the newlines
	*	 are present to make it easy to 'cat' or 'vi' the file.
       */

	    struct    ERRLSTHDR
		 {
		 short	   magic;
		 short	   maxmsgnum;
		 short	   maxmsglen;
		 short	   pad[5];	  /* Reserved */
		 };

	    struct    ERRLST
		 {
		 off_t	   offmsg;
		 short	   lenmsg;
		 };

       #define	 ERRMAGIC  012345

SEE ALSO

       mkerrlst(1), syserrlst(3)

BUGS

       Format of the file isn't necessarily portable between machines.

HISTORY

       This file format is new with 2.11BSD.

3rd Berkeley Distribution					   March 7, 1996						      SYSERRLST(5)

Shell Programming and Scripting