Removing Embedded Newline from Delimited File

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing duplicates from delimited file based on 2 columns

Hi guys,Got a bit of a bind I'm in. I'm looking to remove duplicates from a pipe delimited file, but do so based on 2 columns. Sounds easy enough, but here's the kicker... Column #1 is a simple ID, which is used to identify the duplicate. Once dups are identified, I need to only keep the one...

2. Shell Programming and Scripting

Script for removing newline character from file

Hi below is my file. cat input.dat 101,abhilash,1000 102,prave en,2000 103,partha,4 000 10 4,naresh,5000 (its just a example file) and my output should be: 101,abhilash,1000 102,praveen,2000 103,partha,4000 104,naresh,5000 below is my code cat input.dat |tr -d '\n' >...

3. Shell Programming and Scripting

Removing dupes within 2 delimited areas in a large dictionary file

Hello, I have a very large dictionary file which is in text format and which contains a large number of sub-sections. Each sub-section starts with the following header : #DATA #VALID 1 and ends with a footer as shown below #END The data between the Header and the Footer consists of...

4. UNIX for Dummies Questions & Answers

Removing empty lines at the end of a Tab-delimited file

I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs. I've tried may things, here are a couple: sed /^\t.\t/d File1 > File2 sed /^\t{44}/d File1 > File2 What am I missing?

5. Shell Programming and Scripting

Help with removing embedded linefeeds

6. Shell Programming and Scripting

Removing ^M and the newline that follows it.

Hi Gurus, Apologies as I feel like this must be answered already on here somewhere but I just can't find it. I find many people looking to remove all \n and \r (CR and LF) or one or the other but the only times I've found someone trying to remove them only when both are together they've found...

7. Shell Programming and Scripting

Read Embedded Newline characters with read (builtin) in KSH93

Hi Guys, Happy New Year to you all! I have a requirement to read an embedded new-line using KSH's read builtin. Here is what I am trying to do: run_sql "select guestid, address, email from guest" | while read id addr email do ## Biz logic goes here done I can take care of any...

8. Shell Programming and Scripting

Removing blanks in a text tab delimited file

Hi Experts I am very new to perl and need to make a script using perl. I would like to remove blanks in a text tab delimited file in in a specfic column range ( colum 21 to column 43) sample input and output shown below : Input: 117 102 650 652 654 656 117 93 95...

9. UNIX for Advanced & Expert Users

Issue with Removing Carriage Return (^M) in delimited file

Hi - I tried to remove ^M in a delimited file using "tr -d "\r" and "sed 's/^M//g'", but it does not work quite well. While the ^M is removed, the format of the record is still cut in half, like a,b, c c,d,e The delimited file is generated using sh script by outputing a SQL query result to...

10. Shell Programming and Scripting

Help needed in removing intermediate segments from a pipe delimited segment file

Hi, I just stuckup in doing some regular expressions on a file. I have data which has multiple FHS and BTS segments like: FHS|12121|LOCAL|2323 MSH|10101|POTAMAS|2323 PID|121221|THOMAS|DAVID|23432 OBX|2342|H1211|3232 BTS|0000|MERSTO|LIABLE FHS|12121|LOCAL|2323 MSH|10101|POTAMAS|2323...

LEARN ABOUT DEBIAN

cr

cr(1)							      General Commands Manual							     cr(1)

NAME

       cr - converts text files between nix EOL and dos EOL

SYNOPSIS

       cr  - | +  <input file> <output file>

DESCRIPTION

       Text  files,  such  as tle files, that come from a dos source usualy have the ^M symbol at the end of every line. Cr converts files between
       the dos newline format and the normal *nix newline format by stripping the ^M to convert dos to *nix, using the '-' option, or adding ^M to
       a *nix file to create the proper dos file when the '+' option is used.

       Although this extra character is not often a problem, programs like seesat5, which are data driven will encounter parsing problems when the
       extra character is present. It is these problems that cr is intended to repair.

   Options
       - | +  One or the other of these options is required. The '-' option is used to remove ^M from all newlines found in the dos file. The  '+'
	      option is used to add ^M to every newline found in a *nix file.

       input file
	      Fully delineated path to the input file. As this program is used in the dos environment as well, standard input is not used.

       output file
	      Fully delineated path to the output file. As this program is used in the dos environment as well, standart output is not used.

SEE ALSO

       seesat5(1), seesat5(7), SEESAT5.INI(5), tle(5)

BUGS

       Cr  is  not an inteligent program. It methodicaly replaces/removes the offending character when it finds it in the correct context. Newline
       sequences found in contexts other than 'newline' will be replaced/removed just like those found in the proper  context.	Passing  a  binary
       file through cr is not advised, for this reason. Send all inqueries to Dale Scheetz <dwarf@polaris.net>.

Debian Linux							    2 April 96								     cr(1)

Shell Programming and Scripting