Sponsored Content
Top Forums Shell Programming and Scripting Replacing hex characters '\x0D' with '\x0D\x0A' Post 302377351 by durden_tyler on Thursday 3rd of December 2009 09:08:10 PM
Old 12-03-2009
Quote:
Originally Posted by paragkalra
Thanks 'durden_tyler' your code worked...but just to mention that it didn't work on Windows...
...
I think its due to the fact that interpreters like 'sed', 'Perl' ignores 'CR' (chariage return) characters on windoze..........
Don't know about "sed" on Windows, but that would be highly unusual for Perl on Windows:

Code:
C:\>
C:\>REM create a file that has \x0D characters at multiple places
C:\>perl -le "print \"\x0D The quick \x0D brown fox jumps \x0D over the lazy dog.\x0D\"" >f0
C:\>
C:\>REM check the file contents
C:\>vis <f0
\015 The quick \015 brown fox jumps \015 over the lazy dog.\015
C:\>
C:\>REM do an inline replace of all \x0D characters to \x0D\x0A
C:\>perl -i.bak -pe "s/(\x0D)/$1\x0A/g" f0
C:\>
C:\>REM now check the file contents again
C:\>vis <f0
\015
 The quick \015
 brown fox jumps \015
 over the lazy dog.\015

C:\>
C:\>REM print file using the DOS "type" command
C:\>type f0
 The quick
 brown fox jumps
 over the lazy dog.

C:\>
C:\>

The perl interpreter used in the example above is from the ActiveState Perl installed using MSI.

"vis" is just a small C program that makes non-printable characters visible in octal code.

tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replacing all but last Hex characters in a text line

I must remove hex characters 0A and 0D from several fields within an MS Access Table. Since I don't think it can be done in Access, I am trying here. I am exporting a Table from Access (must be fixed length fields, I think, for my idea to work here) into a text format. I then want to run a... (2 Replies)
Discussion started by: BAH
2 Replies

2. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud (2 Replies)
Discussion started by: budrito
2 Replies

3. UNIX for Dummies Questions & Answers

replacing characters

Hi, I have a script for replacing bad characters in filenames for f in *; do mv $f `echo $f | tr '+' '_'` done; this replaces + for _ But I need to replace all bad characters ? / % + to _ Pls how can i do this in one script ? (3 Replies)
Discussion started by: palmer18
3 Replies

4. Programming

Data formating using C programm with Hex deciamal 'x0d'

:b:Guys, Can some body throw some light on this please..... sprintf(req_line1, "%c%s%c", '\x0b',"TESTING1",'\x0d'); sprintf(req_line2, "%s%c", "TESTING2", '\x0d'); sprintf(req_line3, "%s%c", "Testing3", '\x0d'); sprintf(req_line4, "%s%c%c%c", "Testing4", '\x0d', '\x1c', '\x0d'); ... (6 Replies)
Discussion started by: sudharma
6 Replies

5. Shell Programming and Scripting

Convert hex values to displayable characters

Hi, I am a bit stuck with displaying characters. I am having values like below in the proper displayable characters. which I would want to print the actual value on the right hand side. I dont want to create an array because I would have to create 255 different values. isnt there another way of... (17 Replies)
Discussion started by: ahmedwaseem2000
17 Replies

6. Shell Programming and Scripting

Grepping for hex characters - explanation?

Hello, Yesterday I was looking for a way to grep for a tab in the shell, and found this solution in several places: grep $'a' # Grep for the letter 'a' between two tabs I'm fine with most of this, but I don't understand what the $ (dollar sign) before the first quote does. It doesn't work... (7 Replies)
Discussion started by: mregine
7 Replies

7. UNIX for Dummies Questions & Answers

Replacing hex characters

I have the following file consisting of dates and sample measurements: 05��Oct��2010 1.31�� 06��Oct��2010 1.32�� 07��Oct��2010 1.31�� The hex characters are \xc2\xa0 in sequence. I have tried to remove the characters as follows: sed -i '' -e 's/\xc2\xa0//g' file.dat and as follows... (6 Replies)
Discussion started by: figaro
6 Replies

8. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

9. HP-UX

Replacing Hex Characters In A File Using awk?

Hi guys, First off, i'm a complete noob to UNIX and LINUX so apologies if I don't understand the basics! I have a file which contains a hex value of '0D' at the end of each line when I look at it in a hex viewer. I need to change it so it contains a hex value of '0D0A0A' I thought... (10 Replies)
Discussion started by: AndyBSG
10 Replies

10. Shell Programming and Scripting

Losing carriage return (X0D) after running awk command

Hi Forum. I'm running the following awk command to extract the suffix value (pos 38) from the "AM00" record and append to the end of the "AM01" record. awk 'substr($0,13,4)=="AM00" {SUFFIX = substr($0,38,2)} substr($0,13,4)=="AM01" {$0 = $0 SUFFIX} 1' before.txt > after.txt Before.txt:... (2 Replies)
Discussion started by: pchang
2 Replies
CSV2_TXT(5)							 MaraDNS reference						       CSV2_TXT(5)

NAME
csv2_txt - Description of txt and raw resource records in the csv2 zone file DESCRIPTION
Due to the complexity of TXT and RAW records, this man page is dedicated to describing the csv2 format of this RR. TXT and RAW rrs in MaraDNS' csv2 zone files can store any arbitrary binary data. Additionally, it is possible to arbitrarily divide up TXT records in to chunks (chunks, which RFC1035 call "character-string"s, are described below). ASCII AND UTF-8 DATA If a given TXT field or RAW record contains only ASCII data, creating a record is easy: Place the full data between single quotes, like this: a.example.com. TXT 'This is some text' ~ It is also possible, to place almost any printable ASCII characters between quotes. The '~' (tilde) character is not allowed unless csv2_tilde_handling has a value of 0; the '|' (pipe), '#' (hash) and non-printable ASCII control characters are not allowed in TXT data if the ~ is used to separate records. If there are any bytes with a value of 0x80 or more, the data must be UTF-8 encoded Unicode. The printable ASCII characters not allowed in quotes are the ' character, the '|' character, the '~' (tilde) character, and the '#' character. See BACKSLASH ESCAPE SEQUENCES below for information on adding these characters to TXT or RAW fields. UNQUOTED DATA Note that the record does not have to be quoted. As long as the record only contains ASCII alphanumeric data, and/or the characters '-', '_', '+', '%', '!', '^', and '=', the data can be unquoted as follows: c.example.com. TXT This_is_100%_unquoted_text_+symbols! It is also possible to mix quoted and unquoted text, such as this: d.example.com. TXT This' is a mix 'of_unquoted' and quoted 'text! Which will have its data look like this: This is a mix of_unquoted and quoted text! When mixing quoted and unquoted data, it is important to have all whitespace inside quotes. BACKSLASH ESCAPE SEQUENCES In order to accommodate storing non-UTF-8 high bit characters, the single quote character, non-printable ASCII control codes, the '|', '~', and '#' characters, and to permit multi-line TXT/RAW records (with comments allowed mid-record), the TXT/RAW RR allows backslashes. These backslashes only have significance outside of quoted text; if they are placed inside single quotes, they are not interpreted and result in a literal backslash being added to the resource record data. The following characters can be backslashed: ' When backslashed, the adds a literal quote to the resource record. whitespace When any whitespace is backslashed (space, newline, cr, and tab), this indicates that the record has not ended, and that more data for this resource will follow. This also allows comments to be placed in TXT and RAW resource records. What happens is that the backslash indicates that any whitespace characters (space, tab, carriage return, and line feed) are to be ignored until the next non-whitespace character that is not a # (hash). If a # is seen, this indicates that we ignore any and all characters until the next carriage return or line feed, and continue to ignore everything until the next non-whitespace character. See the section on multi-line and commented records for examples. 0123 When a number between 0 and 3 is backslashed, this indicates the beginning of a three-digit octal number. x When an x is backslashed, this indicates the beginning of a two-digit hexadecimal number. Note that, with the exception of the single quote, the backslash character is not used to remove the meta-significance of a given character. In particular, unlike other environments, it is not possible to backslash spaces. Spaces can be represented either as ' ' in quotes, x20, or as 40. Here are some examples of backslashed data. In this example, we see backslash sequences being used to store non-UTF-8 hi-bit data: e.example.com. TXT x80x81x82x83 ~ This same data can also be created as follows: f.example.com. TXT 200201202203 ~ Octal and hex information can be mixed: g.example.com. TXT 200x81202x83 ~ Literal single quotes can be placed in resource records: h.example.com. TXT 'perl -e '''print "A Perl of a TXT record! "'' ~ The above example produces this record: perl -e 'print "A Perl of a TXT record! "' ~ To render the '~' character, use the escape sequence x7e (outside of quotes). For example: h1.example.com. TXT 'http://ocf.berkeley.edu/'x7e'set' ~ Produces this record: http://ocf.berkeley.edu/~set To render the '|' character, use the escape sequence x7c: h2.example.com. TXT 'ls 'x7c' more' ~ Produces this record: ls | more To render the '#' character, use the escape sequence x23: h3.example.com. TXT 'Press 'x23' for customer service' ~ Produces this record: Press # for customer service MULTI-LINE AND COMMENTED RECORDS By utilizing backslashes followed by comments, it is possible to have multi-line and commented TXT and RAW records. The following resource record will span more than one line on an 80-column display: i.example.com. TXT 'Not only did the quick brown fox jump over the lazy dog, but the lazy dog jumped over the cat.' ~ Without affecting this resource record, the same data can be split over multiple lines: j.example.com. TXT 'Not only did the quick brown fox jump ' 'over the lazy dog, but the lazy dog' ' jumped over the cat.' ~ Some points: * The backslash must be outsize of the quotes (or a literal backslash will be added to the record) * The backslash must be present before any unquoted white space. Usually, the backslash is placed immediately after the quote character. * Unlike other environments, it does not matter whether or not there is invisible whitespace after the backslash. It is also possible to add comments after such a backslash as follows: k.example.com. TXT 'Not only did the quick brown fox jump ' # The fox 'over the lazy dog, but the lazy dog' # The dog ' jumped over the cat.' ~ # The cat Note that, since the third comment is not preceded by a backslash, this indicates the end of the resource record. There can also be multiple lines dedicated to comments (and, optionally, even blank lines) in the middle of TXT and RAW record data: k2.example.com. TXT 'This is some data ' # Here we have some comments followed by a blank line # Now we have some more comments, # followed by the rest of the data 'and this is the rest of the data' ~ MULTIPLE TXT CHUNKS TXT RRs may be divided up in to multiple "chunks" (RFC1035 calls these "character-string"s). A single chunk can be anywhere from zero to 255 bytes long. The default is to have one chunk, as follows: o.example.com. TXT 'TXT record with only one chunk' ~ It is also possible to have a record with multiple chunks. Chunks are delimited by an unquoted ';' character: p.example.com. TXT 'This is chunk one';'This is chunk two' ~ Or: q.example.com. TXT 'This is chunk one'; # Our first chunk This_is_chunk_two; # Our second chunk 'This is chunk three' ~ # Our final chunk Quoted ; characters simply add a ; to the record data. If a single TXT chunk is longer than 255 bytes long, the csv2 parser will report an error in the zone file: Single TXT chunk too long In order to resolve this, place unquoted ; characters in the record data so that each chunk is under 255 octets (bytes or characters) in length. It is possible to have zero length chunks: r.example.com. TXT 'chunk one';;'chunk three' ~ # Chunk two zero-length In particular, is is possible to have zero length chunks at the beginning and end of a TXT record: s.example.com. TXT ;'chunk two'; ~ # Chunks one and three zero-length Do not place semicolons at the beginning nor end of TXT records unless you wish to have these zero-length chunks. Chunk support only exists for TXT records. An unquoted ; character will cause a syntax error in a RAW record. RAW RECORDS With the exception of no support for chunk delimiters, and the addition of a numeric record type before the record data, the format for RAW records is identical to text records. For example, if we wish to have a "Kitchen Sink" RR record, which has the 8-bit binary numbers "16", "1", and "2", followed by the ASCII string "Kitchen sink+ data", we can specify this in any of the following manners: t1.example.com. RAW 40 x10x01x02'Kitchen sink'x2b' data' ~ t.example.com. RAW 40 200102Kitchen' sink+ data' ~ u.example.com. RAW 40 x10x01x02Kitchenx20sink+x20data ~ v.example.com. RAW 40 x1001x02 'Kitchen sink+ data' ~ w.example.com. RAW 40 x10 # Meaning: 16 x01 # Coding: 1 x02 # Sub-coding: 2 'Kitchen sink+ data' ~ # Data: 'Kitchen sink+ data' LEGAL DISCLAIMER
THIS SOFTWARE IS PROVIDED BY THE AUTHORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. AUTHOR
Sam Trenholme http://www.samiam.org/ MARADNS
January 2007 CSV2_TXT(5)
All times are GMT -4. The time now is 09:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy