Sponsored Content
Top Forums Shell Programming and Scripting How to remove special characters? Post 302830811 by ken6503 on Tuesday 9th of July 2013 02:14:15 PM
Old 07-09-2013
Quote:
Originally Posted by wisecracker
Be very careful, do not make an assumption that it is a single byte is size.

For a quick assessment use this command to check:-

Code:
hexdump -C /full/path/to/your/filename

This is an example; I copied your character and put it into an editor:-

Code:
This is the _byte_ ü _end_.

Note the character is between two spaces...

Now using the above command:-

Code:
Last login: Tue Jul  9 18:46:23 on ttys000
AMIGA:barrywalker~> hexdump -C /Users/barrywalker/byte_test.txt
00000000  54 68 69 73 20 69 73 20  74 68 65 20 5f 62 79 74  |This is the _byt|
00000010  65 5f 20 c3 bc 20 5f 65  6e 64 5f 2e 0a           |e_ .. _end_..|
0000001d
AMIGA:barrywalker~>

Note that at position 00000013 and 00000014 the bytes c3 and bc have appeared instead of the single character you are expecting...

So be very, very careful...

Hope this helps...
Thanks for your quick reply.
I run following command and got some result.
Code:
 # echo 'ADDÜL' |hexdump -C
00000000  41 44 44 dc 4c 0a                                 |ADD.L.|
00000006

Actually, I was run following command to split the file with one line to separate lines. when it hits the charactor Ü, it stopped.
what should I do to make the command to spearate file without stop
Code:
awk -v L="$2" '{for (i=1; i<=length($0); i+=L) print substr($0, i, L)}' "$1" > "$1"_split

Thanks in advance
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

remove special and unicode characters

Hi, How do I remove the lines where special characters or Unicode characters appear? The following query does work but I wonder if there is a better way. cat test.txt | egrep -v '\)|#|,|&|-|\(|\\|\/|\.' The following lines show that my query is incomplete. Warning: The word "*Khan" is... (1 Reply)
Discussion started by: shantanuo
1 Replies

2. Shell Programming and Scripting

Remove special characters from string

Hi there, I'd like to write a script that removes any set of character from any string. The first argument would be the string, the second argument would be the characters to remove. For example: $ myscript "My name's Santiago. What's yours?" "atu" My nme's Snigo. Wh's yors? I wrote the... (11 Replies)
Discussion started by: chebarbudo
11 Replies

3. Shell Programming and Scripting

How to remove special characters from each line?

Hello, Is there a simpler way to remove special characters (color codes) from each lines in a log file? I use sed like in the example below but I think there should be a more simple way to achieve the same result: $ cat -vet file1 ^, , , , Maybe to convert the file somehow? ... (5 Replies)
Discussion started by: majormark
5 Replies

4. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks... (5 Replies)
Discussion started by: sandeep_1105
5 Replies

5. UNIX for Dummies Questions & Answers

Files with special characters - how to remove

Hi, I have a directory that has a file which contained special characters in the filename. Can someone please advise how to remove the file, preferably with a rm -i ? Thanks in advance. Listing is as below: {oracle}> ls -1b bplog.bkup.001 bplog.bkup.002 bplog.bkup.003 bplog.bkup.004... (1 Reply)
Discussion started by: newbie_01
1 Replies

6. Shell Programming and Scripting

remove special characters

hello all I am writing a perl code and i wish to remove the special characters for text. I wish to remove all extended ascii characters. If the list of special characters is huge, how can i do this using substitute command s/specialcharacters/null/g I really want to code like... (3 Replies)
Discussion started by: vasuarjula
3 Replies

7. Shell Programming and Scripting

Remove string between two special characters

Hi All, I have a variable like AVAIL="\ BACK:bkpstg:testdb3.iad.expertcity.com:backtest|\ #AUTH:authstg:testdb3.iad.expertcity.com:authiapd|\ TEST:authstg:testdb3.iad.expertcity.com:authiapd|\ " What I want to do here is that If a find # before any entry, remove the entire string... (5 Replies)
Discussion started by: engineermayur
5 Replies

8. Shell Programming and Scripting

Remove the special characters from field

Hi, In source data few of columns are having special charates(like *) due to this i am not able to display the data into flat file.it's displaying the some of junk data into the flat file. source dataExample: Address1="XDERFTG * HYJUYTG" how to remove the special charates in a string (2 Replies)
Discussion started by: koti_rama
2 Replies

9. Shell Programming and Scripting

Sed - remove special characters

Hi, I have a file with this line, it's always in the first line: I want to remove these special characters: ´╗┐ file1 ´╗┐\\bar\c$\test2\;3.348.118 Bytes;160 ;3 \\bar\c$\test\;35 Bytes;2 ;1 I want the same file to be only \\bar\c$\test2\;3.348.118 Bytes;160 ;3 \\bar\c$\test\;35... (4 Replies)
Discussion started by: nakaedu
4 Replies

10. Shell Programming and Scripting

How to remove some special characters in a string?

Hi, I have string like this ="Lookup Procedure" But i want the output like this Lookup Procedure =," should be removed. Please suggest me the solution. Regards, Madhuri (2 Replies)
Discussion started by: srimadhuri
2 Replies
OD(1)							    BSD General Commands Manual 						     OD(1)

NAME
od -- octal, decimal, hex, ascii dump SYNOPSIS
od [-aBbcDdeFfHhIiLlOovXx] [-A base] [-j skip] [-N length] [-t type_string] [[+]offset[.][Bb]] file ... DESCRIPTION
The options are as follows: -A base Specify the input address base. base may be one of 'd', 'o', 'x' or 'n', which specify decimal, octal, hexadecimal addresses or no address, respectively. -a One-byte character display. Display the input offset in octal, followed by sixteen space-separated, three column, space-filled, characters of input data per line. Control characters are printed as their names instead of as C-style escapes. -B Same as -o. -b One-byte octal display. Display the input offset in octal, followed by sixteen space-separated, three column, zero-filled, bytes of input data, in octal, per line. This is the default output style if no other is selected. -c One-byte character display. Display the input offset in octal, followed by sixteen space-separated, three column, space-filled, characters of input data per line. Control characters are printed at C-style escapes, or as three octal digits, if no C escape exists for the character. -d Two-byte decimal display. Display the input offset in octal, followed by eight space-separated, five column, zero-filled, two- byte units of input data, in unsigned decimal, per line. -e Eight-byte floating point display. Display the input offset in octal, followed by two space-separated, twenty-one column, space filled, eight byte units of input data, in floating point, per line. -F Same as -e. -f Four-byte floating point display. Display the input offset in octal, followed by four space-separated, 14 column, space filled, four byte units of input data, in floating point, per line. -H Four-byte hex display. Display the input offset in octal, followed by four space-separated, eight column, zero filled, four byte units of input data, in hex, per line. -h Two-byte hex display. Display the input offset in octal, followed by eight space-separated, four column, zero filled, two byte units of input data, in hex, per line. -I Four-byte decimal display. Display the input offset in octal, followed by four space-separated, eleven column, space filled, four byte units of input data, in decimal, per line. -i Two-byte decimal display. Display the input offset in octal, followed by eight space-separated, six column, space filled, two- byte units of input data, in decimal, per line. -j offset Skip offset bytes from the beginning of the input. By default, offset is interpreted as a decimal number. With a leading 0x or 0X, offset is interpreted as a hexadecimal number, otherwise, with a leading 0, offset is interpreted as an octal number. Appending the character b, k, or m to offset causes it to be interpreted as a multiple of 512, 1024, or 1048576, respectively. -L Same as -I. -l Same as -I. -N length Interpret only length bytes of input. -O Four-byte octal display. Display the input offset in octal, followed by four space-separated, eleven column, zero-filled, four- byte units of input data, in octal, per line. -o Two-byte octal display. Display the input offset in octal, followed by eight space-separated, six column, zero-filled, two-byte units of input data, in octal, per line. -t type_string Specify one or more output types. The type_string option-argument must be a string specifying the types to be used when writing the input data. The string must consist of the type specification characters: a selects US-ASCII output, with control characters replaced with their names instead of as C escape sequences. See also the _u conversion provided by hexdump(1). c selects a standard character based conversion. See also the _c conversion provided by hexdump(1). f selects the floating point output format. This type character can be optionally followed by the characters 4 or F to specify four byte floating point output, or 8 or L to specify eight byte floating point output. The default output format is eight byte floats. See also the e conversion provided by hexdump(1). d, o, u, or x select decimal, octal, unsigned decimal, or hex output respectively. These types can optionally be followed by C to specify char-sized output, S to specify short-sized output, I to specify int-sized output, L to specify long-sized output, 1 to specify one-byte output, 2 to specify two-byte output, 4 to specify four-byte output, or 8 to specify eight-byte output. The default output format is in four-byte quantities. See also the d, o, u, and x conversions provided by hexdump(1). -v The -v option causes od to display all input data. Without the -v option, any number of groups of output lines, which would be identical to the immediately preceding group of output lines (except for the input offsets), are replaced with a line comprised of a single asterisk. -X Same as -H. -x Same as -h. For each input file, od sequentially copies the input to standard output, transforming the data according to the options given. If no options are specified, the default display is equivalent to specifying the -o option. od exits 0 on success and >0 if an error occurred. SEE ALSO
hexdump(1), strings(1) HISTORY
A od command appears in Version 1 AT&T UNIX. This man page was written in February 2001 by Andrew Brown, shortly after he augmented the deprecated od syntax to include things he felt had been missing for a long time. BSD
February 9, 2010 BSD
All times are GMT -4. The time now is 11:13 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy