Sponsored Content
Top Forums UNIX for Dummies Questions & Answers remove special and unicode characters Post 302264962 by otheus on Friday 5th of December 2008 08:16:48 AM
Old 12-05-2008
To eliminate all but printable characters:
Code:
grep -v '[^[:print]]' test.txt

There's also [:punct:] which are all punctuation characters. Or you can search for anything that doesn't have a normal letter or number:
Code:
grep -v '[^A-Za-z0-9]'

 

10 More Discussions You Might Find Interesting

1. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies

2. UNIX for Dummies Questions & Answers

Remove directory that has special Characters

Hi All, I have a script written that creates a new directory within the shell program and if a parameter isn't passed in, it creates a strange directory name by mistake. So I have a directory like "-_12" and I am unable to remove it. I tried removing it using double quote and many others. I have... (12 Replies)
Discussion started by: datherriault
12 Replies

3. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks... (5 Replies)
Discussion started by: sandeep_1105
5 Replies

4. UNIX for Dummies Questions & Answers

Files with special characters - how to remove

Hi, I have a directory that has a file which contained special characters in the filename. Can someone please advise how to remove the file, preferably with a rm -i ? Thanks in advance. Listing is as below: {oracle}> ls -1b bplog.bkup.001 bplog.bkup.002 bplog.bkup.003 bplog.bkup.004... (1 Reply)
Discussion started by: newbie_01
1 Replies

5. Shell Programming and Scripting

remove special characters

hello all I am writing a perl code and i wish to remove the special characters for text. I wish to remove all extended ascii characters. If the list of special characters is huge, how can i do this using substitute command s/specialcharacters/null/g I really want to code like... (3 Replies)
Discussion started by: vasuarjula
3 Replies

6. UNIX for Dummies Questions & Answers

Remove Unicode/special chars from XML

Hi, We are receiving an XML file in Unix which has some special characters between tags like '^' etc <Tag> 1e^O7f%<2304e.$d8f57e8^Bf-&e.^Zh7/327e^O7 </Tag> We need to remove all special characters like ^ ones and also any '&' or '<' or '>' being sent within the start and close tags i.e.... (6 Replies)
Discussion started by: dsrookie7
6 Replies

7. Shell Programming and Scripting

Remove the special characters from field

Hi, In source data few of columns are having special charates(like *) due to this i am not able to display the data into flat file.it's displaying the some of junk data into the flat file. source dataExample: Address1="XDERFTG * HYJUYTG" how to remove the special charates in a string (2 Replies)
Discussion started by: koti_rama
2 Replies

8. Shell Programming and Scripting

How to remove some special characters in a string?

Hi, I have string like this ="Lookup Procedure" But i want the output like this Lookup Procedure =," should be removed. Please suggest me the solution. Regards, Madhuri (2 Replies)
Discussion started by: srimadhuri
2 Replies

9. Shell Programming and Scripting

How to remove special characters?

Hi Gurus, I have file which contains some unicode charachator like "ü". I want to replace it with some charactors. I searched in internet and got command sed "s/ü/-/g", but I don't know how to type ü in unix command line. Please help me for this one. Thanks in advance (7 Replies)
Discussion started by: ken6503
7 Replies

10. Shell Programming and Scripting

Remove Special Characters Within Text

Hi, I have a "|" delimited file that is exported from a database. There is one column in the file which has description/comments entered by some application user. It has "Control-M" character and "New Line" character in between the text. Hence, when i export the data, this record with the new... (4 Replies)
Discussion started by: tarun.trehan
4 Replies
ASCII(1)						      General Commands Manual							  ASCII(1)

NAME
ascii, unicode - interpret ASCII, Unicode characters SYNOPSIS
ascii [ -8 ] [ -oxdbn ] [ -nct ] [ text ] unicode [ -nt ] hexmin-hexmax unicode [ -t ] hex [ ... ] unicode [ -n ] characters look hex /lib/unicode DESCRIPTION
Ascii prints the ASCII values corresponding to characters and vice versa; under the -8 option, the ISO Latin-1 extensions (codes 0200-0377) are included. The values are interpreted in a settable numeric base; -o specifies octal, -d decimal, -x hexadecimal (the default), and -bn base n. With no arguments, ascii prints a table of the character set in the specified base. Characters of text are converted to their ASCII val- ues, one per line. If, however, the first text argument is a valid number in the specified base, conversion goes the opposite way. Control characters are printed as two- or three-character mnemonics. Other options are: -n Force numeric output. -c Force character output. -t Convert from numbers to running text; do not interpret control characters or insert newlines. Unicode is similar; it converts between UTF and character values from the Unicode Standard (see utf(6)). If given a range of hexadecimal numbers, unicode prints a table of the specified Unicode characters -- their values and UTF representations. Otherwise it translates from UTF to numeric value or vice versa, depending on the appearance of the supplied text; the -n option forces numeric output to avoid ambigu- ity with numeric characters. If converting to UTF , the characters are printed one per line unless the -t flag is set, in which case the output is a single string containing only the specified characters. Unlike ascii, unicode treats no characters specially. The output of ascii and unicode may be unhelpful if the characters printed are not available in the current font. The file /lib/unicode contains a table of characters and descriptions, sorted in hexadecimal order, suitable for look(1) on the lower case hex values of characters. EXAMPLES
ascii -d Print the ASCII table base 10. unicode p Print the hex value of `p'. unicode 2200-22f1 Print a table of miscellaneous mathematical symbols. look 039 /lib/unicode See the start of the Greek alphabet's encoding in the Unicode Standard. FILES
/lib/unicode table of characters and descriptions. SOURCE
/sys/src/cmd/ascii.c /sys/src/cmd/unicode.c SEE ALSO
look(1) tcs(1), utf(6), font(6), ASCII(1)
All times are GMT -4. The time now is 07:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy