Sponsored Content
Top Forums UNIX for Dummies Questions & Answers remove special and unicode characters Post 302264962 by otheus on Friday 5th of December 2008 08:16:48 AM
Old 12-05-2008
To eliminate all but printable characters:
Code:
grep -v '[^[:print]]' test.txt

There's also [:punct:] which are all punctuation characters. Or you can search for anything that doesn't have a normal letter or number:
Code:
grep -v '[^A-Za-z0-9]'

 

10 More Discussions You Might Find Interesting

1. Programming

How to display unicode characters / unicode string

I have a stream of characters like "\u8BBE\u5907\u7BA1" and i want to display it. I tried following things already without any luck. 1) printf("%s",L("\u8BBE\u5907\u7BA1")); 2) printf("%lc",0x8BBE); 3) setlocale followed by fwide followed by wprintf 4) also changed the local manually... (3 Replies)
Discussion started by: jackdorso
3 Replies

2. UNIX for Dummies Questions & Answers

Remove directory that has special Characters

Hi All, I have a script written that creates a new directory within the shell program and if a parameter isn't passed in, it creates a strange directory name by mistake. So I have a directory like "-_12" and I am unable to remove it. I tried removing it using double quote and many others. I have... (12 Replies)
Discussion started by: datherriault
12 Replies

3. UNIX for Dummies Questions & Answers

How to Remove Special Characters

Dear Members, We have a file which contains some special characters. I need to replace these special character by a new line character(\n). The Special character is \x85. I am not sure what this character means and how we can remove it. Any inputs are greatly appreciated. Thanks... (5 Replies)
Discussion started by: sandeep_1105
5 Replies

4. UNIX for Dummies Questions & Answers

Files with special characters - how to remove

Hi, I have a directory that has a file which contained special characters in the filename. Can someone please advise how to remove the file, preferably with a rm -i ? Thanks in advance. Listing is as below: {oracle}> ls -1b bplog.bkup.001 bplog.bkup.002 bplog.bkup.003 bplog.bkup.004... (1 Reply)
Discussion started by: newbie_01
1 Replies

5. Shell Programming and Scripting

remove special characters

hello all I am writing a perl code and i wish to remove the special characters for text. I wish to remove all extended ascii characters. If the list of special characters is huge, how can i do this using substitute command s/specialcharacters/null/g I really want to code like... (3 Replies)
Discussion started by: vasuarjula
3 Replies

6. UNIX for Dummies Questions & Answers

Remove Unicode/special chars from XML

Hi, We are receiving an XML file in Unix which has some special characters between tags like '^' etc <Tag> 1e^O7f%<2304e.$d8f57e8^Bf-&e.^Zh7/327e^O7 </Tag> We need to remove all special characters like ^ ones and also any '&' or '<' or '>' being sent within the start and close tags i.e.... (6 Replies)
Discussion started by: dsrookie7
6 Replies

7. Shell Programming and Scripting

Remove the special characters from field

Hi, In source data few of columns are having special charates(like *) due to this i am not able to display the data into flat file.it's displaying the some of junk data into the flat file. source dataExample: Address1="XDERFTG * HYJUYTG" how to remove the special charates in a string (2 Replies)
Discussion started by: koti_rama
2 Replies

8. Shell Programming and Scripting

How to remove some special characters in a string?

Hi, I have string like this ="Lookup Procedure" But i want the output like this Lookup Procedure =," should be removed. Please suggest me the solution. Regards, Madhuri (2 Replies)
Discussion started by: srimadhuri
2 Replies

9. Shell Programming and Scripting

How to remove special characters?

Hi Gurus, I have file which contains some unicode charachator like "ü". I want to replace it with some charactors. I searched in internet and got command sed "s/ü/-/g", but I don't know how to type ü in unix command line. Please help me for this one. Thanks in advance (7 Replies)
Discussion started by: ken6503
7 Replies

10. Shell Programming and Scripting

Remove Special Characters Within Text

Hi, I have a "|" delimited file that is exported from a database. There is one column in the file which has description/comments entered by some application user. It has "Control-M" character and "New Line" character in between the text. Hence, when i export the data, this record with the new... (4 Replies)
Discussion started by: tarun.trehan
4 Replies
unicode(n)						       Unicode normalization							unicode(n)

__________________________________________________________________________________________________________________________________________________

NAME
unicode - Implementation of Unicode normalization SYNOPSIS
package require Tcl 8.3 package require unicode 1.0 ::unicode::fromstring string ::unicode::tostring uclist ::unicode::normalize form uclist ::unicode::normalizeS form string _________________________________________________________________ DESCRIPTION
This is an implementation in Tcl of the Unicode normalization forms. COMMANDS
::unicode::fromstring string Converts string to list of integer Unicode character codes which is used in unicode for internal string representation. ::unicode::tostring uclist Converts list of integers uclist back to Tcl string. ::unicode::normalize form uclist Normalizes Unicode characters list ulist according to form and returns the normalized list. Form form takes one of the following values: D (canonical decomposition), C (canonical decomposition, followed by canonical composition), KD (compatibility decomposi- tion), or KC (compatibility decomposition, followed by canonical composition). ::unicode::normalizeS form string A shortcut to ::unicode::tostring [unicode::normalize $form [::unicode::fromstring $string]]. Normalizes Tcl string and returns normalized string. EXAMPLES
% ::unicode::fromstring "u0410u0411u0412u0413" 1040 1041 1042 1043 % ::unicode::tostring {49 50 51 52 53} 12345 % % ::unicode::normalize D {7692 775} 68 803 775 % ::unicode::normalizeS KD "u1d2c" A % REFERENCES
[1] "Unicode Standard Annex #15: Unicode Normalization Forms", (http://unicode.org/reports/tr15/) AUTHORS
Sergei Golovan BUGS, IDEAS, FEEDBACK This document, and the package it describes, will undoubtedly contain bugs and other problems. Please report such in the category string- prep of the Tcllib SF Trackers [http://sourceforge.net/tracker/?group_id=12883]. Please also report any ideas for enhancements you may have for either package and/or documentation. SEE ALSO
stringprep(n) KEYWORDS
normalization, unicode COPYRIGHT
Copyright (c) 2007, Sergei Golovan <sgolovan@nes.ru> stringprep 1.0.0 unicode(n)
All times are GMT -4. The time now is 07:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy