Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Find and replace special characters in a file Post 302250718 by sujithchandra on Friday 24th of October 2008 06:30:14 AM
Old 10-24-2008
MySQL Find and replace special characters in a file

HI All

I need a shell script ehich removes all special characters from file and converts the file to UTF-* format

Specail characters to be removed must be configurable.

strIllegal = @"?/><,:;""'{[}]|\\+=-)(*&^%$#@!~`";

Please help me in getting this script as my scripting skilla are week.

Regards
Sujith Chandra C
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Replace Special characters in a file

Hi, I have a data like this in a file, 402003279034002000100147626030003300010000000000002000029000000 ær^M^\MÍW^H I need to replace those special char to some other char like # or $ Is there any ways to do it... I tried commands tr,sed and many but it was not able to replace because... (1 Reply)
Discussion started by: solai
1 Replies

2. UNIX for Dummies Questions & Answers

Help with find and replace w/string containing special characters

Can I get some help on this please, I have looked at the many post with similar questions and have tried the solutions and they are not working for my scenario which is: I have a text file (myfile) that contains b_log=$g_log/FILENAME.log echo "Begin processing file FILENAME " >> $b_log ... (4 Replies)
Discussion started by: CAGIRL
4 Replies

3. Shell Programming and Scripting

Single/Multiple Line with Special characters - Find & Replace in Unix Script

Hi, I am creating a script to do a find and replace single/multiple lines in a file with any number of lines. I have written a logic in a script that reads a reference file say "findrep" and populates two variables $FIND and $REPLACE print $FIND gives Hi How r $u Rahul() Note:... (0 Replies)
Discussion started by: r_sarnayak
0 Replies

4. Solaris

How to replace special characters in vi?

Hi , I want to replace the special characters in the file. For eg: cat abc 1234/4455/acb 234/k/lll/ 234`fs`fd I want to replace / and ` with the letter a and the output should like below. How to achieve this. 1234a4455aacb 234akallla 234afsafd (2 Replies)
Discussion started by: rogerben
2 Replies

5. Shell Programming and Scripting

Replace special characters

I have a line ending with special character and 0 The special character is the field separator for this line in VI mode the file will look like below, but while cat the special character wont display i know the hexa code for the special character ^_ is \x1f and ascii code is \0037, ... (0 Replies)
Discussion started by: ratheeshjulk
0 Replies

6. Shell Programming and Scripting

Replace special characters with Escape characters?

i need to replace the any special characters with escape characters like below. test!=123-> test\!\=123 !@#$%^&*()-= to be replaced by \!\@\#\$\%\^\&\*\(\)\-\= (8 Replies)
Discussion started by: laknar
8 Replies

7. Shell Programming and Scripting

How to replace special characters?

Hi Unix Guru, I have an requirement for replace some specail characters in a file, my file came from mainframe. please see below example: when open it with vi 17896660|89059215|04/24/1998 00:00:00.000000| abc 123-453-1312^M<85>^M<85>|124557 if I run cat -v I got following:... (25 Replies)
Discussion started by: ken002
25 Replies

8. Shell Programming and Scripting

Find out special characters from xml file

Hi....I have a xml file which is having lots of special characters which I need to find out and put the distinct list of those into a text file. The list of special characters is not specific, it can be anything at different point of time. Can anyone help me to find out the same and list out? ... (10 Replies)
Discussion started by: Krishanu Saha
10 Replies

9. UNIX for Dummies Questions & Answers

Search special characters in a file and replace with meaningful text messages like Hello

Search special characters in a file and replace with meaningful text messages like Hello (2 Replies)
Discussion started by: raka_rjit
2 Replies

10. UNIX for Advanced & Expert Users

How to replace special characters?

Hi Team, I have data like this. |*|.5|*|0.2|*|A.B|*| Would like to add zero (0) before the decimal point where there is no zero as |*|0.5|*|0.2|*|A.B|*| How to replace |*|. with |*|0. I tried below command which didn't work echo '|*|.5|*|0.2|*|A.B|*' | sed... (4 Replies)
Discussion started by: Ravi.K
4 Replies
TCS(1)							      General Commands Manual							    TCS(1)

NAME
tcs - translate character sets SYNOPSIS
tcs [ -slcv ] [ -f ics ] [ -t ocs ] [ file ... ] DESCRIPTION
Tcs interprets the named file(s) (standard input default) as a stream of characters from the ics character set or format, converts them to runes, and then converts them into a stream of characters from the ocs character set or format on the standard output. The default value for ics and ocs is utf, the UTF encoding described in utf(6). The -l option lists the character sets known to tcs. Processing continues in the face of conversion errors (the -s option prevents reporting of these errors). The -c option forces the output to contain only cor- rectly converted characters; otherwise, 0x80 characters will be substituted for UTF encoding errors and 0xFFFD characters will substituted for unknown characters. The -v option generates various diagnostic and summary information on standard error, or makes the -l output more verbose. Tcs recognizes an ever changing list of character sets. In particular, it supports a variety of Russian and Japanese encodings. Some of the supported encodings are utf The Plan 9 UTF encoding, known by ISO as UTF-8 utf1 The deprecated original UTF encoding from ISO 10646 ascii 7-bit ASCII 8859-1 Latin-1 (Central European) 8859-2 Latin-2 (Czech .. Slovak) 8859-3 Latin-3 (Dutch .. Turkish) 8859-4 Latin-4 (Scandinavian) 8859-5 Part 5 (Cyrillic) 8859-6 Part 6 (Arabic) 8859-7 Part 7 (Greek) 8859-8 Part 8 (Hebrew) 8859-9 Latin-5 (Finnish .. Portuguese) koi8 KOI-8 (GOST 19769-74) jis-kanji ISO 2022-JP ujis EUC-JX: JIS 0208 ms-kanji Microsoft, or Shift-JIS jis (from only) guesses between ISO 2022-JP, EUC or Shift-Jis gb Chinese national standard (GB2312-80) big5 Big 5 (HKU version) unicode Unicode Standard 1.0 tis Thai character set plus ASCII (TIS 620-1986) msdos IBM PC: CP 437 atari Atari-ST character set EXAMPLES
tcs -f 8859-1 Convert 8859-1 (Latin-1) characters into UTF format. tcs -s -f jis Convert characters encoded in one of several shift JIS encodings into UTF format. Unknown Kanji will be converted into 0xFFFD char- acters. tcs -lv Print an up to date list of the supported character sets. SOURCE
/sys/src/cmd/tcs SEE ALSO
ascii(1), rune(2), utf(6). TCS(1)
All times are GMT -4. The time now is 01:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy