Sponsored Content
Top Forums Shell Programming and Scripting Cleaning up incorrect/unknown characters Post 302831977 by m69w on Friday 12th of July 2013 10:34:43 AM
Old 07-12-2013
I didn't know about these until you mentionned them. So i gave it a try using all possible charset available like this but none seems to work out. Either it returns the original character or an error

Code:
recode --list|egrep -v '^\/|^:'|while read s; do recode $s < unknown_character.txt 2>/dev/null ; done|egrep ^Unknown

 

10 More Discussions You Might Find Interesting

1. AIX

doing some spring cleaning....

USERS="me you jim joe sue" for user in ${USERS}; do rmuser -p $user usrdir=`cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` rm -fr `cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` echo Deleting: $user '\t' REMOVING: $usrdir done This is for AIX ONLY!!! but easily ported to... (0 Replies)
Discussion started by: Optimus_P
0 Replies

2. UNIX for Dummies Questions & Answers

Database cleaning software

Hi everybody, I have been given a task to find the names of some products that can clean up databases by removing confidential information. The situation is that a client imports data from public sources (government websites, etc.) but that this data sometimes includes things like Social... (0 Replies)
Discussion started by: rhfrommn
0 Replies

3. SCO

Tape drive cleaning

Hello everyone, First, thank you anyone who might be able to help : ) !! here it is, I am using SCO at my business, and I back up everything to a tape drive. I want to do my cleaning of the drive, and i put in the cartridge to the drive, it recognizes it yet it will not engage the... (5 Replies)
Discussion started by: RichardHeadd
5 Replies

4. Solaris

PING - Unknown host 127.0.0.1, Unknown host localhost - Solaris 10

Hello, I have a problem - I created a chrooted jail for one user. When I'm logged in as root, everything work fine, but when I'm logged in as a chrooted user - I have many problems: 1. When I execute the command ping, I get weird results: bash-3.00$ usr/sbin/ping localhost ... (4 Replies)
Discussion started by: Przemek
4 Replies

5. Shell Programming and Scripting

read in a file character by character - replace any unknown ASCII characters with spa

Can someone help me to write a script / command to read in a file, character by character, replace any unknown ASCII characters with space. then write out the file to a new filename/ Thanks! (1 Reply)
Discussion started by: raghav525
1 Replies

6. UNIX for Dummies Questions & Answers

AWK Data Cleaning

Hello, I am trying to analyze data I recently ran, and the only way to efficiently clean up the data is by using an awk file. I am very new to awk and am having great difficulty with it. In $8 and $9, for example, I am trying to delete numbers that contain 1. I cannot find any tutorials that... (20 Replies)
Discussion started by: carmar87
20 Replies

7. Shell Programming and Scripting

File cleaning

HI , I am getting the source data as below. Source Data CDR_Data,,,,, F1,F2,F3,F4,F5,F6 5,5,6,7,8,7 6,6,g,,, 7,7,76,,, 8,8,gt,,, 9,9,df ,d,d,d ,,,,, (4 Replies)
Discussion started by: wangkc
4 Replies

8. Shell Programming and Scripting

cleaning the file

Hi, I have a file with multiple rows. each row has 8 columns. Column 8 has entries separated by commas. I want to exclude all the rows in which column 8 has more than 3 commas. 1234#0/1 - ABC_1234 3 ATGCATGCATGC HHHIIIGIHVF 1 49:T>C,60:T>C,78:C>A,76:G>T,65:T>G Thanks, Diya (3 Replies)
Discussion started by: Diya123
3 Replies

9. Shell Programming and Scripting

Removing characters from end of line (length unknown)

Hi I have a file which contains wrong XML, There are some garbage characters at the end of line that I want to get rid of. Example: <request type="product" ><attributes><pair><name>q</name><value><!]></value></pair><pair><name>start</name><value>1</value></pair></attributes></request>�J ... (7 Replies)
Discussion started by: dirtyd0ggy
7 Replies

10. Shell Programming and Scripting

Cleaning output using awk

I have some small problem with my code. data.html <TD class="statuscol2">c</TD> <TD class="statuscol3">18</TD> <TD class="statuscol4"><SPAN TITLE="#04">test4</SPAN></TD> <TD... (4 Replies)
Discussion started by: Jotne
4 Replies
PX_SET_TARGETENCODING(3)				     Library Functions Manual					  PX_SET_TARGETENCODING(3)

NAME
PX_set_targetencoding -- Set encoding for output SYNOPSIS
#include <paradox.h> int PX_set_targetencoding(pxdoc_t *pxdoc, const char *encoding) DESCRIPTION
Sets the target encoding for all alpha fields. It is used when data is retrieved from a record. The encoding must be a name as it is described by the man and info pages of recode or iconv whatever is used. A common value is `latin1' on Unix systems. If you do not set the encoding the function PX_get_data_alpha(3) will return field data in the original encoding otherwise it will recode the field data. This function is a short cut form PX_set_parameter(pxdoc, "targetencoding", encoding) and is deprecated. RETURN VALUE
Returns -2 if recoding is not supported and a value < 0 in case of an error, otherwise 0. SEE ALSO
PX_get_data_alpha(3), PX_set_inputencoding(3), PX_set_parameter(3), iconv(1), recode(1) AUTHOR
This manual page was written by Uwe Steinmann uwe@steinmann.cx. PX_SET_TARGETENCODING(3)
All times are GMT -4. The time now is 01:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy