Visit Our UNIX and Linux User Community

Top Forums Programming Issue with Keyboard or Char Encoding During Migration Post 303046235 by hicksd8 on Tuesday 28th of April 2020 07:35:53 AM
Old 04-28-2020
Hi All,

As Neo says I have been spending a bit of time on this migration integrity issue.

The irritating "Thingy" (white diamond with question mark in the middle) is officially the Unicode symbol called "Replacement character". The character set inserts this as a placeholder for a character that it doesn't understand. IMHO, the issue here is simply that the migration script (or whatever process) SHOULD understand all the characters on our old site. Yes, we already have "Replacement characters" on the old site switch probably emanated from a long ago upgrade from ascii to Unicode, or from Unicode version x to Unicode version y. As Neo says, replacement character symbols in our old site must be ignored because there's nothing we can do about them now apart from manually edit them out as time goes on.

However, I believe that the currently used (Discourse provided??) process is stuffed because it doesn't understand some of the perfectly correct text on our old site. It even screws up a thread title on the old site containing the replacement character symbol - look at this......

Post migration
How to grep i?1/2 symbol? - Shell Programming and Scripting - UNIX.COM Community

Pre migration
How to grep � symbol?

So the process doesn't even understand it's own Unicode character set!!!!

So FWIW, I've come to the conclusion that trying to modify our old dB is futile as the process will probably find something else to screw up.

Indeed, if you follow the first link I posted on this thread further back, others are having the same issue.

That's my update thus far. I'll report back again as my investigation continues.

EDIT: Replacement character symbol is U+FFFD
 

7 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how2 get single char from keyboard w/o enter

I am writing a bash shell menu and would like to get a char immediately after a key is pressed. This script does not work but should give you an idea of what I am trying to do.... Thanks for the help #! /bin/bash ANSWER="" echo -en "Choose item...\n" until do $ANSWER = $STDIN ... (2 Replies)
Discussion started by: jwzumwalt
2 Replies

2. Shell Programming and Scripting

Encoding of a text issue

I created one file on windows system and is visible as : TestTable,INSERT,večilnin1ईगल受害者是第,2010-02-02 10:10:10.612447,137277,ईगल受害者是第večilnin!@#$%^&*()_+=-{}] But when send this file to unix system, the file is visible as : TestTable,INSERT,žvečilnin1ई-ल -害...是第,2010-02-02 ... (4 Replies)
Discussion started by: Shaishav Shah
4 Replies

3. Solaris

Solaris 10 p2v migration issue

Hi All, We need to move Physical Solaris 10 system to Virtual Solaris 10(p2v). Both the servers having Solaris 10(Generic_147440-25) means physical server which we are going to move is having Solaris 10 and this physical server will be converted as a virtualserver on another physical server... (9 Replies)
Discussion started by: sb200
9 Replies

4. UNIX for Dummies Questions & Answers

Strange Keyboard and Mouse Issue

Hello All, PC: CuBox-i (*i.MX6) Mini-PC OS: openSUSE 13.1 (Bottle) (armv7hl) Kernel: 3.14.14-cubox-i # uname -a Linux CuBox-HQ 3.14.14-cubox-i #1 SMP Sat Sep 13 03:48:24 UTC 2014 armv7l armv7l armv7l GNU/LinuxSo I've been having this random issue happen on this PC where a few strange... (12 Replies)
Discussion started by: mrm5102
12 Replies

5. AIX

AIX Migration issue with EMC ODM sets

Hi Experts , I want to start migrating our AIX 6.1 to AIX 7.1 . I am planning to use alt_disk_migration . Chris gibson has awesome documentation in the internet. However I am running into an issue with EMC odm filesets . So my current OS is AIX 6.1. and I have this : lslpp -l | grep EMC ... (7 Replies)
Discussion started by: JME2015
7 Replies

6. Shell Programming and Scripting

AIX to RHEL migration - awk treating 0e[0-9]+ as 0 instead of string issue

Greetings Experts, We are migrating from AIX to RHEL Linux. I have created a script to verify and report the NULLs and SPACEs in the key columns and duplicates on key combination of "|" delimited set of big files. Following is the code that was successfully running in AIX. awk -F "|" 'BEGIN {... (5 Replies)
Discussion started by: chill3chee
5 Replies

7. Solaris

View file encoding then change encoding.

Hi all!! Im using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . Im expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you! (2 Replies)
Discussion started by: mrreds
2 Replies
Tcl_UniCharIsAlpha(3)					      Tcl Library Procedures					     Tcl_UniCharIsAlpha(3)

__________________________________________________________________________________________________________________________________________________

NAME
Tcl_UniCharIsAlnum, Tcl_UniCharIsAlpha, Tcl_UniCharIsControl, Tcl_UniCharIsDigit, Tcl_UniCharIsGraph, Tcl_UniCharIsLower, Tcl_UniCharIsPrint, Tcl_UniCharIsPunct, Tcl_UniCharIsSpace, Tcl_UniCharIsUpper, Tcl_UniCharIsWordChar - routines for classification of Tcl_UniChar characters SYNOPSIS
#include <tcl.h> int Tcl_UniCharIsAlnum(ch) int Tcl_UniCharIsAlpha(ch) int Tcl_UniCharIsControl(ch) int Tcl_UniCharIsDigit(ch) int Tcl_UniCharIsGraph(ch) int Tcl_UniCharIsLower(ch) int Tcl_UniCharIsPrint(ch) int Tcl_UniCharIsPunct(ch) int Tcl_UniCharIsSpace(ch) int Tcl_UniCharIsUpper(ch) int Tcl_UniCharIsWordChar(ch) ARGUMENTS
int ch (in) The Tcl_UniChar to be examined. _________________________________________________________________ DESCRIPTION
All of the routines described examine Tcl_UniChars and return a boolean value. A non-zero return value means that the character does belong to the character class associated with the called routine. The rest of this document just describes the character classes associated with the various routines. Note: A Tcl_UniChar is a Unicode character represented as an unsigned, fixed-size quantity. CHARACTER CLASSES
Tcl_UniCharIsAlnum tests if the character is an alphanumeric Unicode character. Tcl_UniCharIsAlpha tests if the character is an alphabetic Unicode character. Tcl_UniCharIsControl tests if the character is a Unicode control character. Tcl_UniCharIsDigit tests if the character is a numeric Unicode character. Tcl_UniCharIsGraph tests if the character is any Unicode print character except space. Tcl_UniCharIsLower tests if the character is a lowercase Unicode character. Tcl_UniCharIsPrint tests if the character is a Unicode print character. Tcl_UniCharIsPunct tests if the character is a Unicode punctuation character. Tcl_UniCharIsSpace tests if the character is a whitespace Unicode character. Tcl_UniCharIsUpper tests if the character is an uppercase Unicode character. Tcl_UniCharIsWordChar tests if the character is alphanumeric or a connector punctuation mark. KEYWORDS
unicode, classification Tcl 8.1 Tcl_UniCharIsAlpha(3)

Featured Tech Videos

All times are GMT -4. The time now is 04:58 PM.
Unix & Linux Forums Content Copyright 1993-2021. All Rights Reserved.
Privacy Policy