Issue with Keyboard or Char Encoding During Migration Post: 303046235

Sponsored Content

Top Forums Programming Issue with Keyboard or Char Encoding During Migration Post 303046235 by hicksd8 on Tuesday 28th of April 2020 07:35:53 AM

04-28-2020

Moderator

Hi All,

As Neo says I have been spending a bit of time on this migration integrity issue.

The irritating "Thingy" (white diamond with question mark in the middle) is officially the Unicode symbol called "Replacement character". The character set inserts this as a placeholder for a character that it doesn't understand. IMHO, the issue here is simply that the migration script (or whatever process) SHOULD understand all the characters on our old site. Yes, we already have "Replacement characters" on the old site switch probably emanated from a long ago upgrade from ascii to Unicode, or from Unicode version x to Unicode version y. As Neo says, replacement character symbols in our old site must be ignored because there's nothing we can do about them now apart from manually edit them out as time goes on.

However, I believe that the currently used (Discourse provided??) process is stuffed because it doesn't understand some of the perfectly correct text on our old site. It even screws up a thread title on the old site containing the replacement character symbol - look at this......

Post migration
How to grep i?1/2 symbol? - Shell Programming and Scripting - UNIX.COM Community

Pre migration
How to grep � symbol?

So the process doesn't even understand it's own Unicode character set!!!!

So FWIW, I've come to the conclusion that trying to modify our old dB is futile as the process will probably find something else to screw up.

Indeed, if you follow the first link I posted on this thread further back, others are having the same issue.

That's my update thus far. I'll report back again as my investigation continues.

EDIT: Replacement character symbol is U+FFFD

hicksd8

View Public Profile for hicksd8

Find all posts by hicksd8

7 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how2 get single char from keyboard w/o enter

I am writing a bash shell menu and would like to get a char immediately after a key is pressed. This script does not work but should give you an idea of what I am trying to do.... Thanks for the help #! /bin/bash ANSWER="" echo -en "Choose item...\n" until do $ANSWER = $STDIN ...

2. Shell Programming and Scripting

Encoding of a text issue

I created one file on windows system and is visible as : TestTable,INSERT,�večilnin1ईगल受害者是第,2010-02-02 10:10:10.612447,137277,ईगल受害者是第�večilnin!@#$%^&*()_+=-{}] But when send this file to unix system, the file is visible as : TestTable,INSERT,žvečilnin1ई�-ल �-害�...是第,2010-02-02 ...

3. Solaris

Solaris 10 p2v migration issue

Hi All, We need to move Physical Solaris 10 system to Virtual Solaris 10(p2v). Both the servers having Solaris 10(Generic_147440-25) means physical server which we are going to move is having Solaris 10 and this physical server will be converted as a virtualserver on another physical server...

4. UNIX for Dummies Questions & Answers

Strange Keyboard and Mouse Issue

Hello All, PC: CuBox-i (*i.MX6) Mini-PC OS: openSUSE 13.1 (Bottle) (armv7hl) Kernel: 3.14.14-cubox-i # uname -a Linux CuBox-HQ 3.14.14-cubox-i #1 SMP Sat Sep 13 03:48:24 UTC 2014 armv7l armv7l armv7l GNU/LinuxSo I've been having this random issue happen on this PC where a few strange...

5. AIX

AIX Migration issue with EMC ODM sets

Hi Experts , I want to start migrating our AIX 6.1 to AIX 7.1 . I am planning to use alt_disk_migration . Chris gibson has awesome documentation in the internet. However I am running into an issue with EMC odm filesets . So my current OS is AIX 6.1. and I have this : lslpp -l | grep EMC ...

6. Shell Programming and Scripting

AIX to RHEL migration - awk treating 0e[0-9]+ as 0 instead of string issue

Greetings Experts, We are migrating from AIX to RHEL Linux. I have created a script to verify and report the NULLs and SPACEs in the key columns and duplicates on key combination of "|" delimited set of big files. Following is the code that was successfully running in AIX. awk -F "|" 'BEGIN {...

7. Solaris

View file encoding then change encoding.

Hi all!! I�m using command file -i myfile.xml to validate XML file encoding, but it is just saying regular file . I�m expecting / looking an output as UTF8 or ANSI / ASCII Is there command to display the files encoding? Thank you!

LEARN ABOUT NETBSD

akbd

AKBD(4) 						   BSD Kernel Interfaces Manual 						   AKBD(4)

NAME

     akbd -- Apple Desktop Bus keyboard driver for wscons

SYNOPSIS

     akbd* at obio?
     wskbd* at akbd? console ?

     options ALTXBUTTONS
     options CAPS_IS_CONTROL
     options FORCE_FUNCTION_KEYS

DESCRIPTION

     This driver provides the wscons(4) driver with support for Apple Desktop Bus keyboards.

     To work around the limited number of buttons found on most ADB mice, one can map key sequences to trigger mouse button events.  To map
     Option+1, Option+2, Option+3 to mouse buttons 1, 2, and 3 respectively, add the following line to your kernel configuration file:

     options ALTXBUTTONS

     On macppc systems it is possible to tweak the keyboard driver to treat the caps lock key on an ADB keyboard as a control key.  This requires
     special remapping because of ADB's strange emulation of a mechanically-locked key.  To enable this code add the following line to your kernel
     configuration file:

     options CAPS_IS_CONTROL

     On macppc PowerBooks, several function keys double as "hot keys" (brightness, volume, eject) when the Fn modifier is held down.  Mac OS X
     likes to reprogram the keyboard controller to send hot key events when Fn is not held down and send function key events when it is.  To
     transform the non-keyboard "button" events back into function key events, place the following line in your kernel configuration file:

     options FORCE_FUNCTION_KEYS

SUPPORTED HARDWARE

     NetBSD is known to support the following ADB keyboards:

	   o   On-board keyboards on PowerBook models

	   o   Apple Standard Keyboard

	   o   Apple Keyboard II

	   o   Apple Extended Keyboard

	   o   Apple Extended Keyboard II

	   o   Apple Adjustable Keyboard

	   o   Most third-party ADB keyboards are supported

SEE ALSO

     adb(4), wscons(4), wskbd(4), wsconsctl(8)

BUGS

     The number pad on extended keyboards does not send out the proper key codes for many applications.

     The LEDs on extended keyboards are not functional under NetBSD.

BSD
								September 21, 2003							       BSD