|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Unable to identify the special characters beyond the range of "[\x80-\xFF]"
Post 302957864 by Abhijit Sen on Friday 16th of October 2015 03:53:29 AM
10-16-2015
My file consist Ä which is an UTF-16 character , which can't be ident
ified by grep -P -n "[\x80-\xFF]".
And in the same file à is present which is an UTF-8 character and will be identified by grep -P -n "[\x80-\xFF]" .
One possibility that all the UTF-16 can be converted to UTF-8 if no regular expression is present. But it will take time to convert all of them.
Also not sure when it will convert UTF-16(2 byte char) to UTF-8(1 byte),how the alignment will be adjusted.
Looking for a way to find them .
ified by grep -P -n "[\x80-\xFF]".
And in the same file à is present which is an UTF-8 character and will be identified by grep -P -n "[\x80-\xFF]" .
One possibility that all the UTF-16 can be converted to UTF-8 if no regular expression is present. But it will take time to convert all of them.
Also not sure when it will convert UTF-16(2 byte char) to UTF-8(1 byte),how the alignment will be adjusted.
Looking for a way to find them .
| Moderator's Comments: | ||
|
Last edited by Don Cragun; 10-16-2015 at 05:15 AM.. Reason: Get rid of HTML tags, add ICODE tags.
|
|
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi friends
I need to splict special character "|" here.
Here is my script which giving error
LINE=INVTRAN|cd /home/msgGoogle
TraxFolderType=`awk -F"|" '{print $1}' $LINE`
filePath=`awk -F"|" '{print $2}' $LINE`
echo "TraxFolderType: "$TraxFolderType
echo "filePath :"$filePath
... (3 Replies)
Discussion started by: krishna9
3 Replies
2. Shell Programming and Scripting
Dear friends, following is the output of a script from which I want to remove spaces and new-line characters.
Example:-
Line1 abcdefghijklmnopqrstuvwxyz
Line2 mnopqrstuvwxyzabcdefghijkl
Line3 opqrstuvwxyzabcdefdefg
Here in above example, at every starting line there is a “tab” &... (4 Replies)
Discussion started by: anushree.a
4 Replies
3. SuSE
keep getting an error when I try to revert to a snapshot:
"VMDB Failure" followed by "Unable to open snapshot file"
Im using vmware server 1.0.4, host OS is windows xp and guest OS is SLES.
Is there anything I can do to recover the snapshot or am I in trouble!?!?! (0 Replies)
Discussion started by: s_linux
0 Replies
4. Shell Programming and Scripting
both ksh/bash support this 2 special variables, Is there any document for reference?
1) "-" is $OLDPWD
2) "$_" is last argument of previous command. (4 Replies)
Discussion started by: honglus
4 Replies
5. Shell Programming and Scripting
Hi,
My input has much more lines, but few of them are below
pin(IDF) {
direction : input;
drc_pinsigtype : signal;
pin(SELDIV6) {
direction : input;
drc_pinsigtype : ... (3 Replies)
Discussion started by: nehashine
3 Replies
6. Shell Programming and Scripting
been a while so i'm a bit rusty and need a little help. writing a script that needs to compare $EXECHOST(a number) against a numerical range and then set a value. below isn't working but should give you folks an idea of my goal:
if ; then
echo "This is a 32B machine, exiting..."
if ;... (4 Replies)
Discussion started by: crimso
4 Replies
7. Shell Programming and Scripting
Need HELP with AWK split. Need to check for "special characters" in string before splitting the file
Hi Experts.
I'm stuck with the below AWK code where i'm trying to move the records containing any special characters in the last field to a bad file.
awk -F, '{if ($NF ~ /^|^/) print >"goodfile";else print >"badfile"}' filename
sample data
1,abc,def,1234,A *
2,bed,dec,342,* A ... (6 Replies)
Discussion started by: shell_boy23
6 Replies
8. Shell Programming and Scripting
Hi all.
I have a .txt file that I need to sort it
My file is like:
1- 88 chain0 MASTER (FF-TE) FFFF 1962510 /TCK T FD2TQHVTT1 /jtagc/jtag_instreg/updateinstr_reg_1 dff1 (TI,SO)
2- ... (10 Replies)
Discussion started by: Behrouzx77
10 Replies
9. UNIX for Dummies Questions & Answers
I'm not really sure which forum this question should go into, so I'm posting it here.
I work with AIX and RHEL systems using PuTTY (Release 0.60_q1.129) from a Windows 7 workstation.
Some of the files we get from z/OS use "special" characters as delimiters. These characters include Hex 18... (7 Replies)
Discussion started by: derndingle
7 Replies
LEARN ABOUT DEBIAN
load_txt_font
load_txt_font(3alleg4) Allegro manual load_txt_font(3alleg4) NAME
load_txt_font - Loads a font script. Allegro game programming library. SYNOPSIS
#include <allegro.h> FONT *load_txt_font(const char *filename, RGB *pal, void *param) DESCRIPTION
This function can be used to load scripted fonts. The script file contains a number of lines in the format "filename start end", which specify the source file for that range of characters, the Unicode value of the first character in the range, and the end character in the range (optional, if left out, the entire input file will be grabbed). If the filename is replaced by a hyphen, more characters will be grabbed from the previous input file. For example, the script: ascii.fnt 0x20 0x7F - 0xA0 0xFF dingbats.fnt 0x1000 would import the first 96 characters from ascii.fnt as the range 0x20-0x7F, the next 96 characters from ascii.fnt as the range 0xA0-0xFF, and the entire contents of dingbats.fnt starting at Unicode position 0x1000. RETURN VALUE
Returns a pointer to the font or NULL on error. Remember that you are responsible for destroying the font when you are finished with it to avoid memory leaks. SEE ALSO
register_font_file_type(3alleg4), load_font(3alleg4) Allegro version 4.4.2 load_txt_font(3alleg4)