Unable to identify the special characters beyond the range of "[\x80-\xFF]" Post: 302957743

Sponsored Content

Top Forums Shell Programming and Scripting Unable to identify the special characters beyond the range of "[\x80-\xFF]" Post 302957743 by Don Cragun on Wednesday 14th of October 2015 01:36:08 PM

10-14-2015

Registered User

Quote:

Originally Posted by bakunin

You might want to use the :print: character class the POSIX BRE regexp provide and negate that: [^:print:] and see how far that gets you.

Basically there is no pattern for what constitutes a printable or non-printable character: character "\9", which is a TAB just is that by convention, not because it is - in principle - any different from "\10" or "\8".

You might also want to identify your locale, which may establish so-called collating sequences. See more about this at this page.

I hope this helps.

bakunin

I don't know of an RE context where \9 would represent a tab (although \0x9 and \011 would when using an ASCII based character set).

The print character class is identified by [:print:]. A BRE matching a character in the print class is [[:print:]] and a BRE matching a character that is not in the print class is [^[:print:]]. The BRE [^:print:] would match any character other than :, i, n, p, r, and t.

Note also that the print class does not include control characters. To select a UTF-8 character that is not a character in the 7-bit ASCII character set (actually select each byte of one of those characters), you could use the BRE [^[:ctrl:][:print:]] while in the C or POSIX locale.

But, when working with ASCII, UTF-8, and 8859-* character sets, just filtering out bytes with the high order bit set should be sufficient.

Last edited by Don Cragun; 10-14-2015 at 03:45 PM.. Reason: Add notes.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to split special characters "|" using awk

Hi friends I need to splict special character "|" here. Here is my script which giving error LINE=INVTRAN|cd /home/msgGoogle TraxFolderType=`awk -F"|" '{print $1}' $LINE` filePath=`awk -F"|" '{print $2}' $LINE` echo "TraxFolderType: "$TraxFolderType echo "filePath :"$filePath ...

2. Shell Programming and Scripting

How to remove "New line characters" and "spaces" at a time

Dear friends, following is the output of a script from which I want to remove spaces and new-line characters. Example:- Line1 abcdefghijklmnopqrstuvwxyz Line2 mnopqrstuvwxyzabcdefghijkl Line3 opqrstuvwxyzabcdefdefg Here in above example, at every starting line there is a �tab� &...

3. SuSE

VMDB Failure" followed by "Unable to open snapshot file"

keep getting an error when I try to revert to a snapshot: "VMDB Failure" followed by "Unable to open snapshot file" Im using vmware server 1.0.4, host OS is windows xp and guest OS is SLES. Is there anything I can do to recover the snapshot or am I in trouble!?!?!

4. Shell Programming and Scripting

Question about special variables: "-" and "$_"

both ksh/bash support this 2 special variables, Is there any document for reference? 1) "-" is $OLDPWD 2) "$_" is last argument of previous command.

5. Shell Programming and Scripting

How to print range of lines using sed when pattern has special character "["

Hi, My input has much more lines, but few of them are below pin(IDF) { direction : input; drc_pinsigtype : signal; pin(SELDIV6) { direction : input; drc_pinsigtype : ...

6. Shell Programming and Scripting

if [ "variable" = "numerical-range" ]; then

been a while so i'm a bit rusty and need a little help. writing a script that needs to compare $EXECHOST(a number) against a numerical range and then set a value. below isn't working but should give you folks an idea of my goal: if ; then echo "This is a 32B machine, exiting..." if ;...

7. Shell Programming and Scripting

Need HELP with AWK split. Need to check for "special characters" in string before splitting the file

Hi Experts. I'm stuck with the below AWK code where i'm trying to move the records containing any special characters in the last field to a bad file. awk -F, '{if ($NF ~ /^|^/) print >"goodfile";else print >"badfile"}' filename sample data 1,abc,def,1234,A * 2,bed,dec,342,* A ...

8. Shell Programming and Scripting

finding the strings beween 2 characters "/" & "/" in .txt file

Hi all. I have a .txt file that I need to sort it My file is like: 1- 88 chain0 MASTER (FF-TE) FFFF 1962510 /TCK T FD2TQHVTT1 /jtagc/jtag_instreg/updateinstr_reg_1 dff1 (TI,SO) 2- ...

9. UNIX for Dummies Questions & Answers

PuTTY displaying "special" characters

I'm not really sure which forum this question should go into, so I'm posting it here. I work with AIX and RHEL systems using PuTTY (Release 0.60_q1.129) from a Windows 7 workstation. Some of the files we get from z/OS use "special" characters as delimiters. These characters include Hex 18...

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to split special characters "|" using awk

Discussion started by: krishna9

2. Shell Programming and Scripting

How to remove "New line characters" and "spaces" at a time

Discussion started by: anushree.a

3. SuSE

VMDB Failure" followed by "Unable to open snapshot file"

Discussion started by: s_linux

4. Shell Programming and Scripting

Question about special variables: "-" and "$_"

Discussion started by: honglus

5. Shell Programming and Scripting

How to print range of lines using sed when pattern has special character "["

Discussion started by: nehashine

6. Shell Programming and Scripting

if [ "variable" = "numerical-range" ]; then

Discussion started by: crimso

7. Shell Programming and Scripting

Need HELP with AWK split. Need to check for "special characters" in string before splitting the file

Discussion started by: shell_boy23

8. Shell Programming and Scripting

finding the strings beween 2 characters "/" & "/" in .txt file

Discussion started by: Behrouzx77

9. UNIX for Dummies Questions & Answers

PuTTY displaying "special" characters

Discussion started by: derndingle