Sponsored Content
Top Forums Shell Programming and Scripting Identify extended ascii characters in a file Post 302909429 by decci_7 on Thursday 17th of July 2014 01:32:33 AM
Old 07-17-2014
Thanks for the solution. It works perfectly fine. There is one more issue though. I'm trying to identify the number of bytes, lines and max line length using below command.
Code:
wc -l -c -L

But it ends up in error whenever it comes across any line having extended ascii characters in it and ends up with below error.
Code:
Invalid or imcomplete multibyte or wide character

Any solution to ensure it doesn't fail? I dont want to remove the extd. ascii characters from the line. One way I could think is to create a temp file, replace the extd. ascii characters with a normal one (e.g. "a") and then run the command but then again not familar with how to replace the same Smilie

---------- Post updated 07-17-14 at 12:32 AM ---------- Previous update was 07-16-14 at 11:27 PM ----------

Looking around the net I found couple of approaches which I can use on the same lines perl solution was suggested.

To find out the lines having extended ascii characters
Code:
grep -P "[\x80-\xFF]" abc.txt > abc1.txt

To replace the extended ascii character with a single byte character(a).
Code:
LANG=C sed 's/[\x80-\xFF]/a/g' abc.txt > abc2.txt

and once its done the wc can be used on the resultant file.

This approach seems to be working fine for me. Please suggest if there is a better approach that can be used.
 

10 More Discussions You Might Find Interesting

1. Programming

Extended ascii

Hi all, I would like to change the extended ascii code ( 128 - 255). I tried to change LC_ALL and LANG in current session ( values from locale -a) and for no good. Thanks. (0 Replies)
Discussion started by: avis
0 Replies

2. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud (2 Replies)
Discussion started by: budrito
2 Replies

3. Shell Programming and Scripting

extended ascii problem

hi i would like to check text files if they contain extended ascii characters within or not. i really dont have any idea how to start your kind help would be very much appreciated thanks. (7 Replies)
Discussion started by: smooth
7 Replies

4. UNIX for Advanced & Expert Users

Processing extended ascii character file names in UNIX (BASH scipts)

Hi, I have a accentuated letter (ö) in a script for an Installer. It's a file name. This is not working and I'm told to try using the octal value for the extended ascii character. Does anyone no how to do this? If I had the word "filförval", can I just put in the value between the letters, like... (9 Replies)
Discussion started by: peli
9 Replies

5. AIX

Printing extended ASCII

Hi All, I'm trying to send extended ascii characters to my HP2055 as part of PCL printer control codes. What I want to do is select a bar code font, print the bar code and reset the printer to the default font. Selecting the bar code font works good. Printing the bar code goes almost ok too. ... (5 Replies)
Discussion started by: petervg
5 Replies

6. Shell Programming and Scripting

Search and Replace Extended Ascii Characters

We are getting extended Ascii characters in the input file and my requirement is to search and replace them with a space. I am using the following command LANG=C sed -e 's// /g' It is doing a good job, but in some cases it is replacing the extended characters with two spaces. So my input... (12 Replies)
Discussion started by: ysvsr1
12 Replies

7. Programming

How to read extended ASCII characters from stdin?

Hi, I want to read extended ASCII characters from keyboard using c language on unix/linux. How to read extended characters from keyboard or by copy-paste in terminal irrespective of locale set in the system. I want to read the input characters from keyboard, store it in an array or some local... (3 Replies)
Discussion started by: sanzee007
3 Replies

8. Shell Programming and Scripting

Removal Extended ASCII using awk

Hi All, I am trying to remove (SELECTIVE - passed as argument) Extended ASCII using Awk based on adhoc basis. Can you please let me know how to do it. I have to implement this using awk only. Thanks & Regads (14 Replies)
Discussion started by: tostay2003
14 Replies

9. Shell Programming and Scripting

Extended ASCII Characters keep on getting reintroduced to text files

I am working with a log file that I am trying to clean up by removing non-English ASCII characters. I am using Bash via Cygwin on Windows. Before I start I set: export LC_ALL=C I clean it up by removing all non-English ASCII characters with the following command; grep -v $''... (4 Replies)
Discussion started by: lewk
4 Replies

10. UNIX for Beginners Questions & Answers

Print byte position of extended ascii character

Hello, I am on AIX. When I encounter extended ascii characters and special characters on a file I need to print.. Byte position, actual character and line number. Is there a simple command that can give me the above result ? Thanks in advance (38 Replies)
Discussion started by: rosebud123
38 Replies
isencrypt(3GEN) 				     String Pattern-Matching Library Functions					   isencrypt(3GEN)

NAME
isencrypt - determine whether a buffer of characters is encrypted SYNOPSIS
cc [flag]... [file]... -lgen [library]... #include<libgen.h> int isencrypt(const char *fbuf, size_t ninbuf); DESCRIPTION
isencrypt() uses heuristics to determine whether a buffer of characters is encrypted. It requires two arguments: a pointer to an array of characters and the number of characters in the buffer. isencrypt() assumes that the file is not encrypted if all the characters in the first block are ASCII characters. If there are non-ASCII characters in the first ninbuf characters, and if the setlocale() LC_CTYPE category is set to C or ascii, isencrypt() assumes that the buf- fer is encrypted If the LC_CTYPE category is set to a value other than C or ascii, then isencrypt() uses a combination of heuristics to determine if the buffer is encrypted. If ninbuf has at least 64 characters, a chi-square test is used to determine if the bytes in the buffer have a uniform distribution; if it does, then isencrypt() assumes the buffer is encrypted. If the buffer has less than 64 characters, a check is made for null characters and a terminating new-line to determine whether the buffer is encrypted. RETURN VALUES
If the buffer is encrypted, 1 is returned; otherwise, zero is returned. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ SEE ALSO
setlocale(3C), attributes(5) NOTES
When compiling multithreaded applications, the _REENTRANT flag must be defined on the compile line. This flag should only be used in mul- tithreaded applications. SunOS 5.11 29 Dec 1996 isencrypt(3GEN)
All times are GMT -4. The time now is 06:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy