Extended ASCII Characters keep on getting reintroduced to text files
I am working with a log file that I am trying to clean up by removing non-English ASCII characters. I am using Bash via Cygwin on Windows.
Before I start I set:
I clean it up by removing all non-English ASCII characters with the following command;
I then check whether there is any non-English ASCII characters left with the following command and it returns nothing, indicating that there is no non-English ASCII characters left.
I then deleted the first line with the following command;
When I check filename_03.csv again for non-English characters it returns quite a few lines with non-English ASCII characters Why is this happening, what am I doing wrong? It somehow got reintroduced when I ran the tail command, how is this possible?
Example of the characters that got introduced back into my text file after I ran the tail command that I initially cleaned.
I think you'll get better response if you post representative samples of your log file indicating which characters you want to remove, or a sample of the result file.
One thing I noticed is that this:
should be:
Also, this :
does not just remove non-ascii characters, it discards entire lines that contains one of those characters that are not [\t\r -~]
Last edited by Scrutinizer; 07-10-2016 at 10:28 AM..
Using the very limited info, done longhand.
Each line in the file contains a single ' and " .
Not sure if this a just a very small part of the string but here goes.
Results:-
Hello,
I am on AIX.
When I encounter extended ascii characters and special characters on a file I need to print..
Byte position, actual character and line number.
Is there a simple command that can give me the above result ?
Thanks in advance (38 Replies)
Hi All,
I am trying to remove (SELECTIVE - passed as argument) Extended ASCII using Awk based on adhoc basis. Can you please let me know how to do it. I have to implement this using awk only.
Thanks & Regads (14 Replies)
Hi,
I want to read extended ASCII characters from keyboard using c language on unix/linux. How to read extended characters from keyboard or by copy-paste in terminal irrespective of locale set in the system. I want to read the input characters from keyboard, store it in an array or some local... (3 Replies)
We are getting extended Ascii characters in the input file and my requirement is to search and replace them with a space. I am using the following command
LANG=C sed -e 's// /g'
It is doing a good job, but in some cases it is replacing the extended characters with two spaces. So my input... (12 Replies)
Hi,
Is there a way to identify the lines in a file having extended ascii characters and display the same?
For instance I have a file abc.txt having below data
aaa|bbb|111|This is first line
aaa|bbb|222|This is secõnd line
aaa|bbb|333|This is third line
aaa|bbb|444|This is foùrth line... (3 Replies)
Well, to make another post at this helpful forum :b::D:
I recently tried something like this, I want to replace all those numberings/letters that are located
between <string>file://localhost/var/mobile/Applications/ and /Documents/</string>
numberings =----
replace with:
first... (6 Replies)
Hi All,
I'm trying to send extended ascii characters to my HP2055 as part of PCL printer control codes. What I want to do is select a bar code font, print the bar code and reset the printer to the default font.
Selecting the bar code font works good. Printing the bar code goes almost ok too. ... (5 Replies)
Hi gurus,
I have a file in unix with ascii values. I need to convert all the ascii values in the file to ascii characters. File contains nearly 20000 records with ascii values. (10 Replies)
hi i would like to check text files if they contain extended ascii characters within or not. i really dont have any idea how to start your kind help would be very much appreciated thanks. (7 Replies)
Hi all,
I would like to change the extended ascii code ( 128 - 255).
I tried to change LC_ALL and LANG in current session ( values from locale -a) and for no good.
Thanks. (0 Replies)