Ignoring the 1st two bytes in your file, it looks like the UTF-16 encoding for "Eve. You haven't answered the question about what locale you're using.
What is the output from the commands:
If, instead of using:
you use:
does the output look like the characters you're expecting with NUL bytes (displayed as \0) between them (which is exactly what we'd expect if your locale is based on a superset of ASCII code set and the data you're viewing is encoded using UTF-16).
What application is creating this file on Windows?
Last edited by Don Cragun; 09-07-2014 at 02:35 PM..
Reason: fix typo.
This is a little tongue-in-cheek and relies entirely in this particular case on a common junk character.
OSX 10.7.5, default bash terminal.
I have no idea if it will work on extremely huge files as it is read into variable and not streamed, however......
Using 'IFS' to your advantage and made as fully readable as possible...
Results:-
I know it is a little tongue-in-cheek, but a couple of remarks:
---
The redirection is unnecessary:
---
To use the same name for a variable and an array in the same assignment is a bit *erm* :
Why not do it all in one go:
or since you are using bash:
One thing to be aware off is that not just one, but any of number of trailing newlines will be removed.
---
The quotes have no function here..
--
Since you are using bash, you may also use:
--
Do not leave out the format field. Leaving it out may bring all kinds of surprises. use:
--
Since you are using arrays instead of a loop you can also use:
Last edited by Scrutinizer; 09-07-2014 at 07:11 PM..
This User Gave Thanks to Scrutinizer For This Post:
The file Which iam downloading from windows box is Scheduler log file.
No. I asked you to run two commands; one to determine what OS you're using (which we now know is AIX) and one to determine what locale you're using (which you have not yet shown us). But it is now obvious (at least to me RudiC and me) that (despite your declaration that the file you have downloaded from Windows is not encoded using UTF-16); it is indeed encoded in UTF-16. If you will tell us what locale you're using, we can then tell you what option-argument to give to the iconv -t option to give you a file you can process in your locale.
I would expect that you will either want:
or:
where filename is the name of the file you downloaded from Windows.
Please don't post the results (since it obviously contains sensitive information), but please look at the output from the command:
as I suggested before. Doesn't the output from od contain the output you're looking for with a null bytes between the characters you want?
Hello All,
I have issues in unix file when I loaded that to database and do select * from table where description like '%'+char(13)+'%' on it I am able to get records. I tried to view the file in unix it is all having blank character which I think is all non ascii which I am not able view.... (11 Replies)
Hi
I want to know how to see junk character in a file.
i am not able to see junk character using vi or cat command.
below is the junk char . which i see in host file
10.178.14.67▒▒▒ ac01sp02-vip
actually it should be like this
10.178.14.67 ac01sp02-vip
i am using secure CRT... (11 Replies)
RedHat Linux 5.8/Korn Shell
I have text file name /etc/oracle/config.loc. It has the following text
#Device/file getting replaced by device +OCR
ocrconfig_loc=+DATA
ocrmirrorconfig_loc=+OCRBut , when I open this file using cat , the PS1 character (for prompt) appears as the last character... (8 Replies)
Dear ALL,
How to remove junk charecter ^M from unix file i am using sun solaris unix.
I already tried few commands
:%s/^M//g
:%s/r//g
but it didnt helped me.
Any help appriciated.
Thanks
Ripudaman
Please view this code tag video for how to use code tags when posting code... (5 Replies)
Hello,
I have two .sql files which I transferred from Windows to Unix (Linux Enterprise Linux Server release 5.3).I want to ensure that these two files have no junk characters in them.How do I do it in the simplest possible way?
Many thanks
DJ (1 Reply)
Hello Is there a way to calculate how many times a particular symbol appeared in a string before a particular word.
Desktop/Myfiles/pet/dog/puppy
So, I want to count number of occurence of"/" in this directory before the word dog lets say.
Cheers,
Bob (3 Replies)
Hi,
Is it possible to download a file using Wget or some other command from a Windows machine?
Say I want to download something from
https server
to C:\ABC\abc.xls
Any ideas,
Thanks. (4 Replies)
Hi Team,
I have a file having size greater than 1 GB. What i want to do is to check if it contains any JUNK character (ie any special charater thats not on the key board stroke). This file has 532 column & seperated with ^~^.
I have found some solution from the file, but it is for a... (4 Replies)
Hi
set filename "./GopiRun.sh"
if } err] {
writeLog "error in exec "
writeLog $a
} else {
writeLog $a
}
The above code will execute a file GopiRun.sh,and will log the output of the exec to a file.
The problem is the file has lot of junk character in it,how to avoid it.
The... (2 Replies)
I have a file with one of the following lines, when opened with vi
33560010686GPT£120600GBPGBP10082007DS
In the above line, I want to get rid of the junk character before the £ (pound sysmbol).
When I tried copying £ from windows and copy in unix vi, it prints as £ and I tried pattern replace... (2 Replies)