Processing extended ascii character file names in UNIX (BASH scipts)


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Processing extended ascii character file names in UNIX (BASH scipts)
# 1  
Old 02-14-2008
Processing extended ascii character file names in UNIX (BASH scipts)

Hi, I have a accentuated letter (ö) in a script for an Installer. It's a file name. This is not working and I'm told to try using the octal value for the extended ascii character. Does anyone no how to do this? If I had the word "filförval", can I just put in the value between the letters, like this; "filf148rval", or is it more to it?

Thank's

peli
# 2  
Old 02-15-2008
You have to insert a backslash followed by zero before the octal code number (\0n):

Code:
"filf\0148rval"

You can try an echo command to see if the output is really what you want:

Code:
echo -e "filf\0148rval"

You need to be sure of the octal code you use. For that character, isn't "\0366" the correct code?

Try and see Smilie

Last edited by robotronic; 02-15-2008 at 08:26 AM..
# 3  
Old 02-17-2008
Thank's. I tried the echo command in the terminal and it works with the regular carachter's but not with the extended.
The script doesn't work either. I did the following to test the script. First an explaining what the script is doing.
It put files in Library—>Application Support and then moves them to the Application folder if there's no such files. It don't overwrite. This is also working with files without accentuated letters.
So, I did used the letter lowercase "o" instead wich have the octal code "0157" like this:
#!/bin/bash
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Jazz filf\0157rval" "/Applications/Finale 2008/Komponentfiler".


The script didn't respond to this, so it seems to me that octal ascii code doesn't work in this kind of script?
peli
# 4  
Old 02-17-2008
If you can't echo the extended characters maybe you should check the character encoding you're using in your environment (like utf8, iso8859-1, cp1252 ...).

Codes greater than 127 displays different characters depending on the locale settings. On my system if I set encoding to iso8859-1 (Western European) or cp1252 (WinLatin1) the code "\0366" corresponds to the special "o" character you've mentioned (ö), while with utf8 isn't recognized. With utf8 that character is multibyte (2 bytes) and for echoing that I do:

Code:
echo -e "\0303\0266"

About the cp command, I confirm that you cannot use that syntax: you may set a variable containing the complete pathname and pass that to cp, e.g.:

Code:
x=$(echo -e "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Jazz filf\0366rval")

cp -R -n -p "$x" "/Applications/Finale 2008/Komponentfiler"

If you have a text file containing that special character and you want to be absolutely sure of the octal code you have to use, try this procedure: make a copy of the file, edit it and leave in the file only that single character, then save and execute:

Code:
od -t oC -An input_file

For each character it prints out the octal code. In my case (I use echo instead of the input file):

With iso8859-1 encoding:
Code:
test ~ $ echo -e "ö" | od -t oC -An
 366 012

With utf8 encoding:
Code:
test ~ $ echo -e "ö" | od -t oC -An
 303 266 012

Notice that the output includes a trailing newline (\0012).
# 5  
Old 02-19-2008
I tried a couple of different text codings that worked but I don't have iso8859-1. Isn't that the same as "western latin 1"? My system was set to UTF-8 and it worked with that echo -e "\0303\0266 command.
When I tried the "test ~ $ echo -e "ö" | od -t oC -An" I'll get this error "-bash: test: too many arguments".

About the cp command . I don't get it to work. I'm not sure how to use the code. Is this two lines:
x=$ (echo -e "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Jazz filf\0157rval")
cp -R -n -p "$x" "/Applications/Finale 2008/Komponentfiler"

replacing this line:
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Jazz filf\0366rval" "/Applications/Finale 2008/Komponentfiler"?

I'll attach the complete script for your imformation:
#!/bin/bash

cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/ensembles.txt" "/Applications/Finale 2008/Komponentfiler"
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/FinaleScript.dat" "/Applications/Finale 2008/Komponentfiler"
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/instrument.txt" "/Applications/Finale 2008/Komponentfiler"
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/MacSymbolFonts.txt" "/Applications/Finale 2008/Komponentfiler"
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/pagesizes.txt" "/Applications/Finale 2008/Komponentfiler"
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Maestro filförval" "/Applications/Finale 2008/Komponentfiler"
cp -R -n -p "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Jazz filförval" "/Applications/Finale 2008/Komponentfiler"
rm -rf "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/"


Thank's

Last edited by peli; 02-20-2008 at 02:50 AM..
# 6  
Old 02-20-2008
Yes, you have to replace that single line with two commands. With utf8 for you the command should be:

Code:
x=$(echo -e "/Library/Application Support/MakeMusic/Finale 2008/Komponenter/Jazz filf\0303\0266rval")
cp -R -n -p "$x" "/Applications/Finale 2008/Komponentfiler"

By the way, if I try copying files with "strange characters" on my box I don't experiment your issue: I can simply use "strange characters" in the cp command and it works perfectly (I'm using an utf8 locale too). Are you using an utf8-aware terminal (like KDE's Konsole or similar)? Maybe you are using an utf8 locale but your terminal doesn't support it or it is not configured correctly.
# 7  
Old 02-20-2008
Yes, now it's working.
Thank you robotronic!
peli
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Print byte position of extended ascii character

Hello, I am on AIX. When I encounter extended ascii characters and special characters on a file I need to print.. Byte position, actual character and line number. Is there a simple command that can give me the above result ? Thanks in advance (38 Replies)
Discussion started by: rosebud123
38 Replies

2. UNIX for Beginners Questions & Answers

Convert ascii character values to number that comes between the numbers in UNIX

I have variable that contains multiple values of number and also include overpunch(i.e. # $ % etc) character so we want to replace it with numbers. here are the example: Code: 11500#.0# 28575$.5$ 527#.7# 42".2" 2794 .4 2279!.9! 1067&.7& 926#.6# 2279!.9! 885".5" 11714$.4$ 27361'.1'... (1 Reply)
Discussion started by: nadeemrafikhan
1 Replies

3. Shell Programming and Scripting

Removal Extended ASCII using awk

Hi All, I am trying to remove (SELECTIVE - passed as argument) Extended ASCII using Awk based on adhoc basis. Can you please let me know how to do it. I have to implement this using awk only. Thanks & Regads (14 Replies)
Discussion started by: tostay2003
14 Replies

4. Shell Programming and Scripting

Identify extended ascii characters in a file

Hi, Is there a way to identify the lines in a file having extended ascii characters and display the same? For instance I have a file abc.txt having below data aaa|bbb|111|This is first line aaa|bbb|222|This is secőnd line aaa|bbb|333|This is third line aaa|bbb|444|This is foůrth line... (3 Replies)
Discussion started by: decci_7
3 Replies

5. Shell Programming and Scripting

Preserve extented ascii character when run echo comand inside bash script

Hi everyone, I'm echo some text with extended ascii characters as below: echo -e "Pr\xE9sentation du spectacle" > output or echo -e "Présentation du spectacle" > outputIf I open the file created I see this text Présentation du spectacleThe text is shown correctly in this created file when... (7 Replies)
Discussion started by: Ophiuchus
7 Replies

6. AIX

Printing extended ASCII

Hi All, I'm trying to send extended ascii characters to my HP2055 as part of PCL printer control codes. What I want to do is select a bar code font, print the bar code and reset the printer to the default font. Selecting the bar code font works good. Printing the bar code goes almost ok too. ... (5 Replies)
Discussion started by: petervg
5 Replies

7. Shell Programming and Scripting

read in a file character by character - replace any unknown ASCII characters with spa

Can someone help me to write a script / command to read in a file, character by character, replace any unknown ASCII characters with space. then write out the file to a new filename/ Thanks! (1 Reply)
Discussion started by: raghav525
1 Replies

8. Shell Programming and Scripting

extended ascii problem

hi i would like to check text files if they contain extended ascii characters within or not. i really dont have any idea how to start your kind help would be very much appreciated thanks. (7 Replies)
Discussion started by: smooth
7 Replies

9. Shell Programming and Scripting

Weird Ascii characters in file names

Hi. I have files in my OS that has weird file names with not-conventional ascii characters. I would like to run them but I can't refer them. I know the ascii # of the problematic characters. I can't change their name since it belongs to a 3rd party program... but I want to run it. is there... (2 Replies)
Discussion started by: yamsin789
2 Replies

10. Programming

Extended ascii

Hi all, I would like to change the extended ascii code ( 128 - 255). I tried to change LC_ALL and LANG in current session ( values from locale -a) and for no good. Thanks. (0 Replies)
Discussion started by: avis
0 Replies
Login or Register to Ask a Question