Piping fails in locale other than English


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Piping fails in locale other than English
# 8  
Old 07-12-2012
The output you're seeing is what one would see if the pipe symbol were to be quoted.
Code:
grep -n '^PAX_PAYLOAD:$' $0 '|' cut -d ':' -f 1

Seeing the entire script and knowing exactly how it's invoked (especially the value of $0) would be helpful. Also, further details of the environment may help. Which sh exactly is being used, for starters.

A shot in the dark (although I don't see how it could be the cause, it's good practice): try double-quoting $0.

Regards,
Alister

---------- Post updated at 12:54 PM ---------- Previous update was at 12:40 PM ----------

Some light may be shed on the problem if you enable tracing at the top of the script (or at least before the problematic section is entered). set -x enables tracing, set +x disables it.

Regards,
Alister
This User Gave Thanks to alister For This Post:
# 9  
Old 07-12-2012
Thanks alister. Turning tracing on was a great idea.

I was able to get around my previous issue by finding an alternative to piping. However, I have another spot where I am also using a pipeline and I am seeing the same issue.

Code:
tail -n +$license_start $0 | head -n $length | uudecode -o /dev/stdout

(I am using a combination of head and tail to get encoded data between two line numbers and then piping that into uudecode. There is probably a better way to do this but for now id like to solve this piping issue.)

Alister, I took your advice and turned tracing on.
In en_US or C for the locale I get this:

Code:
+ tail -n +46382 ./install.sh
+ head -n 2217
+ uudecode -o /dev/stdout

(and then the decoded file)

In other locales like cs_CZ i get this:

Code:
+ tail -n +46382 ./install.sh | head -n 2217 | uudecode -o /dev/stdout

as if it is all one command...


I am very new to shells, encoding etc. Sorry, i do not know exactly which sh I am using.

However, I noticed that when I change the locale from cs_CZ to cs_CZ.UTF-8 piping works. Is this just an encoding problem?

Last edited by Scrutinizer; 07-12-2012 at 05:39 PM.. Reason: code tags
# 10  
Old 07-12-2012
This is probably just an encoding problem. Shells interpreting pipes as literal characters is extremely odd.
# 11  
Old 07-12-2012
If this is just an encoding problem, is there a way to fix it? Is it a matter of my script not having the right encoding? I tried using file -i to determine its encoding but all I get is "regular file."

I suppose I can get around piping by storing the output of each command in a temp file, but is this good practice?
# 12  
Old 07-12-2012
Not a good practice, no. Shells expect ASCII or something compatible with ASCII. Your encoding must be something strange which disagrees with ASCII on what | means.

Try writing up the file from scratch after you've set your encoding to "C" or UTF8 or some other ASCII-compatible encoding.
# 13  
Old 07-12-2012
Quote:
Originally Posted by adam.wis
However, I noticed that when I change the locale from cs_CZ to cs_CZ.UTF-8 piping works. Is this just an encoding problem?
Honestly, I don't know. I have never experienced this problem. Please make sure to report back if you determine the exact cause.

Regards,
Alister

Last edited by alister; 07-12-2012 at 08:14 PM..
# 14  
Old 07-12-2012
The z/OS shell expects an EBCDIC environment. The pipe is a variant character (one of 13 widely-used characters that can vary between EBCDIC flavours) which is represented by different bytes values depending on the locale which is set. This is why a pipe can work in one locale and not in another.

Internally, the z/OS shell expects the script to be encoded in IBM-1047. If the active character set is not IBM-1047, then the script is transcoded to IBM-1047 before execution. The transcode operation modifies a specific set of bytes in the script, i.e. only variant characters are changed to their IBM-1047 equivalents.

To display the variant byte values for the current locale
Code:
locale -ck LC_SYNTAX.


Last edited by fpmurphy; 07-12-2012 at 11:53 PM..
These 4 Users Gave Thanks to fpmurphy For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to convert number to english?

Hi gurus, I have a weird requirement. I need to convert the number to english lecture. I have 1.2 ....19 numbers I need to convert to first second third fourth, fifth, sixth... Is there any way convert it using unix command? thanks in advance. (8 Replies)
Discussion started by: ken6503
8 Replies

2. Shell Programming and Scripting

Translate from english to french

Hi, I wrote a script to convert a given word from English to French. But I am not able to figure out what I am missing here. I am not able to get the translated word Below is my script: French=/root/dict/entofr.txt for i in $* do word="echo $word $i" done while: do cat <<... (1 Reply)
Discussion started by: pinky7630
1 Replies

3. Shell Programming and Scripting

convert english to chinese

Hi Experts, Can anyone help me to convert a english input into chinese in a bash script. help would be highly appreciable. thanks, Deepak (3 Replies)
Discussion started by: naw_deepak
3 Replies

4. UNIX for Dummies Questions & Answers

translate to normal english

lnode * head = temp; (1 Reply)
Discussion started by: rickym2626
1 Replies

5. Red Hat

Displaying the available locales in english

Hi, I am developing a program that would ask the user to set the locale. For that, I need to display them to user in plain english. like English(US) English (Uk) depending on the user selection I need to set the locale. Is there a command in redhat linux that would... (1 Reply)
Discussion started by: eamani_sun
1 Replies

6. Ubuntu

LANG=C not English?

On Ubuntu 7.04, why would the "C" LANG parameter not be English: $ LANG=C locale LANG=C LANGUAGE=he_IL:he:en_GB:en LC_CTYPE="he_IL.utf8" LC_NUMERIC="he_IL.utf8" LC_TIME="he_IL.utf8" LC_COLLATE="he_IL.utf8" LC_MONETARY="he_IL.utf8" LC_MESSAGES="he_IL.utf8" LC_PAPER="he_IL.utf8"... (4 Replies)
Discussion started by: dotancohen
4 Replies

7. SCO

gzip english version

Hello, i'm not skilled on unix, i'd like gzip/gunzip software and, ESPECIALLY, the detailed instructions for installation....please help me......i'm like a baby in unix world!!!!! hello, thanks a lot! mike (3 Replies)
Discussion started by: mfran2002
3 Replies

8. Shell Programming and Scripting

Please decode in English

Hello: Can anyone please decode this script in English. I have also made some comments which I know.. The actual script does not have one comment also.. #! /bin/ksh . odbmsprd_env.ksh #setting the env.. echo $0 Started at : `date '+%d-%m-%Y %H:%M:%S'` # what's echo $0 ... (4 Replies)
Discussion started by: ST2000
4 Replies
Login or Register to Ask a Question