Unexpected results with lists in GNU sed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unexpected results with lists in GNU sed
# 1  
Old 03-15-2012
Unexpected results with lists in GNU sed

I have been living with this problem with GNU sed v4.1.4 for a long time, but now I really need to figure it out.

When using a list in either an address or a search, the expression [a-z] is matching lower and upper-case letters. [A-Z] works as it should.

For example, if I run
Code:
sed -nr "/[a-z]/ p" temp.txt

Where temp.txt contains only
Code:
a
b
c
A
B
C

it matches every line.
If I run it using [A-Z] it matches only the upper-case letters as I would expect. Also, when I run
Code:
grep "/[a-z]/" temp.txt

It matches only the lower-case letters again as I expect. It seems only [a-z] is affected, and only with sed.

Does anyone know why this is? Maybe there is something wrong with the way I have the system set up, but everything else seems to be working ok.

Matt
# 2  
Old 03-15-2012
hmm.. i am using 4.2.1 version and i dont see any issues.

Code:
$ sed -nr '/[a-z]/ p' temp.txt 
a
b
c

$ sed -n '/[a-z]/ p' temp.txt 
a
b
c

# 3  
Old 03-15-2012
1) The first sed expression you gave does not match upper-case characters when I test it.

2) grep does not use / / as regular expression beginning and end markers; your grep expression couldn't possibly have worked.

So, unless your version of sed is extremely strange, I think you've simplified the problem a little too far, reducing it into something which doesn't actually demonstrate your problem. Could you post the actual expression you're having problems with please?
# 4  
Old 03-15-2012
What happens if you use:
Code:
sed -nr "/[[:lower:]]/ p" temp.txt

# 5  
Old 03-15-2012
Quote:
Originally Posted by Corona688
1) 2) grep does not use / / as regular expression beginning and end markers; your grep expression couldn't possibly have worked.

So, unless your version of sed is extremely strange, I think you've simplified the problem a little too far, reducing it into something which doesn't actually demonstrate your problem. Could you post the actual expression you're having problems with please?
Corona, sorry I got careless cutting and pasting, the grep command was
Code:
grep "[a-z]" temp2.txt

I just tried it again and the result was correct.

The sed example in the post is an example of an expression that exhibits the problem on my system (apparently not others'). I verified that also.

Here is something that is easier to cut and paste to the command line.
Code:
echo abAB | sed -r "s/[a-z]/x/g"

The result is xxxx on my machine. Not the expected xxAB.

Perhaps I do have a strange version of sed. One thought is whether it could be that the regex library not correct for the sed version. I don't know if that is the case or how to check.

Scrutinizer, Yes it works with classes, but that doesn't solve the problem.
# 6  
Old 03-15-2012
If it really is a bug in sed, the obvious thing to do would be to upgrade your version of sed...

If your system has busybox on it, it may be a viable alternative.

Code:
$ busybox sed

Usage: sed [-efinr] SED_CMD [FILE]...

Options:
        -e CMD  Add CMD to sed commands to be executed
        -f FILE Add FILE contents to sed commands to be executed
        -i      Edit files in-place
        -n      Suppress automatic printing of pattern space
        -r      Use extended regex syntax

If no -e or -f is given, the first non-option argument is taken as the sed
command to interpret. All remaining arguments are names of input files; if no
input files are specified, then the standard input is read. Source files
will not be modified unless -i option is given.

$

# 7  
Old 03-15-2012
I was thinking this might be a locale issue, therefore I suggested trying [[:lower:]] instead in post #4:

Quote:
[a-z] is case insensitive
You are encountering problems with locales. POSIX mandates that [a-z] uses the current locale's collation order - in C parlance, that means using strcoll(3) instead of strcmp(3). Some locales have a case-insensitive collation order, others don't.
Another problem is that [a-z] tries to use collation symbols. This only happens if you are on the GNU system, using GNU libc's regular expression matcher instead of compiling the one supplied with GNU sed. In a Danish locale, for example, the regular expression ^[a-z]$ matches the string ‘aa', because this is a single collating symbol that comes after ‘a' and before ‘b'; ‘ll' behaves similarly in Spanish locales, or ‘ij' in Dutch locales.

To work around these problems, which may cause bugs in shell scripts, set the LC_COLLATE and LC_CTYPE environment variables to ‘C'.
sed, a stream editor, 7 Reporting Bugs
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Seeing unexpected results when i run through cronjob

Hi I'm having hard time here with below script. If i run script manually i see expected results but, if i keep this script in cron job i'm getting unexpected results. Unexpected results means even though condition is true,cronjob returning output of else condition. This script and cronjob... (2 Replies)
Discussion started by: buzzme
2 Replies

2. Shell Programming and Scripting

Comm giving unexpected results

Hi I am comparing two files with comm -13 < (sort acc11.txt) < (sort acc12.txt) > output.txt purpose: Get non matching records which are in acc12 but not in acc11... TI am getting WRONG output. Is there any constraints with record length with comm? The above files are the two consective ... (2 Replies)
Discussion started by: vedanta
2 Replies

3. Shell Programming and Scripting

sed Unexpected results, missing first search item

I created 3 files with the identical data as follows dial-peer voice 9999 pots trunkgroup CO list outgoing Local translation-profile outgoing LOCAL-7-DIGITS-NO-PREPEND-97 preference 2 shutdown destination-pattern 9......$ forward-digits 7 dial-peer voice 10000 pots ... (6 Replies)
Discussion started by: popeye
6 Replies

4. UNIX for Dummies Questions & Answers

Find command gave unexpected results

Hi, I recently executed a find command that caused unexpected permission changes and we had to do a full system restore. Can someone please explain what this command would do? find /staging/admin/scr * -exec chmod 755 '{}' + It caused file permissions inside / to be modified strangely. ... (1 Reply)
Discussion started by: poornima
1 Replies

5. Shell Programming and Scripting

Unexpected results when triggered from cron

Following script gives different results when triggered from Cron compared to when triggered from command line. It is not able to assign values to some variables when triggered from cron. Can any one help? Its a very simple script Script - #! /bin/ksh sFile=$1 sEnv=$2 sWaitFile=$3... (1 Reply)
Discussion started by: sumeet
1 Replies

6. Shell Programming and Scripting

Unexpected Results (at least I did not expect them)

I have two sripts running in bash. The first one uncompresses log files and moves them to a working directory using uncompress -c and > output to new directory. It then creates one control record to assure our search returns a record. It then calls or executes the second script, which is a grep for... (6 Replies)
Discussion started by: altamaha
6 Replies

7. UNIX for Dummies Questions & Answers

Unexpected Results

Hello, When I run this script, here's what I get: Searching ... found 1111 2222 3333 ..... 7777 ..... 8888 9999 in 95_test Search completed. I expected only to see what number was found in the file, not including the ones not found. Thanks for your help! #!/bin/sh (1 Reply)
Discussion started by: SSims
1 Replies

8. Shell Programming and Scripting

Perl - Iterating a hash through a foreach loop - unexpected results

i've reworked some code from an earlier post, and it isn't working as expected i've simplified it to try and find the problem. i spent hours trying to figure out what is wrong, eventually thinking there was a bug in perl or a problem with my computer. but, i've tried it on 3 machines with the... (5 Replies)
Discussion started by: quantumechanix
5 Replies
Login or Register to Ask a Question