Abnormality while piping tr command output to sed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Abnormality while piping tr command output to sed
# 8  
Old 12-19-2012
Quote:
Originally Posted by chidori
ok , so sed/awk have not worked in here as
tr output : alpha,beta,gamma, did not contain a new line in them. And this behaviour is also sed/awk version specific.
now with awk we can change the record seperator right ? so should not something like this work logically ?
since my tr output does not have a newline i am using RS=""
Code:
# tr -s '\n' ',' < myfile | awk 'BEGIN{RS=""}{sub(/,$/,"");print}'

You can use RS to change the record separator, but that does not alter the fact that the behavior of sed and awk is only defined when the input file is a text file. If there are any characters at the end of a file without a trailing <newline> character, the input is not a text file.

If the input is not a text file, the behavior is unspecified.

Will the unspecified behavior change if the value of RS changes; the answer to that is unspecified. It might work; it might not; it might vary depending on which awk or sed you use.
# 9  
Old 12-19-2012
@Don,

I agree with what you say and I know for a fact that some sed's do process that last line, other's don't so as you say the results are unspecified. However, I have never come across a version of awk that does not process that last unterminated line, and I am wondering why that might be. Even oawk on Solaris will process it:
Code:
$ printf "hello" | awk '{print}'
hello

Code:
$ printf "hello,hello" | awk '{print}' RS=,
hello
hello

Could it perhaps be that awk is almost 'obliged' to process that last line anyway because of the concept of RS, the input record separator which can be set to almost any character. So what might be a consequence of this is that any characters that follow that last RS, should be interpreted as being part of the last record? Otherwise it should have been called RT (Record Terminator)?

Another example would be that if the were not the case then from a UNIX point of view what this printf produces is a valid UNIX text file, but from an awk point of view - if you compare that to the situation where RS is a regular newline - this would be an "unterminated" last record
Code:
$ printf "hello,hello\n" | awk '{print}' RS=,
hello
hello

$

After all that last newline is not a file terminator, but a line terminator..

Last edited by Scrutinizer; 12-19-2012 at 09:55 PM..
# 10  
Old 12-19-2012
Quote:
Originally Posted by Scrutinizer
@Don,

I agree with what you say and I know for a fact that some sed's do process that last line, other's don't so as you say the results are unspecified. However, I have never come across a version of awk that does not process that last unterminated line, and I am wondering why that might be. Even oawk on Solaris will process it:
Code:
$ printf "hello" | awk '{print}'
hello

Code:
$ printf "hello,hello" | awk '{print}' RS=,
hello
hello

Could it perhaps be that awk is almost 'obliged' to process that last line anyway because of the concept of RS, the input record separator which can be set to almost any character. So what might be a consequence of this is that any characters that follow that last RS, should be interpreted as being part of the last record? Otherwise it should have been called RT (Record Terminator)?

Another example would be that if the were not the case then from a UNIX point of view what this printf produces is a valid UNIX text file, but from an awk point of view - if you compare that to the situation where RS is a regular newline - this would be an "unterminated" last record
Code:
$ printf "hello,hello\n" | awk '{print}' RS=,
hello
hello

$

After all that last newline is not a file terminator, but a line terminator..
Yes, the last newline is a line terminator. So the input file consists of lines. And, each line is no more than LINE_MAX bytes (including the terminating newline) and there are no null bytes in any line. Therefore, by definition, the input is a text file.

The behavior of awk and sed is defined when the input is a text file no matter what the record separator is.

If the input file had been created by printf "hello,hello" (note that there is no trailing newline), the results would be unspecified no matter how RS is set because the input fiile would not be a text file.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Facing abnormality in Find command

Whenever the below command is being executed by a scheduler at UNIX environment, we are getting below error cd /tmp/log find . -ignore_readdir_race ! -name . -prune -iname 'XYZ*' -type f -mtime +4 -printf "%f\n" ./tmpfile.script_name.2.31885.201906071336.tmp': No such file or directory... (2 Replies)
Discussion started by: Anirban2208
2 Replies

2. Shell Programming and Scripting

Piping output of ls to a text file

Basically I was wondering if any of you know how to pipe the output of ls to a text file? so in my shell script one of the lines is ls but i want to pipe it into a file called directory listing. Cheers. I have tried ls | Directorylisting.txt but it keeps saying " line 7: DirectoryListing.txt:... (9 Replies)
Discussion started by: LinuxNubBrah
9 Replies

3. Shell Programming and Scripting

need help piping the output from an app... uh, yeah...

Ok, so there is a perl script that runs as a server, on my local host. It tells me which port to use. I want to pipe that output into my browser so I can do the whole thing with a single command. The problem is, I think, that the program doesn't actually exit cause it's running a server, so...... (6 Replies)
Discussion started by: ninjaaron
6 Replies

4. Shell Programming and Scripting

Piping output from a command into bash script

Hi all. I am using procmail to deliver an email to a script I am developing. Procmail delivers the email to the script on standard input. I imagine this is the same as piping input from a command into the script. Hence I've been testing my script by running echo 'test' | sms-autosend-backup.sh ... (2 Replies)
Discussion started by: akindo
2 Replies

5. Shell Programming and Scripting

Piping and assigning output to a variable in Perl

Hi All, I am trying to convert the below Csh code into Perl. But i have the following error. Can any expert help ? Error: ls: *tac: No such file or directory Csh set $ST_file = `ls -rt *$testid*st*|tail -1`; Perl my $ST_file = `ls -rt *$testid*st*|tail -1`; (10 Replies)
Discussion started by: Raynon
10 Replies

6. Shell Programming and Scripting

piping oracle output to a file?

Hi All... Does anyone know how to pipe the output of a "select" statement from a call to Oracle to a file? ANy ideas woule be greatly appreciated! Code is as below... echo "producing CSV file 2..." sqlplus -s $username/$password@$database<<EOF set serveroutput on size 1000000 set... (13 Replies)
Discussion started by: satnamx
13 Replies

7. Shell Programming and Scripting

piping output of tail running in background

Not sure why this does not work in bash: tail -f err.log |& -bash: syntax error near unexpected token `&' I am attempting to continuously read a file that is being updated by doing a "tail -f" on the file and piping the output to stdin which can then be read by the next shell command Thnx (4 Replies)
Discussion started by: anuramdas
4 Replies

8. Shell Programming and Scripting

piping output to echo

Hi, I was wondering why ls * | echo does not print the contents of the directory to the screen? The way I see it, ls * returns a whole lot of information, and then we pipe all this info to echo, so surely it should all come to our screen! Is there a serious flaw in my understanding? ... (3 Replies)
Discussion started by: A1977
3 Replies

9. UNIX for Dummies Questions & Answers

piping the output of find command to grep

Hi, I did not understand why the following did not work out as I expected: find . -name "pqp.txt" | grep -v "Permission" I thought I would be able to catch whichever paths containing my pqp.txt file without receiving the display of messages such as "find: cannot access... Permisson... (1 Reply)
Discussion started by: 435 Gavea
1 Replies

10. Shell Programming and Scripting

Piping output to while read

Hi. Im using cat to output the contents of a file, then piping it to my while read loop.In this loop variables get assigned values. However when i try to use the variables outside the loop their values has been reset.I understand about subshells etc. but I have no idea how to "preserve" the... (3 Replies)
Discussion started by: Ultimodiablo
3 Replies
Login or Register to Ask a Question