Suppress new line at end here string?

Login or Register to Ask a Question and Join Our Community

Suppress new line at end here string?

Tags

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting Suppress new line at end here string?

08-12-2014

Registered User

183, 15

Join Date: Jul 2010

Last Activity: 22 June 2015, 3:25 PM EDT

Posts: 183

Thanks Given: 56

Thanked 15 Times in 13 Posts

Suppress new line at end here string?

I'm trying to fully escape/quote a directory path for use in a regular expression with no characters interpreted symbolically. Printable characters get \* and white space gets "*"

Code:

$ inputDirectory="/home/dir1/dir/"

$ echo $( { while read -r -n1 a; do [[ "$a" =~ [[:blank:]] ]] &&  echo -n "\"""$a""\"" || echo -n "\\""$a"; done; } <<<"$inputDirectory")
\/\h\o\m\e\/\d\i\r\\1\/\d\i\r\\2\/\

$ echo $( { while IFS= read -r -n1 a; do [[ "$a" =~ [[:blank:]] ]] &&  echo -n "\"""$a""\"" || echo -n "\\""$a"; done; } <<<"$inputDirectory")
\/\h\o\m\e\/\d\i\r" "\1\/\d\i\r" "\2\/\

$ echo $( { while IFS= read -r -n1 a; do [[ "$a" =~ [[:blank:]] ]] &&  echo -n "\"""$a""\"" || echo -n "\\""$a"; done; } <<<$(echo -n "$inputDirectory") )
\/\h\o\m\e\/\d\i\r" "\1\/\d\i\r" "\2\/\

Is in a new line but why?

Code:

$ echo $( { while read -r -n1 a; do [[ "$a" =~ [[:blank:]] ]] &&  echo -n "\"""$a""\"" || echo -n "\\""$a"; done; } <<<"$inputDirectory" ) | od -t x1c
0000000  5c  2f  5c  68  5c  6f  5c  6d  5c  65  5c  2f  5c  64  5c  69
          \   /   \   h   \   o   \   m   \   e   \   /   \   d   \   i
0000020  5c  72  5c  5c  31  5c  2f  5c  64  5c  69  5c  72  5c  5c  32
          \   r   \   \   1   \   /   \   d   \   i   \   r   \   \   2
0000040  5c  2f  5c  0a
          \   /   \  \n
0000044

---------- Post updated at 06:59 PM ---------- Previous update was at 06:54 PM ----------

Well, this works fine but here strings seem quirky.

Code:

echo -n "$inputDirectory" | { while read -r -n1 a; do [[ "$a" =~ [[:blank:]] ]] &&  echo -n "\"""$a""\"" || echo -n "\\""$a"; done; }

Mike

Michael Stora

View Public Profile for Michael Stora

Find all posts by Michael Stora

08-13-2014

Registered User

15,129, 5,008

Join Date: Jul 2012

Last Activity: 4 May 2020, 4:31 PM EDT

Location: Aachen, Germany

Posts: 15,129

Thanks Given: 735

Thanked 5,008 Times in 4,483 Posts

I don't think they're quirky. bash expands the here string's variable into a temp text file and redirects stdin (FD 0 below) thereto:

Code:

COMMAND  PID    USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
bash    2655 localusr  0r    REG    8,4       16 390451 /tmp/sh-thd-155296093133582787 (deleted)

Any text file is terminated by a <NL> char, which is read into your "a" variable as well and treated/printed out (maybe converted to a NULL char by the shell)

RudiC

View Public Profile for RudiC

Find all posts by RudiC

08-13-2014

Registered User

12,315, 4,560

Join Date: Jul 2012

Last Activity: 22 November 2019, 4:29 PM EST

Location: San Jose, CA, USA

Posts: 12,315

Thanks Given: 952

Thanked 4,560 Times in 3,818 Posts

Quote:

Originally Posted by RudiC

I don't think they're quirky. bash expands the here string's variable into a temp text file and redirects stdin (FD 0 below) thereto:

Code:

COMMAND  PID    USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
bash    2655 localusr  0r    REG    8,4       16 390451 /tmp/sh-thd-155296093133582787 (deleted)

Any text file is terminated by a <NL> char, which is read into your "a" variable as well and treated/printed out (maybe converted to a NULL char by the shell)

The read built-in always strips the trailing <newline> character from lines it reads.

If read didn't work this way, the common sequence:

Code:

while read line
do      echo "$line"
done < file

(a dangerous way to implement cat file as a shell script) would double-space all of its output.

Similarly command substitution (both `commnd` and $(command)) strips trailing <newline> characters if one or more are present. Try, for example:

Code:

printf "%s" "$(printf "abc\n\n")"|od -c

which produces the output:

Code:

0000000    a   b   c                                                    
0000003

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

08-13-2014

Registered User

15,129, 5,008

Join Date: Jul 2012

Last Activity: 4 May 2020, 4:31 PM EDT

Location: Aachen, Germany

Posts: 15,129

Thanks Given: 735

Thanked 5,008 Times in 4,483 Posts

Then, please help me understand this (built on Michael's sample code):

Code:

cat file
1
3
6

hd file
00000000  31 0a 33 0a 36 0a                                 |1.3.6.|

{ while read -r -n1 a; do printf "%c" "$a"|hd; done; } <file
00000000  31                                                |1|
00000001
00000000  00                                                |.|
00000001
00000000  33                                                |3|
00000001
00000000  00                                                |.|
00000001
00000000  36                                                |6|
00000001
00000000  00                                                |.|
00000001

To me it looks like the file is read char by char, and the <NL>s are replaced by NULL chars.

RudiC

View Public Profile for RudiC

Find all posts by RudiC

08-13-2014

Registered User

12,315, 4,560

Join Date: Jul 2012

Last Activity: 22 November 2019, 4:29 PM EST

Location: San Jose, CA, USA

Posts: 12,315

Thanks Given: 952

Thanked 4,560 Times in 3,818 Posts

The read -n num_chars option is an extension to the standards that overrides the default behavior. Try that again without the -n1, and print using strings and you'll get what I suggested:

Code:

( while read -r a ;do printf "%s" "$a"|od -c;done;) < file

producing:

Code:

0000000    1                                                            
0000001
0000000    2                                                            
0000001
0000000    3                                                            
0000001

(I used od -c instead of hd because hd isn't available on my system.)

When invoked as:

Code:

read -r var...

the current standards describe the requirements as:

Quote:

The terminating <newline> (if any) shall be removed from the input and the results shall be split into fields as in the shell for the results of parameter expansion (see Section 2.6.5, on page 2333); the first field shall be assigned to the first variable var, the second field to the second variable var, and so on. If there are fewer fields than there are var operands, the remaining vars shall be set to empty strings. If there are fewer vars than fields, the last var shall be set to a value comprising the following elements:

The field that corresponds to the last var in the normal assignment sequence described above
The delimiter(s) that follow the field corresponding to the last var
The remaining fields and their delimiters, with trailing IFS white space ignored

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

08-13-2014

Registered User

15,129, 5,008

Join Date: Jul 2012

Last Activity: 4 May 2020, 4:31 PM EDT

Location: Aachen, Germany

Posts: 15,129

Thanks Given: 735

Thanked 5,008 Times in 4,483 Posts

Yes, I thought the same and attributed above behaviour to the -n x option.

So, this non-standard behaviour would explain Michael's findings in his first post of this thread, wouldn't it?

RudiC

View Public Profile for RudiC

Find all posts by RudiC

08-13-2014

Registered User

12,315, 4,560

Join Date: Jul 2012

Last Activity: 22 November 2019, 4:29 PM EST

Location: San Jose, CA, USA

Posts: 12,315

Thanks Given: 952

Thanked 4,560 Times in 3,818 Posts

Quote:

Originally Posted by RudiC

Yes, I thought the same and attributed above behaviour to the -n x option.

So, this non-standard behaviour would explain Michael's findings in his first post of this thread, wouldn't it?

Hi Rudi,
Michael Stora hasn't explained what he means by "quirky". The command and corresponding output shown in his first message in this thread:

Code:

$ echo $( { while read -r -n1 a; do [[ "$a" =~ [[:blank:]] ]] &&  echo -n "\"""$a""\"" || echo -n "\\""$a"; done; } <<<"$inputDirectory" ) | od -t x1c
0000000  5c  2f  5c  68  5c  6f  5c  6d  5c  65  5c  2f  5c  64  5c  69
          \   /   \   h   \   o   \   m   \   e   \   /   \   d   \   i
0000020  5c  72  5c  5c  31  5c  2f  5c  64  5c  69  5c  72  5c  5c  32
          \   r   \   \   1   \   /   \   d   \   i   \   r   \   \   2
0000040  5c  2f  5c  0a
          \   /   \  \n
0000044

ends with a <newline> character because the echo shown in red above adds a <newline> to the output from the command substitution. I see nothing surprising in that. Did I miss some deeper point? I certainly see no evidence that read stored a newline character in a at any time.

I guess I didn't pay much attention to this thread before because all of the echo commands in this thread are non-portable and Michael didn't even bother to tell us what OS or shell he's using. Looking at it closer now, I can deduce that his version of echo uses -n to eliminate the trailing <newline> normally produced as output and that it doesn't expand \ escapes in the strings being processed. But with no explanation of what "quirky" means I didn't see anything that would lead me to guess at what he was expecting that he didn't get.

As I have said in MANY MANY other threads, I always (except when responding to threads like this) avoid echo whenever the 1st argument can have a minus sign as the 1st character (even if the 1st argument is --) and whenever any argument can contain a backslash character. Using printf instead of echo requires two more keystrokes plus a format string, but you always know what you'll get.

Don Cragun

View Public Profile for Don Cragun

Find all posts by Don Cragun

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Printing string from last field of the nth line of file to start (or end) of each line (awk I think)

My file (the output of an experiment) starts off looking like this, _____________________________________________________________ Subjects incorporated to date: 001 Data file started on machine PKSHS260-05CP ********************************************************************** Subject 1,...

2. Red Hat

How to add a new string at the end of line by searching a string on the same line?

Hi, I have a file which is an extract of jil codes of all autosys jobs in our server. Sample jil code: ************************** permission:gx,wx date_conditions:yes days_of_week:all start_times:"05:00" condition: notrunning(appDev#box#ProductLoad)...

3. Shell Programming and Scripting

How to extract text from STRING to end of line?

Hi I have a very large data file with several hundred columns and millions of lines. The important data is in the last set of columns with variable numbers of tab delimited fields in front of it on each line. Im currently trying sed to get the data out - I want anything beetween :RES and...

4. Shell Programming and Scripting

Append this string to end of each line

Platform: Solaris 10 I have a file like below $ cat languages.txt Spanish Norwegian English Persian German Portugese Chinese Korean Hindi Malayalam Bengali Italian Greek Arabic I want to append the string " is a great language" at end of each line in this file.

5. Shell Programming and Scripting

Search a string in a text file and add another string at the end of line

Dear All I am having a text file which is having more than 200 lines. EX: 001010122 12000 BIB 12000 11200 1200003 001010122 2000 AND 12000 11200 1200003 001010122 12000 KVB 12000 11200 1200003 In the above file i want to search for string KVB...

6. Shell Programming and Scripting

put string end of the line

I've a problem to put .h end of the line..below my input file fg_a bb fg_b bb fg_c bb fg_d aa fg_f ee and i want the output file as below fg_a.h bb fg_b.h bb fg_c.h bb fg_d.h

7. Shell Programming and Scripting

Add string end of line

Hello How can I add a string (always the same) at the end of a specific line in a file... The file is: 000000001 041 L $$aspa 000000001 088 L $$aJ.E.N. 551 000000001 090 L $$aINFORMES JEN 000000001 100 L $$aautor 1 ---- 000000002 041 L $$aeng 000000002 088 L $$aJ.E.N. 1...

8. Shell Programming and Scripting

How can I append a string at the end of a line in a file

Hi, guys. I have one question: I have a file called "group", the contents of it is below: ******************************** ... test:x:203: sales:x:204: repair:x:205: research:x:206:brownj ... *********** Now I want to add string ",sherrys" at the end of "research:x:206:brownj", so...

9. UNIX for Dummies Questions & Answers

HOWTO Append String to End of Line

I have a comma delimited text file and need to appened ",000000" to the end of every line. For example: Before: "D700000","2006" ,"5000","Open Year" ,"Conversion" ,"Wk64","Productive Payroll $" ,1103.45 After: "D700000","2006" ,"5000","Open Year" ,"Conversion" ,"Wk64","Productive Payroll...

10. Shell Programming and Scripting

To cut end string from line

HI, I want to cut end string from line. e.g. i have following input line /users/home/test.txt I want to get end string 'test.txt' from above line and length of that end string will change and it always start after '/'. Thanks, Visu

Login or Register to Ask a Question