Visit Our UNIX and Linux User Community


[BASH] read 'line' issue with leading tabs and virtual line breaks


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [BASH] read 'line' issue with leading tabs and virtual line breaks
# 1  
Old 05-08-2015
[BASH] read 'line' issue with leading tabs and virtual line breaks

Heyas

I'm trying to read/display a file its content and put borders around it (tui-cat / tui-cat -t(ypwriter).
The typewriter-part is a 'bonus' but still has its own flaws, but thats for later.

So in some way, i'm trying to rewrite cat using bash and other commands.
But sadly it fails on leading tabs/spaces and virtual line breaks.
[BASH] Getting a semi-tailing backslash when passing (escaped) variables to script , didnt help much, and the other 2 found threads regarding 'tailing backslash' as search criteria are also by me and not helping on this subject.

Code:
	if [ -f "$arg" ]
	then 	while read content #|sed s,"	","    ",g
		do	# Leading dash
			[ "-" = "${content:0:1}" ] && 
				leadingDASHdummy="--" || 
				leadingDASHdummy=""
			# Tabs, become 4 spaces
			echo "$content" | grep -q "\	" && 
				content="$(echo ${content/\	/\    })"
			# Simple conditions
	#		while echo "$content" | grep -q [&|][&|][[:space:]]'\'
	#		do	remover="${content/\	/\n}"
	#			remover="$(echo $content|head -n1)"
	#			tui-printf -E "$remover"
	#			content="${content%${remover:0:${#remover}-3}}"
	#		done
			
			#tui-echo "$LINENO" "$remover" "$content" #; exit
			
	#		content=$(echo "$content" |
	#				sed s,"\	","\\nt",g |
	#				sed s,"\ "$,"\n",g |
	#				sed s,"	","    ",g |
	#				sed s,^"t","    ",g |
	#				sed s,"t","    ",g
	#				)
			# Tailing backslash
			if echo "$content" | grep [&|][&|]' \'
			then	# Loop through virtual line breaks caused by conditions, doesnt really work either, but doesnt break the border lineup.
				while printf "$content" | grep -q "&& \"
				do	#echo in loop > /dev/stderr
					content2=$(echo "$content" | sed s,"&& \","&& \\",g)
					tui-printf -E $leadingDASHdummy"$content2"
					content="${content/${content2%/}}"
				done
			else	tui-printf -E $leadingDASHdummy"${content}"
			fi
		done<"$arg"
	else	RET=1
		tui-printf -S $RET "File not found: $arg"
	fi

The file i'm displaying is the stats.sh scripts (see: [FUN] Get some stats of your project/s)
The code above encapsules the script just fine, with the && \ in the lines-section as a single string properly split (by tui-printf [-E]) onto 2 lines (originaly 4 lines ending with && \).

I'm trying to 'say':
File-to-Display-snippet:
Code:
#	Vars
#
	LINER="########################################"
	#	[ -f "$F" ] && \
	#		COMMENTS=$(( $COMMENTS + $(grep ^"#" "$F" | wc -l) )) && \
	#		LINES=$(( $LINES + $(cat "$F" | wc -l) )) && \
	#		BLANKS=$(( $BLANKS + $(grep ^[[:space:]]$ "$F" | wc -l) ))

Displays as:
Code:
# | # Vars                                                                                                       | #
# | #                                                                                                            | #
# | LINER="########################################"                                                             | #
# | #for F in $(find|grep -ve ".git" -ve ".jpg")                                                                 | #
# | #do [ -f "$F" ] && # COMMENTS=$(( $COMMENTS + $(grep ^"#" "$F" | wc -l) )) && # LINES=$(( $L                 | #
# |                INES + $(cat "$F" | wc -l) )) && # BLANKS=$(( $BLANKS + $(grep ^[[:space:]]$ "$F" | wc -l) )) | #
# | #done                                                                                                        | #

Expected:
Code:
# | #    Vars                                                                                                    | #
# | #                                                                                                            | #
# |     LINER="########################################"                                                         | #
# |     #for F in $(find|grep -ve ".git" -ve ".jpg")                                                             | #
# |     #do    [ -f "$F" ] && \                                                                                  | #
# |     #        COMMENTS=$(( $COMMENTS + $(grep ^"#" "$F" | wc -l) )) && \                                      | #
# |     #        LINES=$(( $LINES + $(cat "$F" | wc -l) )) && \                                                  | #
# |     #        BLANKS=$(( $BLANKS + $(grep ^[[:space:]]$ "$F" | wc -l) ))                                      | #
# |     #done                                                                                                    | #

tui-printf just prints what is passed (and escaped properly), and puts borders around it.
It does a string splitting, as the single string (all the && \ lines are read as ONE line) doesnt match a single output line .

Basicly i need a way to handle the commented out 'Simple Conditions' section (or those - what i call - virtual linebreaks).

Replacing tui-printf -E with echo, results in a faster display, which lacks the same flaws/issues.

I asume i've used the wrong search criteria/text?
And advice / ideas please?

Thank you

---------- Post updated 08-05-15 at 10:49 ---------- Previous update was 07-05-15 at 11:47 ----------

OK, guess i didnt minimize the code enough.
On the way, figured about the thingsi call virutal linebreaks, the read its -r option solved it.

Sample Text:
Code:
#
#	Comment title
#
	Idented text
	[ -z "$EMPTY" ] && \
		echo "yes $EMPTY is empty!"
--hmm
	ident again

Sample Code:
Code:
while read -er line;do
	# Leading: DASH		-- not required for 'plain bash'
	#[ "-" = "${line:0:1}" ] && \
	#	leadingDASHdummy="--" || \
	#	leadingDASHdummy=""
	
	# Converting TABS
	line="$(echo "$line"|sed s,'	','    ',g)"	# Doesnt work for leading tabs??
	# Leading: TAB					-- doesnt seem to work either
	[ "	" = "${line:0:1}" ] && \
		line="${line:0:1}    "
	[ "\t" = "${line:0:2}" ] && \
		line="${line:0:2}    "

	echo "${line}"
done<"sample.txt"


Output - Is:
Code:
#
#    Commment title
#
Idented text
[ -z "$EMPTY" ] && \
echo "yes $EMPTY is empty!"
--hmm


Output - Expected:
Code:
#
#	Commment title
#
	Idented text
	[ -z "$EMPTY" ] && \
		echo "yes $EMPTY is empty!"
--hmm
	ident again

But me still stuck with leading tabs not beeing:
1) Converted to spaces (not required, i thought i'd had more luck to preserve/handle space chars rather than real tabs)
2) Displayed at all Smilie

Any advices please?

---------- Post updated at 11:10 ---------- Previous update was at 10:49 ----------

When i try to apply: (from: Preserve leading white space)
Code:
IFS="" while read -er line
do

It fails with:
Code:
sh sample.sh 
sample.sh: line 1: while: command not found
sample.sh: line 2: syntax error near unexpected token `do'
sample.sh: line 2: `do'

Smilie

EDIT:
hahaha, just had to put IFS="" on the first line, and the read while on the next..
seems to work now....
Code:
IFS=""
while read -r line
do

Smilie

---------- Post updated at 11:17 ---------- Previous update was at 11:10 ----------

Just happy:
Code:
:) tmp $  ../prjs/tui/bin/tui-cat sample.txt 
# | #                                                                                                            | #
# | #    Commment title                                                                                          | #
# | #                                                                                                            | #
# |     Idented text                                                                                             | #
# |     [ -z "$EMPTY" ] && \                                                                                     | #
# |         echo "yes $EMPTY is empty!"                                                                          | #
# | --hmm                                                                                                        | #
# |     ident again                                                                                              | #
✔ tmp $

:dance: Smilie
# 2  
Old 05-08-2015
IFS="" outside the while block has a lasting effect!
In most cases you want the effect only for the read command:
Code:
while IFS="" read -r line
do

This User Gave Thanks to MadeInGermany For This Post:
# 3  
Old 05-08-2015
True, inside the script.
As far i can tell, there is no lasting effect for outside the script:
Code:
:) tmp $  ../prjs/tui/bin/tui-cat -d 0.01 sample.txt 
|| #                                                                                                              ||
|| #    Commment title                                                                                            ||
|| #                                                                                                              ||
||     Idented text                                                                                               ||
||     [ -z "$EMPTY" ] && \                                                                                       ||
||         echo "yes $EMPTY is empty!"                                                                            ||
|| --hmm                                                                                                          ||
||     ident again                                                                                                ||
||                                                                                                                ||
+ tmp $ echo ":$IFS:"
: 	
:

Seems right'ish to me.

EDIT:
I used a single IFS="" for both modes, cat and typewriter (the example above, -d 0.001 is the delay with each char (text only, not borders) is shown.
But changing anyway, for better practice, thank you for pointing out.

Last edited by sea; 05-08-2015 at 09:27 AM..

Previous Thread | Next Thread
Test Your Knowledge in Computers #40
Difficulty: Easy
HTML stands for 'Hyper Technology Transfer Protocol'.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Performance issue - to read line by line

All- We have a performance issue in reading a file line by line. Please find attached scripts for the same. Currently it is taking some 45 min to parse "512444" lines. Could you please have a look at it and provide any suggestions to improve the performance. Thanks, Balu ... (12 Replies)
Discussion started by: balu1729
12 Replies

2. UNIX for Beginners Questions & Answers

Performance issue to read line by line

Hi All- we have performance issue in unix to read line by line. I am looking at processing all the records. description: Our script will read data from a flat file, it will pickup first four character and based on the value it will set up variables accordingly and appended the final output to... (11 Replies)
Discussion started by: balu1729
11 Replies

3. Shell Programming and Scripting

With script bash, read file line per line starting at the end

Hello, I'm works on Ubuntu server My goal : I would like to read file line per line, but i want to started at the end of file. Currently, I use instructions : while read line; do COMMAND done < /var/log/apache2/access.log But, the first line, i don't want this. The file is long... (5 Replies)
Discussion started by: Fuziion
5 Replies

4. Shell Programming and Scripting

Read line, issue with leading - and {}'s

Heyas With my forum search term 'issue with leading dash' i found 2 closed threads which sadly didnt help me. Also me was to eager to add the script, that i didnt properly test, and just now figured this issue. So i have this code: if ] then while read line do line="${line/-/'\-'}"... (7 Replies)
Discussion started by: sea
7 Replies

5. Shell Programming and Scripting

Bash script to read a file from particular line till required line and process

Hi All, Am trying to write wrapper shell/bash script on a utility tool for which i need to pass 2 files as arugment to execute utility tool. Wraper script am trying is to do with above metion 2 files. utility tool accepts : a. userinfo file : which contains username b. item file : which... (2 Replies)
Discussion started by: Optimus81
2 Replies

6. UNIX for Dummies Questions & Answers

Page breaks and line breaks

Hi All, Need an urgent solution to an issue . We have created a ksh file or shell script which generates 1 DAT file. the DAT file contains extract of a select statement . Now the issue is , when we are executing the ksh file , the output is coimng with page breaks and line breaks . We have... (4 Replies)
Discussion started by: Ayaskant
4 Replies

7. Shell Programming and Scripting

Need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line...

Hello, I need a program that read a file line by line and prints out lines 1, 2 & 3 after an empty line... An example of entries in the file would be: SRVXPAPI001 ERRO JUN24 07:28:34 1775 REASON= 0000, PROCID= #E506 #1065: TPCIPPR, INDEX= 003F ... (8 Replies)
Discussion started by: Ferocci
8 Replies

8. Shell Programming and Scripting

Want to remove a line feed depending on number of tabs in a line

Hi! I have been struggling with a large file that has stray end of line characters. I am working on a Mac (Lion). I mention this only because I have been mucking around with fixing my problem using sed, and I have learned far more than I wanted to know about Unix and Mac eol characters. I... (1 Reply)
Discussion started by: user999991
1 Replies

9. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

10. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies

Featured Tech Videos