[BASH] Performance question - Script to STDOUT


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting [BASH] Performance question - Script to STDOUT
# 1  
Old 06-29-2014
RedHat [BASH] Performance question - Script to STDOUT

Hello Coders

Some time ago i was asking about python and bash performances, and i was told i could post the regarding code, and someone would kindly help to make it faster (if possible).

If you have noted, i'm on the way to finalize, finish, stable TUI - Text(ual) User Interface.
It is a framework, aiming to be of help when trying to achieve an Interface on Text level, so it is ment to be used by Scripts, to make it easier for the actual end-user.

This said, the complete package -> all required files are located at: https://github.com/sri-arjuna/tui
Note that it is most optimized for redhad based systems, if you have issues on your system, please let me know about it - and its solution, so i can fix this for others/later.

However, i already have had rewrote it in the aim for performance gain, i'll only include THAT file, representing the core-key-functions for all the presentations done within that package.

The (my) 'logic' behind this is like: printf
Print the information as required, 1 to 3 strings, left, center or right aligned, depending upon arguments, it'll get colored, and depening upon string lengths, even become multilined (not wanted, but required).

Functions are cool, but files/applications are cooler, but took me quite a while to actualy get there -> 2 years...
And i' do like to play with self-supporting-solving-scripts.

So like in most temals, printf is available, and prints on the same line until the line is full.
For my package TUI, tui-printf is the core component for all displaying tasks.

Therefor, it must not only handle several calls, while staying on the very same console line (where possible -> string lenghts), it also must provide color and newlines upon other calls, furthhermore, if the string lenghts provided are too long to be displayed on the current terminal line (whether gui-window or virtual-terminal) it must 'split' them first by arguments-locations (left,cetner,right) then by cutting the actual lenght of a single/certain string, using itself to solve this task.

As a hobby-auto-didactic-learning-kind-of-lazy-guy, i've reached the limits of my brain at this stage for this task (performance).

After this much talking, i'm still not posting the script, as you need to know HOW the script is retrieving its data....
By ENV - so most of the variables required by this TUI-package, are loaded by /etc/profile.d/tui.sh, which not only sources the systemwide configuration in /etc/tui/files, but also in the custom files within $HOME/.config/tui/*.
That way, the ENV is 'filled' with variables(19) like (but not limited to): $DONE (string: [ DONE ] in text, OR [ ✔ ] in GUI.

I'm aware that with the use of tui-status, i could also make another script, returning such similar strings, in which case, in turn, some RET_XY variables would need to be shared by env, which is penetrant within my mind, beeing aware of that, i'm open for suggestions, other than to rewrite it completly to (#)C(++), Python, or whatever.

Finaly, the script, with the most important question:
Do you see anything to increase performance?
File: tui-printf -- Actualy - please see https://github.com/sri-arjuna/tui/bl...bin/tui-printf for a readable structure....
Code:
#!/bin/bash
#
#	Author: 	Simon Arjuna Erat (sea)
#	Contact:	erat.simon@gmail.com
#	License:	GNU Lesser General Public License (LGPL)
#	Created:	2013.05.03
#	Changed:	2014.06.10
	script_version=0.9
#	Description:	Descriptional text goes here
#
#	Variables
#
	#source tui
	[ -z "$BORDER_LEFT$HEAD_BG" ] && source /etc/profile.d/tui.sh
	TITLE="tui-printf"
	ME="${0##*/}"
	ME_DIR="${0/${0##/*/}}"
	help_text="
$ME ($script_version)
$TITLE
Usage: 	$ME [options] [arguments]
	$ME		Prints up to 3 arguments. 
			1: Left
			2: Left, Right
			3: Left, Center, Right
Arguments will tell $ME if and where to colorize, or if to do a linebreak.
Options are:
	-h(elp) 	This screen
	-E(cho)		Simulates 'echo', prints up to 3 strings and a newline.
	-T(itle)	Prints 1st argument centred, if more - aligment like tui-echo
			The inner background is colored white with blue font by default
	-H(header)	Full line has a blue background and white font as default color
"
	FIRST=""
	SECOND=""
	THIRD=""
	WIDTH=$( [ -z $COLUMNS ] && tput cols || printf $COLUMNS )	# Retrieve env or calc it new
	TOTAL="$WIDTH"
	WIDTH=$[ $WIDTH - ${#BORDER_LEFT} - ${#BORDER_RIGHT} - 2 ]	# Subtract default 'borders'
	WIDTH=$[ $WIDTH / 2 * 2 ]					# Make it even 
	EMPTY="$(printf '%*s' $WIDTH)"					# Get 'width' spaces
	doEcho=false
	doHeader=false
	doTitle=false
	optStyle=""
#
#	Catching Arguments
#
	#[[ -z $1 ]] && printf "$help_text" && exit $RET_HELP
	while getopts "EHTh?" name
	do 	case $name in
		E|echo) 	doEcho=true
				optStyle="-E"	;;
		H|header)	doHeader=true
				optStyle="-H"	;;
		T|title)	doTitle=true
				optStyle="-T"	;;
		h|"?"|*)	printf "$help_text"
				exit $RET_HELP
				;;
		esac
	done
	shift $(($OPTIND - 1))
	ARGS=(${*})			# Remaining arguments	| These two fail on spaced-strings
	ARGS_COUNT_org=${#ARGS[@]}	# Amount of remaining
	#FIRST="${ARGS[0]}"
	#SECOND="${ARGS[1]}"
	#THIRD="${ARGS[2]}"
	ARGS_COUNT=0
	[ ! -z "$1" ] && FIRST="$1" && ((ARGS_COUNT++))
	[ ! -z "$2" ] && SECOND="$2" && ((ARGS_COUNT++))
	[ ! -z "$3" ] && THIRD="$3" && ((ARGS_COUNT++))
	[ ! -z "$4" ] && doMore=true || doMore=false
#
#	Verify all fit on 1 line
#
	len=$[ ${#FIRST} + ${#SECOND} + ${#THIRD} ]
	if [[ $len -gt $WIDTH ]]
	then 	len2=$[ ${#FIRST} + ${#SECOND} ]
		#echo $len $len2 $WIDTH
		if [[ $len2 -le $WIDTH ]]
		then 	# first & second match on one line
			tui-printf -E "$FIRST" "$SECOND"
			tui-printf $optStyle "$THIRD"
		else	# it needs further spliting:
			if [[ ${#FIRST} -le $WIDTH ]]
			then 	tui-printf -E "$FIRST"
				tui-printf $optStyle "$SECOND" "$THIRD"
			else	half=$[ ${#FIRST} / 2 ]
				part1="${FIRST:0:$half}"
				part2="${FIRST:$half}"
				tui-printf -E "$part1"
				[[ -z $SECOND ]] && \
					tui-printf -E "$part2" || \
					tui-printf -E "$part2" "$SECOND"
				[[ -z $THIRD ]] || \
					tui-printf $optStyle "$THIRD"
			fi
		fi
		# Since this is special handling, 
		# calling itself with reduced string lengths
		# so we can exit after this is done
		exit
	# else # just continue with the script below
	fi
#
#	Prepare Colors & Strings
#
	if [ $doEcho = true ]
	then	COLOR_LINE_START="\r${TUI_RESET}"
		COLOR_LINE_IDENT=""
		COLOR_LINE_CLOSE=""
		COLOR_LINE_END="\n"
	elif [ $doHeader = true ]
	then	# Full line
		COLOR_LINE_START="\r${HEAD_BG}${HEAD_FG}"
		COLOR_LINE_IDENT=""
		COLOR_LINE_CLOSE=""
		COLOR_LINE_END="${TUI_RESET}\n"
	elif [ $doTitle = true ]
	then	# 'Inlay'
		COLOR_LINE_START="\r"
		COLOR_LINE_IDENT="${TITLE_FG}${TITLE_BG}"
		COLOR_LINE_CLOSE="${TUI_RESET}"
		COLOR_LINE_END="\n"
	else	# The default
		COLOR_LINE_START="\r${TUI_RESET}"
		COLOR_LINE_IDENT=""
		COLOR_LINE_CLOSE=""
		COLOR_LINE_END=""
	fi
	case $ARGS_COUNT in
	0)	FIRST="$EMPTY"
		;;
	1)	if [ $doTitle = true ]
		then 	SECOND="$FIRST"
			num_o="${#SECOND}"
			num=$[ $num_o / 2 * 2  ]
			[ $num_o -gt $num ] && adder=" "||adder="  "
			EMPTY="${EMPTY:${num}+2}"	# The +2 is a temp bugfix
			half=$[ ${#EMPTY} / 2  ]
			FIRST="${EMPTY:$half}"
			THIRD="$FIRST$adder"
		else	# Usual arangement
			SECOND=$"${EMPTY:${#FIRST}}"
		fi
		;;
	2)	THIRD="$SECOND"
		SECOND="${EMPTY:${#FIRST}+${#SECOND}}"
		;;
	*)	# 3 (BETA - and more - handler)
		EMPTY="${EMPTY:${#FIRST}+${#SECOND}+${#THIRD}}"
		half=$[ ${#EMPTY} / 2 ]

		len_strings=$[ ${#FIRST} + ${#SECOND} + ${#THIRD} ]
		len_compare=$[ $len_strings / 2 * 2  ]
		[ $len_compare -lt $len_strings ] && \
			adder=1 || \
			adder=0

		SECOND="${EMPTY:$half}$SECOND${EMPTY:$half+$adder}"
		;;
	esac
#
#	Display & Action
#
	printf "${COLOR_LINE_START}${BORDER_LEFT}${COLOR_LINE_IDENT} ${FIRST}${SECOND}${THIRD} ${COLOR_LINE_CLOSE}${BORDER_RIGHT}${COLOR_LINE_END}"
#
#	BETA "if more"
#
	#unset ARGS[0] ARGS[1] ARGS[2]
	if [ $doMore = true ]
	then	# there are remains..
		C=0
		for A in "${ARGS[*]}";do 
			[ "$1" = "$A" ] && unset ARGS[$C] && echo $A
			[ "$2" = "$A" ] && unset ARGS[$C] && echo $A
			[ "$3" = "$A" ] && unset ARGS[$C] && echo $A
			((C++))
		done
		tui-title "DEBUG"
		tui-echo  "Too many arguments:" "$ARGS_COUNT($ARGS_COUNT_org)=${ARGS[*]}"
		# DEBUG
		echo $ME $optStyle "${ARGS[*]}"
		exit
	fi

Please understand, i've had no understand about lincenses, and still its a jungle to me.. .it'll change, i'm just not sure to what - atm...
I'm aiming to be as FOSS as possible, so i was told GPLv3 was the way to go/change?

Thank you in advance for any constructive input. Smilie
Moderator's Comments:
Mod Comment Tabs are maintained in CODE tags and seem to work just fine in the code shown. PHP tags have been changed to CODE tags. There are, however, lots of places where the code would be much more readable if it contained more tabs to highlight the structure of your code.

Last edited by sea; 07-24-2014 at 07:22 PM..
# 2  
Old 06-29-2014
A little disappointed that the two optimizations I told you about in this post:

https://www.unix.com/302906808-post8.html

are not in the above script, did you give them a try, and if so was there some concerns/failure to improve performance that caused you to not implement them?

Please consider making tui-printf a function definition in this file and sourcing it in any client code, as this is a recursive routine and called many times from each client, you will benefit from not loading a new shell each time it is used.
This User Gave Thanks to Chubler_XL For This Post:
# 3  
Old 06-29-2014
Sorry and thank you.
I was sick for a week and had troubles with several services i didnt want to solve until i was healthy again.. medics-brain and such combinations...
This must have 'passed' my 'awarness' today, but my system is working again.

While i will apply your suggestions (its 4am atm, so it'll be after sleep & understanding them), i can say already, it'll remain in just that script, as the display must (also) match (possible changing) GUI terminal lengths...
If it'd be a 'pre-loaded' env variable, it'll always be either the max lenght, or the lenght of the terminal window upon call, but never ever be the matching length once the terminal window was (if/when) resized once or more times.

Of which i fail to see/provide the requirements if i'd place it in /etc/prfile.d/tui.sh, eventhough i see the performance bonus by ITSELF , it looses its USAGE bonus overall... -> thus the -z $VAR, which i thought to be the fastest check...
-->
Quote:
Originally Posted by Chubler_XL
Please consider making tui-printf a function definition in this file and sourcing it in any client code, as this is a recursive routine and called many times from each client, you will benefit from not loading a new shell each time it is used.
Not really understanding... are you saying, sourcing an additional script, just containing the ($WIDTH) code, is faster than leaving the code in the file that is actualy executed?
Keep in mind, tui-echo, tui-header, tui-title, tui-status, etc, are ALL calling --just-- tui-printf, making that 'application-script' sourcing an addition file, makes the complete procedure (mechanicly) slower, or am i wrong?

Maybe the two lines are just one, its late. Smilie

EDIT5orso...
I've have had a function called printx provided in the script '/bin/tui' which was sourced instead of /etc/profile.d/tui.sh, but in combination with the other things, specialy using that function for its main prupsose.. printf... was slower than it is now.
Since tui-rpintf required tui to be source, so the function was available, this way around, tui-printf IS the function, available right from the terminal, rather than having to source something first.

Please correct me if i'm 'wrong', that was just my testet (1-3x) experience.

EDIT 6 -- PPS-Notes:
/etc/profile.d/tui.sh: Provides only 'hard-coded' and for reusage defined variables.
$WIDTH is very flexible on its definition, that is absolute true in GUI mode with one or more terminal-windows open that may or may not be on the same screen, and due to the GUI be resized to the pleasure/need of the user.

EDIT for Don:
Sorry, seems one of my Firefox addins/plugins scratches that display functionality...

Last edited by sea; 06-30-2014 at 12:15 AM..
# 4  
Old 06-29-2014
4am, get some sleep and look at this later!

When your read consider that even if the function is defined internal to your script (via source or defined directly) it can still dynamically integrate the terminal width.

Here is a little demo of what I'm talking about. I've used MYCOLS instead of the shell COLUMNS variable as it's easier to change and demonstrate:

Code:
function tui-printf
{
   local size1
   WIDTH=${MYCOLS:-$(tput cols)}
   printf "Width = %d\n" "$WIDTH"

   printf "before: Size1 = %d, Size2 = %d\n" ${size1:-0} ${size2:-0}
   ((size1=WIDTH/8))
   ((size2=WIDTH/8))
   printf "after: Size1 = %d, Size2 = %d\n" ${size1:-0} ${size2:-0}
}

tui-printf
MYCOLS=120
tui-printf
unset MYCOLS
tui-printf

Also note above that you use local to control if the values in variables will carry over from the last call or not.

Last edited by Chubler_XL; 06-29-2014 at 11:49 PM.. Reason: Added example of local variables
This User Gave Thanks to Chubler_XL For This Post:
# 5  
Old 06-30-2014
Thank you Chubbler, will test/play with it after sleep.

Just had another look at the end of the script, and a question arised...
May i add an additional question, and ideas about how to handle more than 3 arguments?
I've had some tries handling that, but never was satisfied with my own solutions/attempts...
Script-Tools / Code / [87ec4b] /Included_Functions/system.interface.cli
Lines:
197-208
and / or
240-254

Which includes the double-quesiton:
How many, and how much must/should the argumental strings be alignent?

This is part of usage and performance, thought its a good question before i go sleep Smilie

-----
EDIT'ed as i'm not sure about it:
Personaly, i've rarley used more than 2, only for display handlers (such as tui-dd/tui-tar/tui-bgjob), i've added a third argument - well of course for the centred title too.

So, as an UI, how man alignments do (would) you need on a text base?
For more than 3 strings, i fail to figure an 'algorythm' without the need to provide empty space strings on several occasions.
I somehow want to support more than the current 3 aligments, yet i'm not sure if its worth the trouble.

That part could/might become (yet another) script/application.
# 6  
Old 06-30-2014
I'm not entirely sure what these aliments are supposed to look like but this version fills on either side of each argument to your cliWidth and could be a good example/starting point for you:

Code:
    sP() {
        lentotal=$cliWidth
        lenborders=$(( ${#bl} + ${#br} ))
        argstr="$@"
        lenargs=$(( ${#argstr} - $# + 1))
        SPACES=$(( lentotal - lenborders - lenargs - 2))
        if [ $SPACES -gt $# ]
        then
            # Room for args on single line
            printf -v spc "%*s" $((SPACES / ($# + 1))) ""
            printf -v str "$spc%s" $@
            printf "\r%s %-*s %s" "$bl" $((lenargs + SPACES)) "$str" "$br"
        else
            # Need more than 1 line here
        fi
    }

# 7  
Old 06-30-2014
Here are a few additional comments on your code to consider:
Line 17:
Using bash version GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13) on Mac OS X, the statement: ME_DIR="${0/${0##/*/}}" gives me the error: bad substitution: no closing `}' in ${0##. This may be a bug in this version of bash, since ksh sets ME_DIR to the directory specified when your script is invoked (or an empty string if no directory was specified). There are several other ways (easier or faster or both) to get the directory from $0 depending on how ME_DIR is to be used, but since ME_DIR is never referenced after being set, the best thing to do is to just delete this line.

Line 37:
This can be more simply written as WIDTH=${COLUMNS:−$(tput cols)}

Line 38:
TOTAL is not used; delete this line.

Line 39:
This can be more simply written: ((WIDTH -= (${#BORDER_LEFT} + ${#BORDER_RIGHT} + 2)))

Lines 52, 54, 56, 73, 109, 114, 120, 135, 172:
If you change var=true and var=false to var=1 and var=0, respectively; you can then simplify if [ $var = true ] to if [ $var ]. These changes won't make a huge performance difference, but assigning one character to a variable is a little bit faster than assigning for or five characters and [ $var ] is a little faster than [ $var = string ].

Line 49:
This comment doesn't seem to be needed. If it is needed, it should appear closer to line 58.

Line 64-73:
I have no idea why you are playing all of these games with your command line arguments. If you want to know how many arguments there are, use $#. I see absolutely no reason why it wouldn't be valid to call tui-printf with two empty strings and a third non-empty string tui-print "" "" "right string" to print empty left and middle headers. To determine if $2 was given on the command line, use [ $# -ge 2 ] not [ ! -z "$2" ] (and not [ -n "$2" ] which would have been a simpler way to code that).

Line 64:
Of course ARGS=(${*}) fails to preserve argument boundaries if any of the args contain spaces. If you insist on using $ARGS instead of just using $1 though $n, you need something like ARGS=("$@").

Line 137-140:
I don't understand why you have such complex code to set adder to a single space. I would have thought you would want to set adder to a space if the length of $2 was odd, or to an empty string if the length was even. You could replace these four lines with the simpler (and probably faster):
Code:
	num=$(($#SECOND / 2 * 2))
	[ $num -ne ${#SECOND} ] && adder=' ' || adder=''

Line 155-159:
The variable len_compare is not needed here and your computation of adder is slow compared to other alternatives. Change lines 156-159 to:
Code:
	adder=$((len_strings % 2))

Note tha I also strongly suggest that you change adder to somehting else. You used adder on lines 139 and 143 to hold a string; while on lines 158, 159, and 161 you use adder to hold a number. Please use different variables for each intended use.

Line 167:
All of your calculations of string lengths are completely wrong if any operand to tui_printf contains any percent characters (%), backslash escapes (such as \n, \t, and \b), literal control characters, or non-printing characters.

If the user could ever want literal \ characters or literal % characters to be included in the printed output, you should change:
Code:
printf "${COLOR_LINE_START}${BORDER_LEFT}${COLOR_LINE_IDENT} ${FIRST}${SECOND}${THIRD} ${COLOR_LINE_CLOSE}${BORDER_RIGHT}${COLOR_LINE_END}"

to:
Code:
printf "${COLOR_LINE_START}${BORDER_LEFT}${COLOR_LINE_IDENT} %s ${COLOR_LINE_CLOSE}${BORDER_RIGHT}${COLOR_LINE_END}" \
	"$FIRST$SECOND$THIRD"

This User Gave Thanks to Don Cragun For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the... (15 Replies)
Discussion started by: SDohmen
15 Replies

2. Shell Programming and Scripting

Bash Script Iterating Question

I am trying to look through one of my directories to remove certain files. I am pretty new to Unix and bash so I just need a little help in starting this. I know I would have to write two loops one to iterate the directories and one to iterate the files. How would I write the loops? (3 Replies)
Discussion started by: totoro125
3 Replies

3. Shell Programming and Scripting

Question about writing a bash script

Hello, I want to write a bash script to delete the content after '#'. However, if '#' appears in a string with "", ignore this. For example, input file: test #delete "test #not delete" Output file: test "test #not delete" Does anyone know how to write this script? Thanks (1 Reply)
Discussion started by: jeffwang66
1 Replies

4. Shell Programming and Scripting

Question in bash script.

Hi All, I need an assistance with the issue below. I wrote big script in "bash" that automatically install an LDAP on Clients. I'd be happy to know in order to avoid duplication of entries in files, How i can define into the script, if the specific expressions already exist in the file, do... (7 Replies)
Discussion started by: Aviel.shani
7 Replies

5. Shell Programming and Scripting

Mkbootfs writing to stdout in bash script

Hi, I need to automate some repacking tasks of a boot image for Android When in command line, I can use this command: mkbootfs /path/to/root > /path/to/ramdisk-recovery.cpio;However, if I try to run the command from a shell script under Ubuntu, it fails and outputs to stdout instead of the... (27 Replies)
Discussion started by: Phil3759
27 Replies

6. Shell Programming and Scripting

bash, help with stdout manipulation.

Hey all, Im kind of lost on how to do what I want so I figured I would ask. I want to pipe STDOUT of an app to a log file, but I want to prepend each line of that output with the date and time. Im drawing a complete blank on how to do this?? Any ideas? i.e. output is currently this:... (9 Replies)
Discussion started by: trey85stang
9 Replies

7. Shell Programming and Scripting

bash script question

Can anybody be kind to explaing me what the lines below mean ? eval line=$line export $line ] || echo $line (2 Replies)
Discussion started by: jville
2 Replies

8. Shell Programming and Scripting

BASH script question

Hi, I want to create a script that gets a filename as an argument. The script should generate a listing in long list format of the current directory, sorted by file size. This list must be written to a new file by the filename given on the command line. Can someone help me with this? ... (6 Replies)
Discussion started by: I-1
6 Replies

9. Shell Programming and Scripting

one question for bash shell script

Just one question for bash shell script. In bash script, you can use *.txt to call any files in current folder that ends with .txt, like rm *.txt will remove all txt file in current folder. My question is can you actually remember or use the file name among *.txt, I know file=*.txt will not... (9 Replies)
Discussion started by: zx1106
9 Replies

10. Shell Programming and Scripting

BASH shell script question

I want to make shell script that takes a list of host names on my network as command line arguments and displays whether the hosts are up or down, using the ping command to display the status of a host and a for loop to process all the host names. Im new to shell scripting so Im not quite sure... (3 Replies)
Discussion started by: ewarmour
3 Replies
Login or Register to Ask a Question