Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

[BASH] Read value of variable, but not comment on the same line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users [BASH] Read value of variable, but not comment on the same line
# 1  
Old 08-30-2015
[BASH] Read value of variable, but not comment on the same line
Heyas

So while i'm trying to increase security/usability of my TUI commands, i'm currently at tui-conf-get.
There was also a bug, i wanted to fix, but sadly that bugfix is more insecure - but twice as fast as my current buggy method.

I've added a getopts toggle, '-s' to toggle beSecure, meaning to use either source or awk.

Anyway, here's the code after getopts:
Code:
	CONFFILE="$1"
	VARNAME="$2"
	
	# If VARNAME starts with Pass|password|pw, do not source the conf file but use awk
	case "$VARNAME" in
	[Pp][Aa][Ss][Ss][Ww][Oo][Rr][Dd]|[Pp][Ww]|[Pp][Aa][Ss][Ss])
		beSecure=true
		;;
	esac
	
	[ ! -f "$CONFFILE" ]  && tui-printf -S 1 "$CONFFILE does not exist!" && exit 1
	[ "$CONFFILE" = "${CONFFILE##*/}" ] && CONFFILE="./$CONFFILE"
#
#	Display & Action
#
	if $beSecure
	then	$AWK -F= -v VAR="$VARNAME" '$1 ~ "^[^#]*" VAR "$"{print $2,$3,$4,$5}' "$CONFFILE"|$SED s,'"','',g|head -n1
	else	# This method is about twice the speed of the above
		source "$CONFFILE" && echo "${!VARNAME}"
	fi

To illustrate my bug:
Code:
$ echo "variable=value # unwanted comment" > tmp

$ tui-conf-get -s tmp variable
value # unwanted comment

$ tui-conf-get tmp variable
value

Furthermore, if the variable is a list, i'm getting either too few and too many entries with the same awk call on the very only variable read:
Code:
echo "list=\"one two three four five six\" # bad comment" >> tmp

bin/tui-conf-get tmp list
one two three four five six

bin/tui-conf-get -s tmp list
one two three four # bad comment

I'm aware that this is because in awk i only 'use' $2-$5, but then again, why is the comment shown?

My questions:
  1. Is there a way to tell awk to print all columns but not $1?
    I mean, i could add like $2,$3,....$22,$23 -- but what if the list read has 25 or more items?
  2. As i already use sed with the awk command, how would i modify it to remove the quotes, and the hash followed by the comment?

When trying basic regex with sed, i cant get it to work as i expect it to:
Code:
$ echo "\"1 2 3 4 5 6\" # comment" | sed s,[\"\#*],,g
1 2 3 4 5 6  comment

Any ideas/advice?
Thank you
# 2  
Old 08-30-2015
Trying to emulate how the shell parses a file is going to be slower - but I guess you are trying to avoid side-effects if beSecure is true.

What functionality is the -s providing?

Also, a suggestion:
Code:
	case "$VARNAME" in
	[Pp][Aa][Ss][Ss][Ww][Oo][Rr][Dd]|[Pp][Ww]|[Pp][Aa][Ss][Ss])
		beSecure=true
		;;
	esac

could be simpified to:
Code:
	case "${VARNAME^^*}" in
	PASSWORD|PW|PASS)
		beSecure=true
		;;
	esac

Also, I am curious about this construct:
Code:
[ "$CONFFILE" = "${CONFFILE##*/}" ] && CONFFILE="./$CONFFILE"

Why are you prepending ./?
This User Gave Thanks to derekludwig For This Post:
sea
# 3  
Old 08-30-2015
Quote:
Originally Posted by sea
.
.
.
My questions:
  1. Is there a way to tell awk to print all columns but not $1?
    I mean, i could add like $2,$3,....$22,$23 -- but what if the list read has 25 or more items?
  2. As i already use sed with the awk command, how would i modify it to remove the quotes, and the hash followed by the comment?
1) Try sub ($1 FS, "") - should work as long as FS is one single char.
2) Why the sed invocation? And the head -n1? Try
Code:
awk -F= -v VAR="$VARNAME" '$1 ~ "^[^#]*" VAR "$"{gsub ("#.*$|\"|"$1 FS, ""); print; exit}'

Quote:
When trying basic regex with sed, i cant get it to work as i expect it to:
Code:
$ echo "\"1 2 3 4 5 6\" # comment" | sed s,[\"\#*],,g
1 2 3 4 5 6  comment

Any ideas/advice?
Thank you
What exactly do you expect?
This User Gave Thanks to RudiC For This Post:
sea
# 4  
Old 08-30-2015
@ derekludwig
Quote:
Originally Posted by Post #1
I've added a getopts toggle, '-s' to toggle beSecure , meaning to use either source or awk .
True about that ${VAR^^}, didnt thought of that. Smilie
The prefixing of CONFFILE with ./ was required to avoid an error message when it was sourced.

@ RudiC
Since i've gotten either the 'full line' or $2-$5 incl quotes and comments, i wanted to remove the both of them.
Now that is already done with your awk/gsub suggestion. Smilie

EDIT: The head was if a variable definition was copied to another line, at some occasions i had 2 output lines.

/solved & improved
Thank you guys
Last edited by sea; 08-30-2015 at 09:12 AM..
# 5  
Old 08-30-2015
The error message was generated because . was not specified as a directory to search by the PATH environmental variable (and bash is not in POSIX mode).

If a variable occurs more than once in a configuration file, wouldn't you want the LAST occurrence?
This User Gave Thanks to derekludwig For This Post:
sea
# 6  
Old 08-30-2015
Yikes true, happened because i had commented out the lower line in the testfile.
You asked why i added the ./ Smilie

Currently have:
Code:
+ tui $ bin/tui-conf-get -s tmp list
one two three four five six 
+ tui $ bin/tui-conf-get tmp list
1 2 3 4 5 6
+ tui $ cat tmp
variable=value # unwanted commend
list="one two three four five six" # bad comment
list='1 2 3 4 5 6'

I tried to also remove single quotes with the awk, and appended a tail, but it fails:
Code:
awk -F= -v VAR="$VARNAME" '$1 ~ "^[^#]*" VAR "$"{gsub ("#.*$|\"|\'|"$1 FS, ""); print}' "$CONFFILE" | tail -n1

But now fails with:
Code:
LC_ALL=C bin/tui-conf-get -s tmp list
bin/tui-conf-get: line 136: unexpected EOF while looking for matching `"'
bin/tui-conf-get: line 138: syntax error: unexpected end of file

As soon i remove the \'| it works again like:
Code:
LC_ALL=C bin/tui-conf-get -s tmp list
'1 2 3 4 5 6'

Last edited by sea; 08-30-2015 at 10:11 AM..
# 7  
Old 08-30-2015
Try
Code:
awk -F= -v VAR="$VARNAME" '$1 ~ "^[^#]*" VAR "$"{gsub ("#.*$|\"|\047|"$1 FS, ""); RES=$0} END {print RES}' "$CONFFILE"
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Using read to assign value to bash variable not working

Hi, I am attempting to assign the output of the following command, to two bash variables, var1 and var2 using "read," but it doesn't seem to be working. # openstack hypervisor stats show | awk -F'|' 'NR==14{print $2,$3}' vcpus 92 # echo $? 0 # openstack hypervisor... (4 Replies)
Discussion started by: sand1234
4 Replies

2. Shell Programming and Scripting

How to read the output of a command line by line and pass it as a variable?

Hi, I have some 2000 names in a table like below. Java Oracle/SQL ANSI SQL SQL,DWH,DB DB&Java And by using for loop in my code i am able to get a single word but if there is any special character or space then it is considering as a next line. I have to execute the below queries in... (10 Replies)
Discussion started by: Samah
10 Replies

3. Shell Programming and Scripting

Ksh: Read line parse characters into variable and remove the line if the date is older than 50 days

I have a test file with the following format, It contains the username_date when the user was locked from the database. $ cat lockedusers.txt TEST1_21062016 TEST2_02122015 TEST3_01032016 TEST4_01042016 I'm writing a ksh script and faced with this difficult scenario for my... (11 Replies)
Discussion started by: humble_learner
11 Replies

4. Shell Programming and Scripting

With script bash, read file line per line starting at the end

Hello, I'm works on Ubuntu server My goal : I would like to read file line per line, but i want to started at the end of file. Currently, I use instructions : while read line; do COMMAND done < /var/log/apache2/access.log But, the first line, i don't want this. The file is long... (5 Replies)
Discussion started by: Fuziion
5 Replies

5. Shell Programming and Scripting

[BASH] read 'line' issue with leading tabs and virtual line breaks

Heyas I'm trying to read/display a file its content and put borders around it (tui-cat / tui-cat -t(ypwriter). The typewriter-part is a 'bonus' but still has its own flaws, but thats for later. So in some way, i'm trying to rewrite cat using bash and other commands. But sadly it fails on... (2 Replies)
Discussion started by: sea
2 Replies

6. Shell Programming and Scripting

Bash script to read a file from particular line till required line and process

Hi All, Am trying to write wrapper shell/bash script on a utility tool for which i need to pass 2 files as arugment to execute utility tool. Wraper script am trying is to do with above metion 2 files. utility tool accepts : a. userinfo file : which contains username b. item file : which... (2 Replies)
Discussion started by: Optimus81
2 Replies

7. Shell Programming and Scripting

How to read a two files, line by line in UNIX script and how to assign shell variable to awk ..?

Input are file and file1 file contains store.bal product.bal category.bal admin.bal file1 contains flip.store.bal ::FFFF:BADC:CD28,::FFFF:558E:11C5,6,8,2,1,::FFFF:81C8:CA8B,::FFFF:BADC:CD28,1,0,0,0,::FFFF:81C8:11C5,2,1,0,0,::FFFF:81DC:3111,1,0,1,0 store.bal.... (2 Replies)
Discussion started by: veeruasu
2 Replies

8. Shell Programming and Scripting

How to read a file line by line and store it in a variable to execute a program ?

Hello, I am quite new in shell scripting and I would like to write a little scritp to run a program on some parameters files. all my parameters files are in the same directory, so pick them up with ls *.para >>dirafter that I have a dir file like that: param1.para param2.para etc... I... (2 Replies)
Discussion started by: shadok
2 Replies

9. Shell Programming and Scripting

bash: read file line by line (lines have '\0') - not full line has read???

I am using the while-loop to read a file. The file has lines with null-terminated strings (words, actually.) What I have by that reading - just a first word up to '\0'! I need to have whole string up to 'new line' - (LF, 10#10, 16#A) What I am doing wrong? #make file 'grb' with... (6 Replies)
Discussion started by: alex_5161
6 Replies

10. Shell Programming and Scripting

In bash, read to a variable with a default value to edit

My script needs to read a variable from the user. But before the user types the input, I want to give a default value so the user can edit the default value for input. How can I implement it? My script is something like: #!/bin/sh read -p 'Port number: ' -e port_number echo "Port... (7 Replies)
Discussion started by: pankai
7 Replies
Login or Register to Ask a Question