Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

File Processing: Handling spaces in a line

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users File Processing: Handling spaces in a line
# 1  
Old 11-15-2013
File Processing: Handling spaces in a line
Hi All,

Iam trying to get a file processed and some lines have spaces...the below is not working

Want to remove empty line
Want to remove lines that start with #
Avoid line with substring WHOA
When trying to get the substring from the var also Iam having trouble

file is like VAR=VALUE, an fiel to set environment to be exact


HTML Code:
for var in $(cat -r $file | sed '/^$/d' | grep -v -E ""^\#"" )
do
      subvar=`echo $var | cut -c1-4`
      if [ "$subvar" != "WHOA" ]
      then
                eval export $var
      fi
done
Sample Input

Code:
MONITOR_OFF=YES
DIR=/usr/as/brs
DATADIR=/usr/df/brs
SHELLDIR1=/usr/as/brs
SHELLDIR2=/usr/as/brs
WHOA= %s : %S - %C [%c xxxx]    --- This is the line Iam trying to avoid by substring logic, this has spaces in between, need logic to get only WHOA also from this line

I dont know how to pass the file using while and IFS as Iam doing some stuff before I use it


Regards
Last edited by baanprog; 11-15-2013 at 01:18 PM..
# 2  
Old 11-15-2013
for x in `cat something` is basically always wrong because of the problem you have discovered among others. You could be doing a while read line ; do ... done < inputfile loop.

If your input file is of the form VAR=VALUE, you can do this:

Code:
while IFS="=" read VAR VALUE
do
        [ -z "$VAR" ] && continue # Ignore blank lines
        case "$VAR" in
        WHOA*)  ;;

        *)
                read $VAR <<EOF
$VALUE
EOF
                export $VAR
                ;;
        esac
done < $file

Which means that, if someone puts `rm -Rf ~/` in your configuration file, it will store that strong instead of running that string like eval would!
This User Gave Thanks to Corona688 For This Post:
baanprog
# 3  
Old 11-15-2013
Let me add little mroe to this
Thanks a lot you seem to be making sense

Let me get you what Iam doing and you can put the whole thing may be, if that is not asking too much

Iam doing below
Want to remove empty line
Want to remove lines that start with #
When trying to get the substring from the var also Iam having trouble. I dont want to read a line with such substrings.

Read the file and process exactly as you said VAR=VALUE line only, an environment file to be exact


HTML Code:
for var in $(cat -r $file | sed '/^$/d' | grep -v -E ""^\#"" )
do
      subvar=`echo $var | cut -c1-4`
      if [ "$subvar" != "WHOA" ]
      then
                eval export $var
      fi
done
Sample Input added as per CarloM request

HTML Code:
MONITOR_OFF=YES
DIR=/usr/as/brs
DATADIR=/usr/df/brs
SHELLDIR1=/usr/as/brs
SHELLDIR2=/usr/as/brs
WHOA= %s : %S - %C [%c xxxx]    --- This is the line Iam trying to avoid by substring logic
Output is to set them as environment variables....
Last edited by baanprog; 11-15-2013 at 12:42 PM..
# 4  
Old 11-15-2013
It would be easier if you provided some sample input & expected output/behaviour.
This User Gave Thanks to CarloM For This Post:
MadeInGermany
# 5  
Old 11-15-2013
Quote:
Originally Posted by baanprog
Iam doing below
Want to remove empty line
Want to remove lines that start with #
When trying to get the substring from the var also Iam having trouble. I dont want to read a line with such substrings.
I noticed that you are using way too many external commands when all of those tasks can be achieved using shell builtins.

Here is an example:
Code:
#!/bin/bash

while read line
do
        # Want to remove empty line
        [[ -z "$line" ]] && continue
        # Want to remove lines that start with #
        [[ "$line" =~ ^# ]] && continue
        # trying to get the substring
        subvar="${line:0:4}"
done < file

This User Gave Thanks to Yoda For This Post:
baanprog
# 6  
Old 11-15-2013
As a first query, could we get rid of the sed part? It more simply written as:-egrep -v "^$|^#" $file

More importantly, do you have a file that contains your environment variables you wish to set? Does it look like this:-
Code:
# My variables
# Some other comment

WHOA1=Hello
WHOA2=Bye
Other=stuff

If so, you can simply source in the file (or an extract of it) rather than worrying about the eval and the very real risks that creates.

In your script, you would simply have:-
Code:
. file

If you just need the selection of variables starting WHOA, you could easily:-
Code:
grep "^WHOA" $file > /tmp/myenv.$$
. /tmp/myenv.$$
rm /tmp/myenv.$$



Does this help, or have I missed the point?


I would like to know more about where Corona688 is leading us, especially the here document bit. I'm a little puzzled and it would be nice to understand the suggestion.


Robin
Liverpool/Blackburn
UK
Last edited by rbatte1; 11-15-2013 at 12:41 PM.. Reason: Spling correxions
This User Gave Thanks to rbatte1 For This Post:
baanprog
# 7  
Old 11-15-2013
Code:
subvar="${line:0:4}"

gives bad substitution error on command line

Well Iam trying to get a substring from VAR which is WHOA and avoid those lines

how to get only WHOA from line
Code:
WHOA= %s : %S - %C [%c xxxx]

Last edited by baanprog; 11-15-2013 at 01:12 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Handling filenames with spaces

I'm trying to handle some files with spaces in their name using "" or \ . Like "file 1" or file\ 1. My current confusion can be expressed by the following shell script: #!/bin/bash touch "file 1" "file 2" echo -n "ls: " ; ls echo --- for file in "file 1" "file 2" ; do echo $file... (9 Replies)
Discussion started by: Ralph
9 Replies

2. Shell Programming and Scripting

[awk] line by line processing the same file

Hey, not too good at this, so I only managed a clumsy and SLOW solution to my problem that needs a drastic speed up. Any ideas how I write the following in awk only? Code is supposed to do... For every line read column values $6, $7, $8 and do a calculation with the same column values of every... (6 Replies)
Discussion started by: origamisven
6 Replies

3. Shell Programming and Scripting

BASH - Handling background processes - distributed processing

NOTE: I am using BASH and Solaris 10 for this. Currently in the process of building a script that has a main "watcher" daemon that reads a configuration file and starts background processes based on it's global configuration. It is basically an infinite loop of configuration reading. Some of the... (4 Replies)
Discussion started by: dcarrion87
4 Replies

4. Shell Programming and Scripting

AWK: Remove spaces before processing each line?

Hi, all I have a file containing the following data: name: PRODUCT_1 date: 2010-01-07 really_long_name: PRODUCT_ABCDEFG I want to get the date (it is "2010-01-07" here), I could use the following code to do that: awk... (6 Replies)
Discussion started by: kevintse
6 Replies

5. Shell Programming and Scripting

Handling directory with spaces in for loops

Hi everyone, I have been a big fan here since a couple years (since I started being an admin ...) and finally decided to become a member and help ppl and perhaps being helped Now I have a problem that might interest some of the gurus. I am abig fan of what I call "one liners". I am trying... (2 Replies)
Discussion started by: plmachiavel
2 Replies

6. UNIX for Dummies Questions & Answers

handling white spaces with getopt

Hi I'm trying to ensure that I have catered for all situations with my getopt cases. One other situation I want to cover is should the user enter the script without any preceding arguments eg: ./script_eg I need the script to the direct the user to the helpfile I have tried... (3 Replies)
Discussion started by: ladyAnne
3 Replies

7. Shell Programming and Scripting

Handling blank spaces

Hi, I am trying to replace a specific column values in a csv file with double quotes when I am find embedded spaces with in the fields. Example: SNO,NAME,ZIPCODE,RANK,SEX,ADDRESS 1,Robert,74538,12,34, M,Robert Street, NY 2,Sam,07564,13,M,12 Main Ave, CA 3,Kim, Ed,12345,14,M,123D ,... (1 Reply)
Discussion started by: techmoris
1 Replies

8. Shell Programming and Scripting

reading a file inside awk and processing line by line

Hi Sorry to multipost. I am opening the new thread because the earlier threads head was misleading to my current doubt. and i am stuck. list=`cat /u/Test/programs`; psg "ServTest" | awk -v listawk=$list '{ cmd_name=($5 ~ /^/)? $9:$8 for(pgmname in listawk) ... (6 Replies)
Discussion started by: Anteus
6 Replies

9. Shell Programming and Scripting

Reading a file line by line and processing for each line

Hi, I am a beginner in shell scripting. I have written the following script, which is supposed to process the while loop for each line in the sid_home.txt file. But I'm getting the 'end of file' unexpected for the last line. The file sid_home.txt gets generated as expected, but the script... (6 Replies)
Discussion started by: sagarparadkar
6 Replies

10. Shell Programming and Scripting

handling spaces in unix

I am testing a ksh script for email. In the script I receive several parameters. One of them is a subject. The subject may contain spaces. Ex. Test this. When I am running the script on telnet to test, how should the syntax at the command line be written. I have this: ksh ResendE.sh '001111'... (2 Replies)
Discussion started by: supercbw
2 Replies
Login or Register to Ask a Question