Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to filter only comments while reading a file including line break characters.

View Poll Results: Is this useful ?
no 2 100.00%
yes 0 0%
Voters: 2. This poll is closed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to filter only comments while reading a file including line break characters.
# 1  
Old 05-03-2010
Question How to filter only comments while reading a file including line break characters.
[Solved]
How do I filter only comments and still keep Line breaks at the end of the line!?

This is one of the common tasks we all do,, How can we do this in a right way..!?

I try to ignore empty lines and commented lines using following approach.

test.sh
Code:
# \040 --> SPACE character octal representation. \t --> TAB character octal representation
# echo $IN_FILE_FILTER | od -bcx
# This filter is to ignore empty lines(any space char) && Commented lines
IN_FILE_FILTER=$(echo "^([\040|\t]*(#|$))")

egrep -v "$IN_FILE_FILTER" input.txt | while IFS="=" read var_name var_value
do
        echo "$var_name=$var_value"
done

input.txt
Code:
# Need to ignore this line
# All empty lines with any combination of space characters should be ignored.
                  # need to ignore this line as well 
one="1111111" # Anything after # should be ignored,, still keep line break.
two="22222222"
          
         
        three=333333333

$> test.sh
Code:
one="1111111" # we want anthing after # ignored,, but stll keep line break.
two="22222222"
        three=333333333

From above result, I want to see ..
First Line --> Ignore only comment part ,, while still keeping line break.
Third line --> Trim prefix space characters. ( This one is nice to have but not critical)


Second approach I have seen is to use sed. But it removes line breaks at the end of each line,, thus presenting a different challenge of loosing line breaks. I really never understood how to use this..

Code:
$> echo $(sed 's/#.*//' input.txt | /usr/xpg4/bin/awk -F= ' NF { printf $1 $2; } END'  }

one"1111111" two"22222222" three333333333

Let us see what characters make for spaces according to standards body?

Code:
    http://www.w3.org/TR/REC-xml/#sec-white-space


S (white space) consists of one or more space (#x20)  characters, carriage returns, line feeds, or tabs.
 White Space

    [3]    S    ::=    (#x20 | #x9 | #xD | #xA)+

Please try test the above code, requirements are in input.txt file.
Last edited by kchinnam; 05-05-2010 at 11:53 PM..
# 2  
Old 05-03-2010
Something like this?
Code:
awk 'NF && $1 != "#"{sub("#.*","");print}' file

# 3  
Old 05-03-2010
Code:
awk '$1!="#" && NF{gsub(/^ */,"",$0);sub("#.*",x);print}' file

# 4  
Old 05-03-2010
Code:
$> awk 'NF && $1 != "#"{sub("#.*","");print}' input.txt
awk: syntax error near line 1
awk: bailing out near line 1

I needed to use xpg4 version of awk on my Solaris8 box.

Code:
$>/usr/xpg4/bin/awk 'NF && $1 != "#"{sub("#.*","");print}' input.txt
one="1111111" 
two="22222222"
        three=333333333

Frank That is great., Its working..
Do you have any ideas to trim prefix spaces also?

I am not sure what we are getting by added complexity.. here.. Dan Please explain what the intent is ..

Code:
$>/usr/xpg4/bin/awk '$1!="#" && NF{gsub(/^ */,"",$0);sub("#.*",x);print}' input.txt
one="1111111" 
two="22222222"
        three=333333333

Again,, Please give me ideas to trim space characters at the beginning of lines. I know something line
Code:
cat input.txt | awk '{print $1}'

can do that,, but is there a more elegant one ?
# 5  
Old 05-03-2010
Quote:
Originally Posted by kchinnam
Code:
$> awk 'NF && $1 != "#"{sub("#.*","");print}' input.txt
awk: syntax error near line 1
awk: bailing out near line 1

I needed to use xpg4 version of awk on my Solaris8 box.

Code:
$>/usr/xpg4/bin/awk 'NF && $1 != "#"{sub("#.*","");print}' input.txt
one="1111111" 
two="22222222"
        three=333333333

Frank That is great., Its working..
Do you have any ideas to trim prefix spaces also?

I am not sure what we are getting by added complexity.. here.. Dan Please explain what the intent is ..

Code:
$>/usr/xpg4/bin/awk '$1!="#" && NF{gsub(/^ */,"",$0);sub("#.*",x);print}' input.txt
one="1111111" 
two="22222222"
        three=333333333

Again,, Please give me ideas to trim space characters at the beginning of lines. I know something line
Code:
cat input.txt | awk '{print $1}'

can do that,, but is there a more elegant one ?
Try:
Code:
awk 'NF && $1 != "#"{sub("#.*","");sub(/^ */,"");print}' file

# 6  
Old 05-03-2010
Both, my and Franklin52 solution work for me Smilie

Code:
# awk '$1!="#" && NF{gsub(/^ */,"",$0);sub("#.*",x);print}' file
one="1111111"
two="22222222"
three=333333333
# awk 'NF && $1 != "#"{sub("#.*","");sub(/^ */,"",$0);print}' file
one="1111111"
two="22222222"
three=333333333

# 7  
Old 05-03-2010
I still get the space prefix chracter[s] on third line.. I used a TAB + spacebar on third line in my input.. Again trimming should work for any combination of space characters.

Code:
$> /usr/xpg4/bin/awk 'NF && $1 != "#"{sub("#.*","");sub(/^ */,"");print}' input.txt
one="1111111" 
two="22222222"
         three=333333333
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Including Hash / in sed command filter

Hello All, I want to print data in between two lines in a file sample.txt through more or cat command on the screen. For that I am using below sed command to give the BEGIN and END text. Content of sample.txt server01:~ # cat /proc/mdstat Hello this is a text message 1 Hello this is a... (5 Replies)
Discussion started by: Xtreme
5 Replies

2. UNIX for Dummies Questions & Answers

add a string to a file without line break

I searched and found "echo -n" and "printf" are solution for this, but they are not here: $ echo "hello" >> test $ cat test hello $ echo -n "world" >> test $ cat test hello world $ echo -n " seriously?" >> test $ cat test hello world seriously? This is not successful... (15 Replies)
Discussion started by: stunn3r
15 Replies

3. Shell Programming and Scripting

Add line break for each line in a file

I cannot seem to get this to work.. I have a file which has about 100 lines, and there is no end of line (line break \n) at the end of each line, and this is causing problem when i paste them into an application. the file looks like this this is a test that is a test balblblablblhblbha... (1 Reply)
Discussion started by: fedora
1 Replies

4. Shell Programming and Scripting

[Solved] Problem in reading a file line by line till it reaches a white line

So, I want to read line-by-line a text file with unknown number of files.... So: a=1 b=1 while ; do b=`sed -n '$ap' test` a=`expr $a + 1` $here do something with b etc done the problem is that sed does not seem to recognise the $a, even when trying sed -n ' $a p' So, I cannot read... (3 Replies)
Discussion started by: hakermania
3 Replies

5. Shell Programming and Scripting

filter record from a file reading another file

Hi, I want to filter record from a file if the records in the second column matches the data in another file. I tried the below awk command but it filters the records in the filter file. I want the opposite, to include only the records in the filter file. I tried this: awk -F'|'... (8 Replies)
Discussion started by: gpaulose
8 Replies

6. Shell Programming and Scripting

Break line after last "/" if length > X characters

Hello. I am a french newbie in unix shell scripting (sorry if my english speaking is wrong). I have a file with path and filenames in it. I want to limit the number of characters on each line and break the line if necessary. But the "break" should occur after a slash caracter "/". Example of... (9 Replies)
Discussion started by: SportBilly
9 Replies

7. UNIX for Dummies Questions & Answers

Reading a line including spaces

Hi All, I have a script that reads a file and echo it back to std out. Test.txt 1aaaaaaaaaaa . The script is ReadLine.sh #!/bin/ksh cat $1 | while read file do echo $file done I invoke the script as ReadLine.sh Test.txt The output that I get is (1 Reply)
Discussion started by: aksarben
1 Replies

8. Shell Programming and Scripting

Replacing characters in file with line break

Hi, Apologies if this has been asked before, but I searched and was not able to find an answer. It's probably a simple question to answer for those of you with some experience, though... I have a relatively long string where tokens are separated by the colon (':') character. Let's say the... (10 Replies)
Discussion started by: johnemb
10 Replies

9. Shell Programming and Scripting

Reading a path (including ref to shell variable) from file

Hi! 1. I have a parameter file containing path to log files. For this example both paths are the same, one is stated directly and the second using env variables. /oracle/admin/orcl/bdump/:atlas:trc:N ${ORACLE_BASE}/admin/${ORACLE_SID}/bdump/:${ORACLE_SID}:trc:N 2. I try to parse the path... (1 Reply)
Discussion started by: lojzev
1 Replies

10. Programming

Reading special characters while converting sequential file to line sequential

We have to convert a sequential file to a 80 char line sequential file (HP UX platform).The sequential file contains special characters. which after conversion of the file to line sequential are getting coverted into "new line" or "tab" and file is getting distorted. Is there any way to read these... (2 Replies)
Discussion started by: Rajeshsu
2 Replies
Login or Register to Ask a Question