How to remove spaces using awk,sed,perl?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to remove spaces using awk,sed,perl?
# 8  
Old 08-20-2010
Quote:
Originally Posted by Franklin52
Or:
Code:
awk '$1=$1' file

Interesting.
How does this work?
Code:
$1=$1    #$1 is the first column



---------- Post updated at 05:40 PM ---------- Previous update was at 05:36 PM ----------

Quote:
Originally Posted by kevintse
Wow, this is cool, would you please explain a little bit for this line?
Just do it:
Code:
awk '{print $1}' file
awk '{print $1" "$2}' file
awk '{$1=$1;print}' file

# 9  
Old 08-20-2010
Quote:
Originally Posted by cola
Interesting.
How does this work?
Code:
$1=$1    #$1 is the first column

$1=$1 is a typical awk idiom to force awk to rebuild $0. mostly used to apply a new OFS.
It removes all unnecessary double spaces.
# 10  
Old 08-20-2010
Quote:
Originally Posted by cola
...Why are there so many different types of regular expression?
That shouldn't be so surprising. They are just different methods of arriving at the same solution. Just as there are different routes to go from point A to point B.

The regex in the first awk solution used a character class with the repetition quantifier "+".

Code:
/^[ 	]+/

There's a blank space and a tab character in there which means that this regex searches for one or more occurrences, from the beginning, of either a space or tab character (or any combination of both). The sub function substitutes those with a zero-length string.
I think "\t" for the actual tab character should work; at least it does for gawk:

Code:
gawk '{sub(/^[ \t]+/,""); print}' file

The "+" repetition quantifier is an Extended Regular Expression (ERE). Gnu sed allows it, but the sed binaries in most Unix systems do not. So the other type of regex used in the sed solution was this -

Code:
/^ \{1,\}/

The bracket repetition operator is for finer control over repetition. "+" means one or more - there's no limit for "more". Whereas {m,n} means at least m at the most n repetitions. Further, {m,} means at least m repetitions - there's no upper limit here. So {1,} becomes equivalent to "+", which is why the sed solution works more or less the same. (It doesn't take care of tab characters though).

Quote:
...Perl regular expression is not similar to sed regular expression.
Perl is a different beast altogether. All the above concepts, BREs as well as EREs, are for POSIX regexes. Perl, on the other hand, started off by implementing Henry Spencer's regular expression library. It's regex syntax is richer, more consistent and more extensive than those of POSIX compliant regexes.

HTH,
tyler_durden
# 11  
Old 08-20-2010
this is an very excellent resource, which will be very handy to you while working across several commands: http://www.greenend.org.uk/rjk/2002/06/regexp.html
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can't remove spaces with sed when calling it from sh -c

The following command works echo "some text with spaces" | sh -c 'sed -e 's/t//g''But this doesn't and should echo "some text with spaces" | sh -c 'sed -e 's/ //g''Any ideas? (3 Replies)
Discussion started by: Tribe
3 Replies

2. Shell Programming and Scripting

Using sed, awk or perl to remove substring of all lines except the first

Greetings All, I would like to find all occurences of a pattern and delete a substring from the all matching lines EXCEPT the first. For example: 1234::group:user1,user2,user3,blah1,blah2,blah3 2222::othergroup:user9,user8 4444::othergroup2:user3,blah,blah,user1 1234::group3:user5,user1 ... (11 Replies)
Discussion started by: jacksolm
11 Replies

3. Shell Programming and Scripting

PERL : Remove spaces in a variable

I have a variable I want to remove the spaces in between. The output should be How can this be done Any help will be appreciated. Thanks in advance (1 Reply)
Discussion started by: irudayaraj
1 Replies

4. Shell Programming and Scripting

Need an awk / sed / or perl one-liner to remove last 4 characters with non-unique pattern.

Hi, I'm writing a ksh script and trying to use an awk / sed / or perl one-liner to remove the last 4 characters of a line in a file if it begins with a period. Here is the contents of the file... the column in which I want to remove the last 4 characters is the last column. ($6 in awk). I've... (10 Replies)
Discussion started by: right_coaster
10 Replies

5. Shell Programming and Scripting

sed remove newlines and spaces

Hi all, i am getting count from oracle 11g by spooling it to a file. Now there are some newline characters and blank spaces i need to remove these. pl provide me a awk/sed solution. the spooled file is attached. i tried this.. but not getting req o/p (6 Replies)
Discussion started by: rishav
6 Replies

6. Shell Programming and Scripting

tr and sed remove spaces. how to stop this?

if the answer is obvious, sorry, I'm new here. anyway, I'm using tr to encrypt with rot-13: echo `cat $script | tr 'a-zA-Z' 'n-za-mN-ZA-M'` > $script it works, but it removes any consecutive spaces so that there is just one space between words. I've had this problem before while using sed to... (5 Replies)
Discussion started by: Trichopterus
5 Replies

7. Shell Programming and Scripting

Command to remove duplicate lines with perl,sed,awk

Input: hello hello hello hello monkey donkey hello hello drink dance drink Output should be: hello hello monkey donkey drink dance (9 Replies)
Discussion started by: cola
9 Replies

8. Shell Programming and Scripting

How to delete ending/trailing spaces using awk,sed,perl?

How to delete ending/trailing spaces using awk,sed,perl? Input:(each line has extra spaces at the end) 3456 565 3 7 35 878 Expected output: 3456 565 3 7 35 878 (5 Replies)
Discussion started by: cola
5 Replies

9. Shell Programming and Scripting

sed over writes my original file (using sed to remove leading spaces)

Hello and thx for reading this I'm using sed to remove only the leading spaces in a file bash-280R# cat foofile some text some text some text some text some text bash-280R# bash-280R# sed 's/^ *//' foofile > foofile.use bash-280R# cat foofile.use some text some text some text... (6 Replies)
Discussion started by: laser
6 Replies

10. Shell Programming and Scripting

how to remove spaces in a string using sed.

Hello, I have the following to remove spaces from beginning and end of a string. infile=`echo "$infilename" | sed 's/^ *//;s/ *$//` How do I modify the above code to remove spaces from beginning, end and in the middle of the string also. ex: ... (4 Replies)
Discussion started by: radhika
4 Replies
Login or Register to Ask a Question