Sponsored Content
Top Forums Shell Programming and Scripting SED help - cleaning up code, extra spaces won't go away Post 302416010 by durden_tyler on Saturday 24th of April 2010 09:16:05 AM
Old 04-24-2010
Quote:
Originally Posted by Matthias03
...
What I don't understand, why 'this' isn't working:
Code:
VAR=`awk -F, '{print $2}' input.txt | sed 's/-i/\|/g;s/^.\{1\}//g;s/ //g'`

???
It'll produce something like this:
b e h
... I'm losing the "|", and the spaces are still there?
...
Well, I do see the "|" characters in my output:

Code:
$ 
$ cat -n input.txt
     1    -h a, -i b, -j c
     2    -h d, -i e, -j f
     3    -h g, -i h, -j i
$ 
$ awk -F, '{print $2}' input.txt | sed 's/-i/\|/g;s/^.\{1\}//g;s/ //g'
|b
|e
|h
$

But if I assign the quoted output of that pipeline to a shell variable, then spaces are introduced.

Code:
$ 
$ VAR="`awk -F, '{print $2}' input.txt | sed 's/-i/\|/g;s/^.\{1\}//g;s/ //g'`"
$ 
$ echo $VAR
|b |e |h
$

The command pipeline's output did *not* have spaces at the end:

Code:
$ 
$ awk -F, '{print $2}' input.txt | sed 's/-i/\|/g;s/^.\{1\}//g;s/ //g' | od -bc
0000000 174 142 012 174 145 012 174 150 012
          |   b  \n   |   e  \n   |   h  \n
0000011
$

So I guess the shell replaces those newlines by blank spaces when it is assigned to a (shell) variable:

Code:
$ 
$ VAR=`awk -F, '{print $2}' input.txt | sed 's/-i/\|/g;s/^.\{1\}//g;s/ //g'`
$ echo $VAR
|b |e |h
$ 
$ ## or quoted
$ VAR="`awk -F, '{print $2}' input.txt | sed 's/-i/\|/g;s/^.\{1\}//g;s/ //g'`"
$ echo $VAR
|b |e |h
$ 
$ echo $VAR | od -bc
0000000 174 142 040 174 145 040 174 150 012
          |   b       |   e       |   h  \n
0000011
$

Here's another way to extract a pipe-delimited output from the file using plain awk:

Code:
$ 
$ cat -n input.txt
     1    -h a, -i b, -j c
     2    -h d, -i e, -j f
     3    -h g, -i h, -j i
$ 
$ awk '{sub(",","",$4); x = NR==1 ? $4 : x"|"$4} END{print x}' input.txt
b|e|h
$

tyler_durden
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

To remove the extra spaces in unix

Hi... I am quite new to Unix and would like an issue to be resolved. I have a file in the format below; 4,Reclaim,ECXTEST02,abc123,Harry Potter,5432 6730 0327 5469,0603,,MC,,1200,EUR,sho-001,,1,,,abc123,1223 I would like my output to be as follows; 4,Reclaim,ECXTEST02,abc123,Harry... (4 Replies)
Discussion started by: Sho
4 Replies

2. Shell Programming and Scripting

Remove extra spaces in a line

Hi, I need a help in deleting extra spaces in a text. I have a huge file, a part of it is :- 3 09/21/08 03:32:07 started undef mino Oracle nmx004.wwdc.numonyx.com Message Text : The Oracle session with the PID 1103 has a CPU time ... (6 Replies)
Discussion started by: vikas027
6 Replies

3. Shell Programming and Scripting

remove extra spaces between fields

Hi, I have a source file as mentioned below: I want to remove all the extra spaces between the fields. a b--------|sa df-------|3232---|3 sf sa------|afs sdf-----|43-----|33 a b c------|adfsa dsf---|23-32|23 *Here '-' idicates spaces Now, I want output as below: a b|sa df|3232|3... (7 Replies)
Discussion started by: srilaxmi
7 Replies

4. Shell Programming and Scripting

How to remove extra spaces from a string??

Hi, I have a string like this and i want to remove extra spaces that exists between the words. Here is the sentence. $string="The small DNA genome of hepadnaviruses is replicated by reverse transcription via an RNA intermediate. This RNA "pregenome" contains ... (2 Replies)
Discussion started by: vanitham
2 Replies

5. Shell Programming and Scripting

clear extra spaces and tabs in a file

Any help appreciated Thanks sample input: > (extra spaces&tabs in here) test1 (extra spaces&tabs in here) 123.123.123.123 (extra spaces&tabs in here) abc (extra spaces&tabs in here) 123 --- < (extra spaces&tabs in... (3 Replies)
Discussion started by: goofist
3 Replies

6. Shell Programming and Scripting

Remove of extra spaces from the trailing

HI, I need the help from the experts like I have created one file with text like: a b c d e f g h i j k l So my question is that i have to write the script in which like in the first sentence it will take only one space after d and remove all the extra space in the end.I dont... (8 Replies)
Discussion started by: bhanudhingra
8 Replies

7. Shell Programming and Scripting

Removing extra unwanted spaces

hi, i need to remove the extra spaces in the filed. Sample: abc~bd ~bkd123 .. 1space abc~badf ~bakdsf123 .. 2space abc~bqed ~bakuowe .. 3space output: abc~bd ~bkd123 .. 1space abc~badf~bakdsf123 .. 2space abc~bqed~bakuowe .. 3space i used the following command, (2 Replies)
Discussion started by: anshaa
2 Replies

8. Shell Programming and Scripting

Removing extra unwanted spaces

hi, i need to remove the extra spaces in the 2nd field. Sample: abc|bd |bkd123 .. 1space abc|badf |bakdsf123 .. 2space abc|bqe |bakuowe .. 3space Output: abc|bd|bkd123 abc|badf|bakdsf123 abc|bqe|bakuowe i used the following command, (9 Replies)
Discussion started by: anshaa
9 Replies

9. Shell Programming and Scripting

Have problem with extra EOLs in my CSV - need help cleaning out

Hi Everyone, Searching the forum, I came across another closed thread, that appears to be either the same problem, or very close to what I'm experiencing. Closed thread for reference is at: https://www.unix.com/shell-programming-and-scripting/241664-removing-cr-lf-till-number-fields-full.html ... (4 Replies)
Discussion started by: richardsantink
4 Replies

10. Shell Programming and Scripting

Grep string causes extra spaces

Hello, I have an xml file and my aim is to grab each line in keywords file and search the string in another file. When keyword is found in xml file,I expect the script to go to previous line in the xml file and grab the string/value between two strings. It's almost working with an error. tab... (6 Replies)
Discussion started by: baris35
6 Replies
egrep(1)																  egrep(1)

NAME
egrep - search a file for a pattern using full regular expressions SYNOPSIS
/usr/bin/egrep [-bchilnsv] [-e pattern_list] [-f file] [strings] [file...] /usr/xpg4/bin/egrep [-bchilnsvx] [-e pattern_list] [-f file] [strings] [file...] The egrep (expression grep) utility searches files for a pattern of characters and prints all lines that contain that pattern. egrep uses full regular expressions (expressions that have string values that use the full set of alphanumeric and special characters) to match the patterns. It uses a fast deterministic algorithm that sometimes needs exponential space. If no files are specified, egrep assumes standard input. Normally, each line found is copied to the standard output. The file name is printed before each line found if there is more than one input file. /usr/bin/egrep The /usr/bin/egrep utility accepts full regular expressions as described on the regexp(5) manual page, except for ( and ), ( and ), { and }, < and >, and , and with the addition of: 1. A full regular expression followed by + that matches one or more occurrences of the full regular expression. 2. A full regular expression followed by ? that matches 0 or 1 occurrences of the full regular expression. 3. Full regular expressions separated by | or by a NEWLINE that match strings that are matched by any of the expressions. 4. A full regular expression that can be enclosed in parentheses ()for grouping. Be careful using the characters $, *, [, ^, |, (, ), and in full regular expression, because they are also meaningful to the shell. It is safest to enclose the entire full regular expression in single quotes '... '. The order of precedence of operators is [], then *?+, then concatenation, then | and NEWLINE. /usr/xpg4/bin/egrep The /usr/xpg4/bin/egrep utility uses the regular expressions described in the EXTENDED REGULAR EXPRESSIONS section of the regex(5) manual page. The following options are supported for both /usr/bin/egrep and /usr/xpg4/bin/egrep: -b Precede each line by the block number on which it was found. This can be useful in locating block numbers by context (first block is 0). -c Print only a count of the lines that contain the pattern. -e pattern_list Search for a pattern_list (full regular expression that begins with a -). -f file Take the list of full regular expressions from file. -h Suppress printing of filenames when searching multiple files. -i Ignore upper/lower case distinction during comparisons. -l Print the names of files with matching lines once, separated by NEWLINEs. Does not repeat the names of files when the pat- tern is found more than once. -n Precede each line by its line number in the file (first line is 1). -s Work silently, that is, display nothing except error messages. This is useful for checking the error status. -v Print all lines except those that contain the pattern. /usr/xpg4/bin/egrep The following option is supported for /usr/xpg4/bin/egrep only: -x Consider only input lines that use all characters in the line to match an entire fixed string or regular expression to be matching lines. The following operands are supported: file A path name of a file to be searched for the patterns. If no file operands are specified, the standard input is used. /usr/bin/egrep pattern Specify a pattern to be used during the search for input. /usr/xpg4/bin/egrep pattern Specify one or more patterns to be used during the search for input. This operand is treated as if it were specified as -epattern_list. USAGE
See largefile(5) for the description of the behavior of egrep when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). See environ(5) for descriptions of the following environment variables that affect the execution of egrep: LC_COLLATE, LC_CTYPE, LC_MES- SAGES, and NLSPATH. The following exit values are returned: 0 If any matches are found. 1 If no matches are found. 2 For syntax errors or inaccessible files (even if matches were found). See attributes(5) for descriptions of the following attributes: /usr/bin/egrep +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ |CSI |Not Enabled | +-----------------------------+-----------------------------+ /usr/xpg4/bin/egrep +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWxcu4 | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ fgrep(1), grep(1), sed(1), sh(1), attributes(5), environ(5), largefile(5), regex(5), regexp(5), XPG4(5) Ideally there should be only one grep command, but there is not a single algorithm that spans a wide enough range of space-time tradeoffs. Lines are limited only by the size of the available virtual memory. /usr/xpg4/bin/egrep The /usr/xpg4/bin/egrep utility is identical to /usr/xpg4/bin/grep -E (see grep(1)). Portable applications should use /usr/xpg4/bin/grep -E. 23 May 2005 egrep(1)
All times are GMT -4. The time now is 08:39 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy