Handle special characters in awk -F


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Handle special characters in awk -F
# 1  
Old 04-27-2015
Handle special characters in awk -F

Hello Folks,

Need to bisect strings based on a subset.
Below works good.
Code:
echo /a/b/c/d | awk -F"/c/d$" '{print $1}'
/a/b

However, it goes awry with special characters.
Code:
echo /a/b/c+/d | awk -F"/c+/d$" '{print $1}'
/a/b/c+/d

Desired output:
Code:
/a/b

Escaping the special characters didn't help as well
Code:
echo "/a/b/c\+/d" | awk -F"/c\+/d$" '{print $1}'
/a/b/c\+/d

All the arguments get their values from variables.
Help on how to handle special characters here.

Last edited by Don Cragun; 04-27-2015 at 02:54 AM.. Reason: Add CODE tags.
# 2  
Old 04-27-2015
Quote:
Originally Posted by vibhor_agarwali
Hello Folks,

Need to bisect strings based on a subset.
Below works good.
Code:
echo /a/b/c/d | awk -F"/c/d$" '{print $1}'
/a/b

What is the real source of the strings you are processing? (If you are echoing constant strings into awk, it would make much more sense to remove the awk and just echo the string you want.)
Quote:
However, it goes awry with special characters.
Code:
echo /a/b/c+/d | awk -F"/c+/d$" '{print $1}'
/a/b/c+/d

Desired output:
Code:
/a/b

Escaping the special characters didn't help as well
Code:
echo "/a/b/c\+/d" | awk -F"/c\+/d$" '{print $1}'
/a/b/c\+/d

Again, if you can modify the input string as well as the ERE, why are you using awk to modify it instead of just changing the echo to begin with.

Furthermore, if you are always trying to remove a fixed string from the end of an input line, use match() to find the fixed string instead of worrying about modifying special characters in an extended regular expression.

If you really need to use an ERE to split fields, give us a clear specification of what characters might be in the input that are special in an ERE.

For the specific examples your provided you could try:
Code:
echo /a/b/c+/d | awk -F"/c[+]/d$" '{print $1}'
/a/b

and:
Code:
echo /a/b/c\+/d | awk -F"/c[\][+]/d$" '{print $1}'
/a/b

Quote:
All the arguments get their values from variables.
Help on how to handle special characters here.
Also note that using echo to feed data that might start with a minus sign or might contain a backslash character can produce radically different output depending on what shell you're using and on what system you're using when you use that shell.
# 3  
Old 04-27-2015
Thanks for the inputs.

Both the input string & ERE are dynamically generated.
It's basically a folder path which need to be bisected based on current directory.
Both folder path & current directory depend on the machine & application being used.

We were using this happily for sometime now.
Recently a guy added directories with '++' where its breaking. It cannot have extreme cases as it need to be a directory name.

Do we have other options here?
Can parse the variable argument & change the special characters as shown above if nothing else works.
# 4  
Old 04-27-2015
FWIW, this would work:
Code:
echo /a/b/c+/d | awk -F'/c\\+/d$' '{print $1}'

Single quotes and double escape of the +-character...

Or with double quotes:
Code:
echo /a/b/c+/d | awk -F"/c\\\+/d$" '{print $1}'

# 5  
Old 04-27-2015
In effect it will require parsing the regular expression of awk & escaping it.
It will be the last option as value will be contained in a variable & will be dynamic.

More options up anyone's sleeve Smilie
# 6  
Old 04-27-2015
For maximum portability assume FS to be something simple. However, since the world has gone Linux (gawk)... realize that FS can be a single character or if not, then it's a regex. So... you want:

Code:
echo '/a/b/c+/d' | awk -F'/c[+]/d$' '{print $1}'

Which returns:

Code:
/a/b

# 7  
Old 04-27-2015
Anything that doesn't require escaping the special characters will be nice.
Awk is not a must & any other utility will do.
Found awk to be giving the best results till now though.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk: split column if special characters

Hi, I've data like these: Gene1,Gene2 snp1 Gene3 snp2 Gene4 snp3 I'd like to split line if comma and then print remaining information for the respective gene. My code: awk '{ if($1 ~ /,/){ n = split($0, t, ",") (7 Replies)
Discussion started by: genome
7 Replies

2. Shell Programming and Scripting

awk conditions failing (special characters?)

This is really frustrating because I can't figure it out. I'm running a health check script. One of the items I'm checking is the amount of memory on a server. I use the free command, which outputs something like this (excerpt) Mem: 100 100 100 100 Swap: 100 100 100 100 In my debugging... (5 Replies)
Discussion started by: JustaDude
5 Replies

3. Shell Programming and Scripting

awk match shell variable that contains special characters?

How to match a shell variable that contains parenthesis (and other special characters like "!") file.txt contains: Charles Dickens Matthew Lewis (writer) name="Matthew Lewis (writer)"; awk -v na="$name" ' $0 ~ na' file.txt Ideally this would match $name in file.txt (in this... (3 Replies)
Discussion started by: Mid Ocean
3 Replies

4. UNIX for Dummies Questions & Answers

awk for removing special characters and extra commas

Hi, I have a .csv file which as empty lines with comma and some special characters in 3rd column as below. Source data 1,2,3,4,%#,6 ,,,,,, 1,2,3,4,5,6 Target Data 1,2,3,4,5,6I need to remove blank lines and special charcters I am trying to get this using the below awk awk -F","... (2 Replies)
Discussion started by: shruthidwh
2 Replies

5. Shell Programming and Scripting

Sed or awk : pattern selection based on special characters

Hello All, I am here again scratching my head on pattern selection with special characters. I have a large file having around 200 entries and i have to select a single line based on a pattern. I am able to do that: Code: cat mytest.txt | awk -F: '/myregex/ { print $2}' ... (6 Replies)
Discussion started by: usha rao
6 Replies

6. Shell Programming and Scripting

awk loop: display special characters

Hi everybody; I have a code and this fetches data from first.txt,modify it and outputs it to second.txt file. l awk 'NR>1 {print "l ./gcsw "$1" lt all lset Data="$2" Value "$3}' /home/gcsw/first.txt > /home/gcsw/second.txt this outputs as: l ./gcsw 123 lt all lset Data=456 Value 789 ... (1 Reply)
Discussion started by: gc_sw
1 Replies

7. Shell Programming and Scripting

awk print $1 escape all special characters

I'm using awk '{print $1}' and it works most of the time to print the contents of a mysql query loop, but occationally I get a field with some special character in it, is there a way to tell awk to ignore all special characters between my FS? I have >186K records, so building a list of ALL special... (6 Replies)
Discussion started by: unclecameron
6 Replies

8. Shell Programming and Scripting

awk search pattern with special characters passed from CL

I'm very new to awk and sed and I've been struggling with this for a while. I'm trying to search a file for a string with special characters and this string is a command line argument to a simple script. ./myscript "searchpattern" file #!/bin/sh awk "/$1/" $2 > dupelistfilter.txt sed... (6 Replies)
Discussion started by: cue
6 Replies

9. Shell Programming and Scripting

Handling special characters using awk

Hi all, How do I extract a value without special characters? I need to extract the value of %Used from below and if its greater than 80, need to send a notification. I am doing this right now..Its giving 17%..Is there a way to extract the value and assign it to a variable in one step? df |grep... (3 Replies)
Discussion started by: sam_78_nyc
3 Replies

10. Shell Programming and Scripting

awk/sed with special characters

i have this script that searches for a pattern. However it fails if the pattern includes some special characters. So far, it fails with the following strings: 1. -Cr 2. $Mj 3. H'412 would a sed or awk be more effective? i don't want the users to put the (\) during the search (they... (5 Replies)
Discussion started by: apalex
5 Replies
Login or Register to Ask a Question