How can I get rid of the ` character from input file?

gugs · #1 (**permalink**) 08-07-2008

A sed command works most of the time however it fails sometimes.

I put each line (record) I read of a file through the following command
data=$(cat file | sed 's/[^a-zA-Z0-9+_:-]//g' | sed 's|.*ex:Msg$.*$ex:Msg.*|\1|' )

When I run the script I get a message that states that there is an invalid format character.

The list of invalid chracters I am getting is as below, is there a way to solve this issue?. How can I get rid of the ` character from the input data?

`w'
`m'
`k'
``'
`;'
`O'
`m'
`H'
`^'
` '
`:'
`v'
`S'
`k'
`j'
`)'
`!'
`['
`m'
`@'
`T'

joeyg · #2 (**permalink**) 08-07-2008

Since the character is the first in each line...

Code:

cat file | cut -c2-

gugs · #3 (**permalink**) 08-07-2008

I am afraid its not the first character, I have specified the message that is displayed on the screen when I run my script which is of the format ./records.sh invalid chracters....etc

shamrock · #4 (**permalink**) 08-07-2008

Can you provide a sample of the input file

cfajohnson · #5 (**permalink**) 08-07-2008

To answer the question in the subject line:
Code:
tr -d '`' < FILE

Quote:

Originally Posted by gugs

A sed command works most of the time however it fails sometimes.

Under what conditions does it fail?

Quote:

I put each line (record) I read of a file through the following command

I hope you don't really run the entire file through two instances of sed (not to mention cat) for every line in the file.

Quote:

You are using three external commands where you only need one:
Code:
data=$( sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg$.*$ex:Msg.*|\1|' file )

Quote:

When I run the script I get a message that states that there is an invalid format character.

Please post the exact message you get. (Cut and paste it, don't retype it.)

Quote:

The list of invalid chracters I am getting is as below, is there a way to solve this issue?. How can I get rid of the ` character from the input data?

Why do you think the problem is the ` character?

The invalid characters are those between the quotes.

Quote:

`w'
`m'
[snip]
`@'
`T'

gugs · #6 (**permalink**) 08-08-2008

cfajohnson, unfortunately I cannot post a copy of the data because I am not allowed, confidential info. The only reason I know that these chars are causing a problem is because I get an on line message: ./counter.sh line 5: printf: `w: invalid format and this is repeated for characters that I have mentioned in my initial posting. The contents of counter.sh is:

counter=1
while read line
do
# Test the file
printf "$line" > temp$counter
pref=$(sed <$temp$counter -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg$.*$ex:Msg.*|\1|')
printf"
let counter=counter+1
done < temp01

Also you mention that I could use only one command instead of three for extarcting the data. Could you please specify? I am new to scripting so any help would be greatly appreciated.

Code:
data=$( sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg$.*$ex:Msg.*|\1|' file )

cfajohnson · #7 (**permalink**) 08-08-2008

Quote:

Originally Posted by gugs

cfajohnson, unfortunately I cannot post a copy of the data because I am not allowed, confidential info. The only reason I know that these chars are causing a problem is because I get an on line message: ./counter.sh line 5: printf: `w: invalid format

As the message says, you are using an invalid format character with printf.

The syntaxt for printf is:
Code:
printf FORMAT_STRING ARG ...
What you need is:
Code:
printf "%s\n" "$line"

Quote:

and this is repeated for characters that I have mentioned in my initial posting. The contents of counter.sh is:

Please put code inside code tags.

Quote:

Code:

 
counter=1
while read line
do
  # Test the file 
  printf "$line" > temp$counter
  pref=$(sed <$temp$counter -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|')
printf"

Don't use sed for every line in a file; it will be extremely slow.

Use sed on the entire file and work on its output.

Quote:

let counter=counter+1

Use the standard form for arithmetic:
Code:
counter=$(( $counter + 1 ))

Quote:

Code:

 
done < temp01

Also you mention that I could use only one command instead of three for extarcting the data. Could you please specify? I am new to scripting so any help would be greatly appreciated.

Code:

data=$( sed -e 's/[^a-zA-Z0-9+_:-]//g' -e 's|.*ex:Msg\(.*\)ex:Msg.*|\1|' file )

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
Can I read a file character by character?	murtaza	Shell Programming and Scripting	4	04-27-2009 05:51 AM
Using input file to filter data from another file	tumblez	UNIX for Dummies Questions & Answers	1	07-21-2008 04:21 PM
check for a particular character inside a file and substitute with a given character?	karthikprasathk	AIX	1	07-01-2008 03:29 AM
Can i read a file character by character	karnan	Shell Programming and Scripting	6	05-19-2008 02:22 AM
how to get input from file	ajaya	Shell Programming and Scripting	1	04-05-2006 02:02 PM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by