Remove box like special character from end of string


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove box like special character from end of string
# 1  
Old 01-30-2009
Error Remove box like special character from end of string

Hi All,

How to remove a box like special character which appears at the end of a string/line/record. I have no clue what this box like special character is. It is transparent square like box. This appears in a .DAT file at the end of header.

I'm to compare a value in header with a parameter. Even when the both are same, due to this special character, the script returns as non matching.

Example of the header:

H20090130QWERTY ASDFGH.DAT[]

As I can't exactly make that box appear here I'm typing square braces in bold. That's the place where the box like special character appears. Is it a new line?

I tried doing
Var=`echo $Var1 | tr -d "\n"`
but no use.

Any help is highly appreciated.
# 2  
Old 01-30-2009
Try to remove the last character with sed:

Code:
Var=`echo $Var1 | sed 's/.$//'`

To remove more characters you can increase the dots.

Regards
# 3  
Old 01-30-2009
was this file transferred from dos to unix?

anyhoo.. . . typically, i'll remove non-ascii chars like so:

Code:
sed -e 's/[^ -~]//g' file_in > file_out

# 4  
Old 02-02-2009
Quote:
Originally Posted by Franklin52
Try to remove the last character with sed:

Code:
Var=`echo $Var1 | sed 's/.$//'`

To remove more characters you can increase the dots.

Regards

Hi,
But this code is removing the last 'T' from ASDFG.DAT
Is this piece of code not able to pick the box like character and remove it?
Please help.
# 5  
Old 02-02-2009
Quote:
Originally Posted by quirkasaurus
was this file transferred from dos to unix?

anyhoo.. . . typically, i'll remove non-ascii chars like so:

Code:
sed -e 's/[^ -~]//g' file_in > file_out


Hi,
My file is very huge sometimes 1GB, instead of removing non-ascii characters from the file, can they be removed only from this string i.e, $var?
Please suggest.
# 6  
Old 02-02-2009
Quote:
Originally Posted by Qwerty123
Hi,
My file is very huge sometimes 1GB, instead of removing non-ascii characters from the file, can they be removed only from this string i.e, $var?
Please suggest.
Unfortunately, you really can't avoid the cost of fixing this file.
it's going to take some time and disk space.

What's the next step, loading it into a database?

If that's the case -- you will be able to do the read,
weird character removal and insert-into-the-database
all in perl -- that's be fairly cost effective.

If you're interested in that solution, lemme know.
# 7  
Old 02-02-2009
Quote:
Originally Posted by quirkasaurus
Unfortunately, you really can't avoid the cost of fixing this file.
it's going to take some time and disk space.

What's the next step, loading it into a database?

If that's the case -- you will be able to do the read,
weird character removal and insert-into-the-database
all in perl -- that's be fairly cost effective.

If you're interested in that solution, lemme know.
Hi,

My requirement is that I store this string in suppose say $string...and compare it with a parameter that is input say $param ...
like

if [ $string = $param ]
then
echo 'True'
else
echo 'False'
fi

This $string is part of header (i.e another big string) which can be treated as a record (line) and this $string appears at the end of line. I fetch it based on positions...for ex: 30 to 60. In case the $string is of length 10 only, 11th character i.e, at position 41 there'll be this box like character (since I'm fetching 30 to 60 into $string). This character doesn't get trimmed. And since this box like character is present my comparision returns 'False' even when it's 'True'.

I have tried things like

string=`echo $string|tr -d '\n'`
and \t, \r, \222,\221---Ascii equivalents...and all that and then thought its not Ascii at all.

Still I couldn't solve this. Please helpSmilie
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove blank space and insert special character

Hi Folks, I have a huge data of the below format abc #apple 1200 06/23 ghj #orange 1500 06/27 uyt #banana 2300 05/13 efg #vegetable 0700 04/16 After first 3 letters, i have 9 spaces and after fruit there are no specific fixed space, but it varies... (4 Replies)
Discussion started by: jayadanabalan
4 Replies

2. Shell Programming and Scripting

Remove lines between the start string and end string including start and end string Python

Hi, I am trying to remove lines once a string is found till another string is found including the start string and end string. I want to basically grab all the lines starting with color (closing bracket). PS: The line after the closing bracket for color could be anything (currently 'more').... (1 Reply)
Discussion started by: Dabheeruz
1 Replies

3. Shell Programming and Scripting

Remove some special ascii character

Hello I have this special caracter after retreving rows from sql server: "....spasses: Entrem al valort 6050108002811 El donem..." I would like a sed command to remove it..or just know it's ascii code in order to replace it into my sql sentence.. Hope some one knows how to do that.... (7 Replies)
Discussion started by: ldiaz2106
7 Replies

4. Shell Programming and Scripting

Remove 3rd character from the end of a random-length string

Hi, I hope someone can share there scripting fu on my problem, I would like to delete the 3rd character from a random length of string starting from the end Example Output Hope you can help me.. Thanks in advance.. (3 Replies)
Discussion started by: jao_madn
3 Replies

5. Shell Programming and Scripting

Remove special character ($) from file names

Hello I've searched here and on the 'net for examples of a script or command line function that will remove the $ character from all file names only that can be done within the directory that contains the file names - which are all html files. ie, I have a directory that contains html files... (6 Replies)
Discussion started by: competitions
6 Replies

6. Shell Programming and Scripting

remove special character from a specific column

Hello , i have a text file like this : A123 c12AB c32DD aaaa B123 23DS 12QW bbbb C123 2GR 3RG cccccc i want to remove the numbers from second and third column only. i tried this : perl -pe 's///g' file.txt > newfile.txt but it will remove the number from... (7 Replies)
Discussion started by: shelladdict
7 Replies

7. Shell Programming and Scripting

Remove special char from end of the file

Hi I am working on a bash script and would know how to use cut or sed to remove (F/.M/d h) from a text file. Before 1 text to save (F/.M/d h) after 1 text to save Thanks in advance (5 Replies)
Discussion started by: pelle
5 Replies

8. Shell Programming and Scripting

remove special character from a textfile

Can any one plse help me writing shell script to removing some special character pattern (like / > -------, / > / > ------- etc....as shown below) from the text file ASAP. / > ------- <tag-normalization tag-name="EXECSERVPRODUCT" read-only="false" part="body"> ... (3 Replies)
Discussion started by: bkc
3 Replies

9. Shell Programming and Scripting

Adding a special character at the end of the line

I used following to add * at the end of the line in file1. It adds * at the end but has a space before it for some lines but some other lines it adds exactly after the last character. How do I take out the space ? sed 's/$/*/' file1 > file2 example: contents of file1 : ... (2 Replies)
Discussion started by: pitagi
2 Replies
Login or Register to Ask a Question

Featured Tech Videos