Not able to remove special character ^I in file in Linux system


 
Thread Tools Search this Thread
The Lounge What is on Your Mind? Technology Illustrated Not able to remove special character ^I in file in Linux system
# 1  
Old 09-18-2016
Not able to remove special character ^I in file in Linux system

Hi All,

We are fetching data from database and writing a file in linux system.
Where in space between the data in one of the columns get converted into ^I which is creating an issue.

Column data.
Code:
710060     123CH
723000     CH786
813300     123RC
6HR001     045RU

There should be 5 spaces in between(710060 123CH)
this space is getting converted like below when we open the file using cat -vet filename command.

Code:
710060  ^I   123CH
723000^I     CH786
813300     123RC
6HR001   ^I045RU

To remove this special character, i used below command.
Code:
<filename LC_ALL=C sed 's/[^-~]/     /g'> filename

.

Above command is removing special char ^I but space between the data in the column is more than 5 as ^I is occurring at different positions.

could some one please guide me,how else the special char can be removed by retaining 5 spaces in between the data 710060 123CH



Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 09-18-2016 at 01:39 PM.. Reason: Added CODE tags.
# 2  
Old 09-18-2016
As the <TAB> (0x09) character can be composed with the two keys <CTRL> and I , it sometimes is displayed as ^I (read: control-I). It belongs to the [[:blank:]] character class and is frequently used as a field/column separator in DB output, when its about aligning columns, regardless of how long the previous one was, and the position where it terminated. For this reason it is quite pointless to replace ^I with 5 spaces as it could be any value between 1 and (usually, default) 8 characters.

It would be suprising if a DB system would mix <TAB> and space characters for column separation, or use <TAB>s inconsistently in different positions.

Try finding the original query/script and modify that.
# 3  
Old 09-18-2016
Hi Rudic,

Thanks for your suggestion.

But in database side i could see that there is no issue. columns are separated by | symbol.
The issue im facing is for only one column where in 5 spaces are required in between the data 71000 CH678.

Is there any other way to remove this ^I
# 4  
Old 09-18-2016
If you use the following sed command, replacing each occurrence of <space> with a single space character and replacing <tab> with a single tab character:
Code:
sed 's/<space>*<tab>[[:space:]]*/<space><space><space><space><space>/g' input_file > new_file

it will replace all sequences of zero or more spaces followed by a tab character followed by zero of more spaces and tabs in the file named input_file to exactly five space characters in the output file named new_file.

If someone else wants to try this on a Solaris/SunOS system, change sed to /usr/xpg4/bin/sed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove a word that ends with special character "!"

Hi all. I want to use sed to remove a word that ends with "!" in the first page of a file. The word I want to remove is: "DNA!". I have search for an answer and nothing of what I found helped me. ~faizlo (2 Replies)
Discussion started by: faizlo
2 Replies

2. Shell Programming and Scripting

Remove word after special character "/"

Hi There, I have one requirement to remove word after character "/". Input file is 2017-07-12|02|user1l|domain1/userl|0 2017-07-12|02|user2|domain1/user2|5 2017-07-12|02|user3|domain2/user3|0 2017-07-12|02|user4|domain1/user4|432 and require OP file is ... (2 Replies)
Discussion started by: anshu ranjan
2 Replies

3. Shell Programming and Scripting

Remove blank space and insert special character

Hi Folks, I have a huge data of the below format abc #apple 1200 06/23 ghj #orange 1500 06/27 uyt #banana 2300 05/13 efg #vegetable 0700 04/16 After first 3 letters, i have 9 spaces and after fruit there are no specific fixed space, but it varies... (4 Replies)
Discussion started by: jayadanabalan
4 Replies

4. Shell Programming and Scripting

Remove some special ascii character

Hello I have this special caracter after retreving rows from sql server: "....spasses: • Entrem al valort 6050108002811 • El donem..." I would like a sed command to remove it..or just know it's ascii code in order to replace it into my sql sentence.. Hope some one knows how to do that.... (7 Replies)
Discussion started by: ldiaz2106
7 Replies

5. Linux

Linux script to remove a character in a file based on position.

Greetings, We have a requirement where we need to loop in a fixed width file in linux and remove a character based on a position for every record. It would highly appreciate if someone can help to automate this. Appreciate your time and help! Regards (3 Replies)
Discussion started by: mailme0205
3 Replies

6. Shell Programming and Scripting

Remove special character ($) from file names

Hello I've searched here and on the 'net for examples of a script or command line function that will remove the $ character from all file names only that can be done within the directory that contains the file names - which are all html files. ie, I have a directory that contains html files... (6 Replies)
Discussion started by: competitions
6 Replies

7. Shell Programming and Scripting

remove special character from a specific column

Hello , i have a text file like this : A123 c12AB c32DD aaaa B123 23DS 12QW bbbb C123 2GR 3RG cccccc i want to remove the numbers from second and third column only. i tried this : perl -pe 's///g' file.txt > newfile.txt but it will remove the number from... (7 Replies)
Discussion started by: shelladdict
7 Replies

8. Shell Programming and Scripting

remove special character from a textfile

Can any one plse help me writing shell script to removing some special character pattern (like / > -------, / > / > ------- etc....as shown below) from the text file ASAP. / > ------- <tag-normalization tag-name="EXECSERVPRODUCT" read-only="false" part="body"> ... (3 Replies)
Discussion started by: bkc
3 Replies

9. Shell Programming and Scripting

Remove box like special character from end of string

Hi All, How to remove a box like special character which appears at the end of a string/line/record. I have no clue what this box like special character is. It is transparent square like box. This appears in a .DAT file at the end of header. I'm to compare a value in header with a parameter.... (16 Replies)
Discussion started by: Qwerty123
16 Replies

10. UNIX for Dummies Questions & Answers

Special character in my file

I have a special character in my file. It displays as a '#' sign but when I do this command I do not find the line. fgrep 'G#ant' file1 I want to replace the special character with another value but I need to know what character it really is. Any ideas on how to replace this '#' value with... (3 Replies)
Discussion started by: Ryan2786
3 Replies
Login or Register to Ask a Question