Visit Our UNIX and Linux User Community


issue with ^B characters in the file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting issue with ^B characters in the file
# 1  
Old 08-11-2009
issue with ^B characters in the file

Hello-

I have a csv file with japan email ids as the last column(pipe delimited file).
One of the email ids has some special characters which are not displayed on my UNIX box, however when ftped to local(Windows machine) I can see the email id terminated with a square box instead of pipe.

How can I remove these records from the csv file on the unix box?

Sample Data:
123456|23456|432|21|12|ABC-BCE| |N|email@email.ne

Thanks
Raghs....
# 2  
Old 08-11-2009
You can try something like:

Code:
sed 's/..$//' file > newfile

Regards
# 3  
Old 08-11-2009
We are building a collection of unix and MSDOS methods for identifying non-display characters on another post. Once you know what is in that byte it is easier to make a match with sed, grep or whatever.

https://www.unix.com/shell-programmin...scii-code.html

Not clear whether you wanted to lose the whole line containing the unusual character, or just the unusual character.

Beware that text mode FTP will strip the high order bit such that the character can change between unix and MSDOS.

Previous Thread | Next Thread
Test Your Knowledge in Computers #92
Difficulty: Easy
The Unix ls command can only lists files in the current directory.
True or False?

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Outputting characters after a given string and reporting the characters in the row below --sed

I have this fastq file: @M04961:22:000000000-B5VGJ:1:1101:9280:7106 1:N:0:86 GGGGGGGGGGGGCATGAAAACATACAAACCGTCTTTCCAGAAATTGTTCCAAGTATCGGCAACAGCTTTATCAATACCATGAAAAATATCAACCACACCA +test-1 GGGGGGGGGGGGGGGGGCCGGGGGFF,EDFFGEDFG,@DGGCGGEGGG7DCGGGF68CGFFFGGGG@CGDGFFDFEFEFF:30CGAFFDFEFF8CAF;;8... (10 Replies)
Discussion started by: Xterra
10 Replies

2. Linux

File conversion and removing special characters from a file in Linux

I have a .CSV file when I check for the special characters in the file using the command cat -vet filename.csv, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below file filename.csv I get the output as I have a... (2 Replies)
Discussion started by: dhruuv369
2 Replies

3. Shell Programming and Scripting

awk script to count characters in file 1 in file 2

I need a scripting AWK to compare 2 files. file 1 and 2 are list of keywords 1 is a b c d 2 is aa aaa b bb ccc d I want the AWK script to give us the number of times every keyword in file 1 occurs in file 2. output should be a 2 (7 Replies)
Discussion started by: anhtt
7 Replies

4. Shell Programming and Scripting

sed special characters issue

Hi I'm trying to replace string1 by string2 in file homepage.htm as follows but is not working. Please Help: sedsed -i "s@'://your server name:port/test/owa'@'://11.22.33.44:5555/pls/SAMPLE'@g" homepage.htm Where, String1 ://your server name:port/test/owa String2... (3 Replies)
Discussion started by: koazter
3 Replies

5. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

6. Shell Programming and Scripting

Need assistance with a file issue and a terminal issue

Hello everyone, I'm in need of some assistance. I'm currently enrolled in an introductory UNIX shell programming course and, well halfway through the semester, we are receiving our first actual assignment. I've somewhat realized now that I've fallen behind, and I'm working to get caught up, but for... (1 Reply)
Discussion started by: MrMagoo22
1 Replies

7. Shell Programming and Scripting

Unix Script file to Append Characters before rows in file.

Hi Experts, I am working on HP-UX. I am new to shell scripting. I would like to have a shell script which will prefix: 1. "H|" before first row of my file and, 2. "T" for all other rows of the file. For Example - File before running the script 20100430|4123451810|218.50|TC 20100430 ... (4 Replies)
Discussion started by: phani333
4 Replies

8. Solaris

Issue with image file name contaning Japaneese characters

Hi, i have created some image files with name conatining Japaneese characters "Banner Image 2 _ with Japan char_こくさいかんけい.jpg", but when i see this file in unix file system (through WinSCP) its showing lilke "Banner Image 2 _ with Japan char_こくさいかんけい.jpg". Becoz of this the image is not... (1 Reply)
Discussion started by: chandran.m
1 Replies

9. UNIX for Dummies Questions & Answers

Rename file based on first 3 characters of data in file

I'm looking to determine if I can use a grep command to read file and rename the file based on the first 3 characters of the data in the file. An example is: Read FileA If the first 3 positions of the data in the file are "ITP", then rename the file as FileA_ITP, else if the first 3... (3 Replies)
Discussion started by: jchappel
3 Replies

Featured Tech Videos