Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Cleanup Weird character in Unix

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Cleanup Weird character in Unix
# 1  
Old 02-26-2010
Cleanup Weird character in Unix
Hi,

I have a pipe delimited file and I am running into an issue where a field is having some weird character and I am not able to clean it up. It is 2nd field and the weird character is M- . See below.


Code:
cat -v test.dat
02169| M- PATRICKM- MCALEER |Y
01318| M- LARRYM- PETERSON |Y
30319| M- PRISCILLAM- JOHNSON |Y
10065| DAUMM- M- M- M-  |Y

Please advise on how I will be able to clean it up.

Thanks.
# 2  
Old 02-26-2010
Somehow that file got padded with 'M- ' wherever a space happened..
Code:
sed 's/M- //g' < infile > outfile

# 3  
Old 02-26-2010
I already tried sed earlier but the character is not a M- . It is some special character that is not getting cleaned up.

See below.

Code:
grep "PRISCILLA" test.dat | sed 's/M- //g' | cat -v

 
YTB %M- PRISCILLAM- JOHNSON

Thanks..
# 4  
Old 02-26-2010
Please post a few lines of the output from this command. It will tell us whether this is a text file from a M$ platform which needed conversion to unix text file format. The command does not change the file, it just makes M$ line terminators visible.


Code:
sed -n l test.dat

# 5  
Old 02-26-2010
Code:
od file | head -5

output would be better, along with
Code:
locale

I do not think there is an 'M-' character in any 8bit locale. You may have wide characters your locale screws up somehow.
# 6  
Old 02-26-2010
M- in the context of "cat -v" just means unprintable character. Once we know the octal code for the character it can be identified or changed to something else.
Ignoring M$ line terminator characters for the minute (my mistake) it could well be one or more different foreign accented characters.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

UNIX script for cleanup

Hello, I need some help from unix guru's here..I am looking for some advanced level script to cleanup the directories and files from specific directories under a file system.. The folders are created under /opt/modules And under modules, there are multiple subfolders with the application... (6 Replies)
Discussion started by: mb525
6 Replies

2. SCO

SCO UNIX ftp - weird kind of working

hi, i am working on sco unix 5.07, i have 10 individual servers networked locally in which sco os is installed. one of them is not transferring the file to other servers but accepting all files. means Server (A) can accept the file through FTP from any server but not transferring FTP its own... (9 Replies)
Discussion started by: aafflatoon
9 Replies

3. Red Hat

How to Cleanup Multipathing

I have a server running redhat 5.5 and it has one SAN device presented to it as LUN9. How can I clean up the remaining entries. I cannot afford to interupt the service. Please assist. # multipath -l mpath0 (36000097000019260298953666633436) dm-11 EMC,SYMMETRIX \_ round-robin 0 \_ 2:0:0:9 ... (2 Replies)
Discussion started by: Tirmazi
2 Replies

4. Shell Programming and Scripting

Cleanup between parenthesis

Hi, I am trying to clean up data between parenthesis () in a file. See example below.... Input File : (New York) Chicago (London) New York (Chicago) London New York Chicago (London) (New York) (Chicago) (London) New York (Chicago) ... (3 Replies)
Discussion started by: msalam65
3 Replies

5. Solaris

/home cleanup

Hi All, I have this script for linux on cleaning up orphaned folder. But I need to use this on solaris 8/9/10 for user in $(ls | grep -v lost+found) ; do id $user >/dev/null 2>&1 if ] then ls -ld $user grep $user /etc/passwd fi done Can someone please convert this script? ... (1 Reply)
Discussion started by: itik
1 Replies

6. UNIX for Dummies Questions & Answers

What is the default temp dir for UNIX AIX 5.3 and cleanup guide.

Hi, I'm new with UNIX. In fact I'm just an Oracle Apps programmer runs and maintaining the system in UNIX platform. I've got this question, what is the UNIX AIX 5.3 default temp directory. Is it /tmp or /var/tmp. Additional to that I was noted that OS also created temp/swap files in this... (1 Reply)
Discussion started by: Alikuching
1 Replies

7. UNIX for Dummies Questions & Answers

Weird character in between echo function

Hi All, Appreciate if anyone can help. I've a script where it does echo function like this while do FILE_ARG="cu0${w}_${FILE}_${DT}.av" ORACLE_ERROR=`grep "ORA-" ${FILE_ARG}` if ]; then Func_Log_Writer "Fail! ${FILE_ARG}\n" Func_Log_Writer "Error message:... (2 Replies)
Discussion started by: agathaeleanor
2 Replies

8. Shell Programming and Scripting

Help with cleanup

I am trying to add a unique string to a variable to prevent some name space collisions. DATAFILE=/u001/app/unica/affinium644/campaign/partitions/limited/tmp/ebf9aaah.t~# DATETIME=`date +%Y%m%d_%H%M%S` echo $DATAFILE > tmpnme.txt sed 's_/_ _g' tmpnme.txt > tmpnme2.txt DATA=$(cat tmpnme2.txt)... (2 Replies)
Discussion started by: whdr02
2 Replies

9. AIX

Login ID cleanup

Hello I have many old IDs on my AIX and would like to know the simplest way of knowing the last time an ID was used. I am familiar with the "last" command. Thanks for any info :) (1 Reply)
Discussion started by: MILLERJ62
1 Replies

10. UNIX for Dummies Questions & Answers

sendmail cleanup

What is the correct procedures to clean up /var/spool/mqueue? Any help appreciated. This directory gets really clogged up at times. :( :( (1 Reply)
Discussion started by: thomi39
1 Replies
Login or Register to Ask a Question