Removing non printing characters from a csv file


 
Thread Tools Search this Thread
Operating Systems Linux Removing non printing characters from a csv file
# 1  
Old 03-18-2010
Removing non printing characters from a csv file

Hi,

I have an csv file and there are some non printable characters(extended ascii) so I am trying to create a clean copy of the csv file . I am using
this command:
Code:
tr -cd "[:print:]" < /opt/informatica/PowerCenter8.6.0/server/infa_shared/SrcFiles/ThirdParty/locations.csv  > 
/opt/informatica/PowerCenter8.6.0/server/infa_shared/SrcFiles/locations.csv ...

or

Code:
tr -cd '\11\12\40-\176' < /opt/informatica/PowerCenter8.6.0/server/infa_shared/SrcFiles/ThirdParty/locations.csv 
 > /opt/informatica/PowerCenter8.6.0/server/infa_shared/SrcFiles/locations2.csv

But this command creates just empty output file.
What do you think?

Last edited by joeyg; 03-18-2010 at 04:32 PM.. Reason: please use codetags
# 2  
Old 03-18-2010
tr [:cntrl:]

have you tried the [:cntrl:] option?
if you
Code:
man tr

you will probably see a listing of all the special sets, like
Code:
[:print:] & [:alpha:]

# 3  
Old 03-18-2010
Actually I am using informatica command transformation. I am just adding this shell overthere
# 4  
Old 03-18-2010
Tools

Not sure of your 2nd command, but your first should send output to a different filename. Unix commands act funny when the input and output filenames are the same.
# 5  
Old 03-18-2010
You might want to pipe it to a pager like more or less to see what it's doing, but definitely use both a distinct output file and note (unless my eyes deceive me...) your tr command seems to be deleting the printing chars you're looking to keep. Try to reverse the set using the full octal values (ie, \000-\011, \013-\014, etc)...

---------- Post updated at 16:52 ---------- Previous update was at 16:50 ----------

Might be useful to pipe it to od -c too, as it would indicate explicitly the chars you're working with...

---------- Post updated at 16:52 ---------- Previous update was at 16:52 ----------

Might be useful to pipe it to od -c too, as it would indicate explicitly the chars you're working with...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Removing commas from CSV file

Hi I'm creating a sh script to generate a csv file. The CSV contains the values from a sql table. The content looks this: a,b,c,c2,c3,,,,,,,,,,,d,e I have some code that can separate the fields using the comma as delimiter, but some values actually contain commas, such as... (2 Replies)
Discussion started by: preema
2 Replies

2. Shell Programming and Scripting

Remove non printing characters from file

How do I remove the printer escape sequence, the first 5 characters, that occurs on every 33rd line in a file, see hex dump of line 1. 0000 1e 00 00 00 00 0a 0a 0a 20 0a 20 20 20 20 20 20 .... 0010 20 20 20 20 20 20 20 20 20 20 0a 42 49 4c 4c 20 Thanks, (2 Replies)
Discussion started by: jgt
2 Replies

3. Shell Programming and Scripting

Removing last and first characters in a file

bash-3.00$ cat temp.txt ./a/REA01/ces1/apps/ces_ces1_init3_aa.ear/ces.war/WEB-INF/classes/reds/common/environment.properties ./a/REA01/ces1/apps/ces_ces1_init3_aa.ear/commonproperties/hi/HostIntegration.properties... (9 Replies)
Discussion started by: bhas85
9 Replies

4. Shell Programming and Scripting

Removing these non-ASCII characters from a file

Hi, I have many text files which contain some non-ASCII characters. I attach the screenshots of one of the files for people to have a look at. The issue is even after issuing the non-ASCII removal commands one of the characters does not go away. The character that goes away is the black one with a... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

5. Shell Programming and Scripting

Removing \n from .csv file

Hi, I have a requirement like my .csv file is generating from a db2 table using export command like below: file format: ----------- 2011 4 0 0 N S C C "BHPC BHPC" 0 0 0 2011 5 0 0 N S C C "BHPC BHPC" 0 0 0 here BHPC is having new line character and because this when i am trying... (4 Replies)
Discussion started by: RRVARMA
4 Replies

6. Shell Programming and Scripting

removing file with bad characters

I have the following files in the same directory but if you look at the od output you can see one of the files has and "\n" as part of the file name. Is there a way I can only remove the file with the "\n" as part of the file name without affecting the other file. I was thinking about... (4 Replies)
Discussion started by: BeefStu
4 Replies

7. Shell Programming and Scripting

Removing lines of a .csv file

Hello, Does anyone have a one-liner to remove lines of a csv file if the value in a specific column is zero? For example, I have this file, 12345,COM,5,0,N,29.95,Y 12345,MOM,1,0,N,29.95,Y 12345,COM,4,0,N,9.99,Y 12345,MOM,0,2,N,9.99,Y 12345,REN,0,1,N,9.99,Y and I want to remove lines... (4 Replies)
Discussion started by: palex
4 Replies

8. Shell Programming and Scripting

Removing ^M characters from a file

Hi, I want to removing ^M characters from a file and combine the line with the next line. ex: issue i have: ABC^M^M DEF solution i need: ABCDEF I found that you by using the following command you can remove new line characters. tr -d '\r' < infile.csv > outfile.csv still... (10 Replies)
Discussion started by: mwrg
10 Replies

9. Shell Programming and Scripting

removing unwanted characters from a file

i have a file like this 1111_2222#$#$dudgfdk 11111111_343434#$#$334 1111_22222#43445667 i want to remove all those charachetrs from # how can i do this Thank in advance Saravanan (4 Replies)
Discussion started by: saravanan71184
4 Replies

10. Shell Programming and Scripting

Removing certain characters in a file

Hi I have a file that has semicolons in it (;) is there a way to just remove these in the file. Example name: Joe Smith; group: Group1; name: Mary White; group: Group2; (2 Replies)
Discussion started by: bombcan
2 Replies
Login or Register to Ask a Question