Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: File conversion and removing special characters from a file in Linux
Operating Systems Linux File conversion and removing special characters from a file in Linux Post 302863485 by dhruuv369 on Monday 14th of October 2013 11:22:23 AM
Old 10-14-2013
File conversion and removing special characters from a file in Linux
I have a .CSV file when I check for the special characters in the file using the command
Code:
cat -vet filename.csv

, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below
Code:
file filename.csv

I get the output as
Quote:
filename.csv: Little-endian UTF-16 Unicode English character data, with very long lines, with CRLF, CR line terminators
I have a script to remove the control M (^M) from the file, whose output returns me an error saying : cannot execute binary file.

I know that ^I represent a tab. I also have a script to convert ^I to comma delimited file but Can anyone help me format the file with respect to the error and also ^@.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing special characters in file

I have file special.txt with the following data. <header info> 123$ty5%98&0asd 1@356fgbv78 09*&^5jkns43( ...........some more rows. In my output file, I want to eliminate all the special characters in my file and I want all other data. need some help. (6 Replies)
Discussion started by: srivsn
6 Replies

2. Shell Programming and Scripting

removing special characters @ EOL

How to remove special chracters @ END OF EACH LINE in a file file1.txt: 0003073413^M 0003073351^M 0003073379^M 0003282724^M 0003323334^M 0003217159^M 0003102760^M 0002228911^M I used the below command but it is not working ? perl -pi -e 's/^M\/g' file1.txt (6 Replies)
Discussion started by: ali560045
6 Replies

3. AIX

Removing a filename which has special characters passed from a pipe with xargs

Hi, On AIX 5200-07-00 I have a find command as following to delete files from a certain location that are more than 7 days old. I am being told that I cannot use -exec option to delete files from these directories. Having said that I am more curious to know how this can be done. an sample... (3 Replies)
Discussion started by: jerardfjay
3 Replies

4. Solaris

removing special characters, white spaces from a field in a file

what my code is doing, it is executing a sql file and the resullset of the query is getting stored in the text file in a fixed format. for that fixed format i have used the following code:: Code: awk -F":"... (2 Replies)
Discussion started by: priyanka3006
2 Replies

5. Shell Programming and Scripting

Removing special characters

Dear Friends, I want to remove text between two patters. Problem is, it has random special characters like \ / | * ` ~ ! $ etc. These random special characters has no fixed length. But these special characters are appearing between a fixed pattern e.g. DM&^%#|#!\/?CT Expected output... (14 Replies)
Discussion started by: anushree.a
14 Replies

6. Shell Programming and Scripting

Removing Special Character from File.

Hi, My file has this special character "^M" I would like to remove this characters. eg: abc,abc,^M i tried using sed but doesnt work. i used octal dump command to see special character it returns following: 015 \r Appreciate your reply. (6 Replies)
Discussion started by: pinnacle
6 Replies

7. UNIX for Dummies Questions & Answers

awk for removing special characters and extra commas

Hi, I have a .csv file which as empty lines with comma and some special characters in 3rd column as below. Source data 1,2,3,4,%#,6 ,,,,,, 1,2,3,4,5,6 Target Data 1,2,3,4,5,6I need to remove blank lines and special charcters I am trying to get this using the below awk awk -F","... (2 Replies)
Discussion started by: shruthidwh
2 Replies

8. Shell Programming and Scripting

Removing special characters - Control M

I have developed a small script to remove the Control M characters that get embedded when we move any file from Windows to Unix. For some reason, its not working in all scenarios. Some times I still see the ^M not being removed. Is there anything missing in the script: cd ${inputDir}... (7 Replies)
Discussion started by: vskr72
7 Replies

9. Shell Programming and Scripting

Removing blank/white spaces and special characters

Hello All , 1. I am trying to do a task where I need to remove Blank spaces from my file , I am usingawk '{$1=$1}{print}' file>file1Input :- ;05/12/1990 ;31/03/2014 ; Output:- ;05/12/1990 ;31/03/2014 ;This command is not removing all spaces from... (6 Replies)
Discussion started by: himanshu sood
6 Replies

10. UNIX for Beginners Questions & Answers

Replacing string/special characters using a 'conversion' table

Hi, Does anyone know if there is a script or program available out there that uses a conversion table to replace special characters from a file? I am trying to remove some special characters from a file but there are several unprintable/control characters that some I need to remove but some I... (2 Replies)
Discussion started by: newbie_01
2 Replies

LEARN ABOUT CENTOS

paps

PAPS(1) 						      General Commands Manual							   PAPS(1)

NAME

       paps - UTF-8 to PostScript converter using Pango

SYNOPSIS

       paps [options] files...

DESCRIPTION

       paps reads a UTF-8 encoded file and generates a PostScript language rendering of the file. The rendering is done by creating outline curves
       through the pango ft2 backend.

OPTIONS

       These programs follow the usual GNU command line syntax, with long options starting with  two  dashes  (`-').   A  summary  of  options	is
       included below.

       --landscape
	      Landscape output. Default is portrait.

       --columns=cl
	      Number of columns output. Default is 1.

       --font=desc
	      Set the font description. Default is Monospace 12.

       --rtl  Do rtl layout.

       --paper ps
	      Choose paper size. Known paper sizes are legal, letter, a4. Default is A4.

       --bottom-margin=bm
	      Set bottom margin in postscript points (1/72 inch). Default is 36.

       --top-margin=tm
	      Set top margin. Default is 36.

       --left-margin=lm
	      Set left margin. Default is 36.

       --right-margin=rm
	      Set right margin. Default is 36.

       --help Show summary of options.

       --header
	      Draw page header for each page.

       --markup
	      Interpret the text as pango markup.

       --encoding=ENCODING
	      Assume the documentation encoding is ENCODING.

       --lpi  Set the lines per inch. This determines the line spacing.

       --cpi  Set the characters per inch. This is an alternative method of specifying the font size.

       --stretch-chars
	      Indicates  that  characters  should be stretched in the y-direction to fill up their vertical space. This is similar to the texttops
	      behaviour.

AUTHOR

       paps was written by Dov Grobgeld <dov.grobgeld@gmail.com>.

       This manual page was written by Lior Kaplan <kaplan@debian.org>, for the Debian project (but may be used by others).

								  April  17, 2006							   PAPS(1)
All times are GMT -4. The time now is 08:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy