Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: File conversion and removing special characters from a file in Linux
Operating Systems Linux File conversion and removing special characters from a file in Linux Post 302863485 by dhruuv369 on Monday 14th of October 2013 11:22:23 AM
Old 10-14-2013
File conversion and removing special characters from a file in Linux
I have a .CSV file when I check for the special characters in the file using the command
Code:
cat -vet filename.csv

, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below
Code:
file filename.csv

I get the output as
Quote:
filename.csv: Little-endian UTF-16 Unicode English character data, with very long lines, with CRLF, CR line terminators
I have a script to remove the control M (^M) from the file, whose output returns me an error saying : cannot execute binary file.

I know that ^I represent a tab. I also have a script to convert ^I to comma delimited file but Can anyone help me format the file with respect to the error and also ^@.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing special characters in file

I have file special.txt with the following data. <header info> 123$ty5%98&0asd 1@356fgbv78 09*&^5jkns43( ...........some more rows. In my output file, I want to eliminate all the special characters in my file and I want all other data. need some help. (6 Replies)
Discussion started by: srivsn
6 Replies

2. Shell Programming and Scripting

removing special characters @ EOL

How to remove special chracters @ END OF EACH LINE in a file file1.txt: 0003073413^M 0003073351^M 0003073379^M 0003282724^M 0003323334^M 0003217159^M 0003102760^M 0002228911^M I used the below command but it is not working ? perl -pi -e 's/^M\/g' file1.txt (6 Replies)
Discussion started by: ali560045
6 Replies

3. AIX

Removing a filename which has special characters passed from a pipe with xargs

Hi, On AIX 5200-07-00 I have a find command as following to delete files from a certain location that are more than 7 days old. I am being told that I cannot use -exec option to delete files from these directories. Having said that I am more curious to know how this can be done. an sample... (3 Replies)
Discussion started by: jerardfjay
3 Replies

4. Solaris

removing special characters, white spaces from a field in a file

what my code is doing, it is executing a sql file and the resullset of the query is getting stored in the text file in a fixed format. for that fixed format i have used the following code:: Code: awk -F":"... (2 Replies)
Discussion started by: priyanka3006
2 Replies

5. Shell Programming and Scripting

Removing special characters

Dear Friends, I want to remove text between two patters. Problem is, it has random special characters like \ / | * ` ~ ! $ etc. These random special characters has no fixed length. But these special characters are appearing between a fixed pattern e.g. DM&^%#|#!\/?CT Expected output... (14 Replies)
Discussion started by: anushree.a
14 Replies

6. Shell Programming and Scripting

Removing Special Character from File.

Hi, My file has this special character "^M" I would like to remove this characters. eg: abc,abc,^M i tried using sed but doesnt work. i used octal dump command to see special character it returns following: 015 \r Appreciate your reply. (6 Replies)
Discussion started by: pinnacle
6 Replies

7. UNIX for Dummies Questions & Answers

awk for removing special characters and extra commas

Hi, I have a .csv file which as empty lines with comma and some special characters in 3rd column as below. Source data 1,2,3,4,%#,6 ,,,,,, 1,2,3,4,5,6 Target Data 1,2,3,4,5,6I need to remove blank lines and special charcters I am trying to get this using the below awk awk -F","... (2 Replies)
Discussion started by: shruthidwh
2 Replies

8. Shell Programming and Scripting

Removing special characters - Control M

I have developed a small script to remove the Control M characters that get embedded when we move any file from Windows to Unix. For some reason, its not working in all scenarios. Some times I still see the ^M not being removed. Is there anything missing in the script: cd ${inputDir}... (7 Replies)
Discussion started by: vskr72
7 Replies

9. Shell Programming and Scripting

Removing blank/white spaces and special characters

Hello All , 1. I am trying to do a task where I need to remove Blank spaces from my file , I am usingawk '{$1=$1}{print}' file>file1Input :- ;05/12/1990 ;31/03/2014 ; Output:- ;05/12/1990 ;31/03/2014 ;This command is not removing all spaces from... (6 Replies)
Discussion started by: himanshu sood
6 Replies

10. UNIX for Beginners Questions & Answers

Replacing string/special characters using a 'conversion' table

Hi, Does anyone know if there is a script or program available out there that uses a conversion table to replace special characters from a file? I am trying to remove some special characters from a file but there are several unprintable/control characters that some I need to remove but some I... (2 Replies)
Discussion started by: newbie_01
2 Replies

LEARN ABOUT DEBIAN

mongoexport

MONGOEXPORT(1)							  Mongo Database						    MONGOEXPORT(1)

NAME

       mongoexport - the Mongo export tool

SYNOPSIS

       mongoexport [OPTIONS]

DESCRIPTION

       mongoexport  is a tool to export a MongoDB collection to either JSON or CSV. The query can be filtered or a list of fields to output can be
       given.

       If the output is CSV, the fields must be specified in order.

EXAMPLES

       mongoexport -d test -c test1 --csv -f name,num
	      export documents from test.test1 in CSV format

OPTIONS

       --help show usage information

       -h, --host HOST
	      server to connect to (default HOST=localhost)

       -d, --db DATABASE
	      database to use

       -c, --c COLLECTION
	      collection to use

       -q, --query QUERY
	      query filter

       -f, --fields FIELDS
	      comma-separated list of field names

       --csv  export to CSV instead of JSON

       -o, --out FILE
	      output file, if not specified, stdout is used

       --dbpath PATH
	      directly access mongod data files in this path, instead of connecting to a mongod instance

COPYRIGHT

       Copyright 2007-2009 10gen

SEE ALSO

       For more information, please refer to the MongoDB wiki, available at http://www.mongodb.org.

AUTHOR

       Kristina Chodorow

10gen								     June 2009							    MONGOEXPORT(1)
All times are GMT -4. The time now is 01:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy