File conversion and removing special characters from a file in Linux
I have a .CSV file when I check for the special characters in the file using the command
, i get very lengthy lines with "^@", "^I^@" and "^@^M" characters in between each alphabet in all of the records. Using the code below
I get the output as
Quote:
filename.csv: Little-endian UTF-16 Unicode English character data, with very long lines, with CRLF, CR line terminators
I have a script to remove the control M (^M) from the file, whose output returns me an error saying : cannot execute binary file.
I know that ^I represent a tab. I also have a script to convert ^I to comma delimited file but Can anyone help me format the file with respect to the error and also ^@.
I have file special.txt with the following data.
<header info>
123$ty5%98&0asd
1@356fgbv78
09*&^5jkns43(
...........some more rows.
In my output file, I want to eliminate all the special characters in my file and I want all other data. need some help. (6 Replies)
How to remove special chracters @ END OF EACH LINE in a file
file1.txt:
0003073413^M
0003073351^M
0003073379^M
0003282724^M
0003323334^M
0003217159^M
0003102760^M
0002228911^M
I used the below command but it is not working ?
perl -pi -e 's/^M\/g' file1.txt (6 Replies)
Hi,
On AIX 5200-07-00 I have a find command as following to delete files from a certain location that are more than 7 days old. I am being told that I cannot use -exec option to delete files from these directories.
Having said that I am more curious to know how this can be done.
an sample... (3 Replies)
what my code is doing, it is executing a sql file and the resullset of the query is getting stored in the text file in a fixed format. for that fixed format i have used the following code::
Code:
awk -F":"... (2 Replies)
Dear Friends,
I want to remove text between two patters.
Problem is, it has random special characters like \ / | * ` ~ ! $ etc.
These random special characters has no fixed length. But these special characters are appearing between a fixed pattern
e.g.
DM&^%#|#!\/?CT
Expected output... (14 Replies)
Hi,
My file has this special character "^M"
I would like to remove this characters.
eg:
abc,abc,^M
i tried using sed but doesnt work.
i used octal dump command to see special character it returns following:
015
\r
Appreciate your reply. (6 Replies)
Hi,
I have a .csv file which as empty lines with comma and some special characters in 3rd column as below.
Source data
1,2,3,4,%#,6
,,,,,,
1,2,3,4,5,6
Target Data
1,2,3,4,5,6I need to remove blank lines and special charcters
I am trying to get this using the below awk
awk -F","... (2 Replies)
I have developed a small script to remove the Control M characters that get embedded when we move any file from Windows to Unix. For some reason, its not working in all scenarios. Some times I still see the ^M not being removed. Is there anything missing in the script:
cd ${inputDir}... (7 Replies)
Hello All ,
1. I am trying to do a task where I need to remove Blank spaces from my file , I am usingawk '{$1=$1}{print}' file>file1Input :-
;05/12/1990 ;31/03/2014 ;
Output:-
;05/12/1990 ;31/03/2014 ;This command is not removing all spaces from... (6 Replies)
Hi,
Does anyone know if there is a script or program available out there that uses a conversion table to replace special characters from a file?
I am trying to remove some special characters from a file but there are several unprintable/control characters that some I need to remove but some I... (2 Replies)
Discussion started by: newbie_01
2 Replies
LEARN ABOUT SUNOS
diffmk
diffmk(1) User Commands diffmk(1)NAME
diffmk - mark differences between versions of a troff input file
SYNOPSIS
diffmk oldfile newfile markedfile
DESCRIPTION
diffmk compares two versions of a file and creates a third version that includes "change mark" (.mc) commands for nroff(1) and troff(1).
oldfile and newfile are the old and new versions of the file. diffmk generates markedfile, which, contains the text from newfile with
troff(1) "change mark" requests (.mc) inserted where newfile differs from oldfile. When markedfile is formatted, changed or inserted text
is shown by | at the right margin of each line. The position of deleted text is shown by a single *.
USAGE
See largefile(5) for the description of the behavior of diffmk when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes).
EXAMPLES
Example 1: An example of the diffmk command.
diffmk can also be used in conjunction with the proper troff requests to produce program listings with marked changes. In the following
command line:
example% diffmk old.c new.c marked.c ; nroff reqs marked.c | pr
the file reqs contains the following troff requests:
.pl 1
.ll 77
.nf
.eo
.nh
which eliminate page breaks, adjust the line length, set no-fill mode, ignore escape characters, and turn off hyphenation, respectively.
If the characters | and * are inappropriate, you might run markedfile through sed(1) to globally change them.
ATTRIBUTES
See attributes(5) for descriptions of the following attributes:
+-----------------------------+-----------------------------+
| ATTRIBUTE TYPE | ATTRIBUTE VALUE |
+-----------------------------+-----------------------------+
|Availability |SUNWdoc |
+-----------------------------+-----------------------------+
SEE ALSO diff(1), nroff(1), sed(1), troff(1), attributes(5), largefile(5)BUGS
Aesthetic considerations may dictate manual adjustment of some output. File differences involving only formatting requests may produce
undesirable output, that is, replacing .sp by .sp 2 will produce a "change mark" on the preceding or following line of output.
SunOS 5.10 14 Sep 1992 diffmk(1)