06-07-2017
Need to remove Junk characters
Hi All,
I have a issue that we are getting Junk characters from source and i am not able to load that records to Database.
- Line breakers
- Junk Characters (Â and different every time)
- Japanese Characters [“ ネットワールド(サーバー、ネットワーク、ストレージ) ]
Every time I am using
grep command and
awk -F "\007" to find them and delete that record manually.
Can I have a one script too handle all this issue ? My delimiter is
\007 for all files.
Script may remove this Junk characters (
Â) and remove/adjust line breakers and make record valid.
Can anyone help me in this ?
Last edited by rbatte1; 06-08-2017 at 04:01 AM..
Reason: Converted textual numeric list to formatted numeric list with LIST=1 tags and added ICODE tags
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have written the following code ...to include the Subject, Message Body and Attachment with sendmail.
When I send mail from my Unix account to diffrent mail servers like Yahoo , Hotmail etc..I recv the Message Body but there is no newline character at the end of each line....
Also I recv the... (5 Replies)
Discussion started by: Amruta Pitkar
5 Replies
2. Shell Programming and Scripting
Guys,
can you help me in removing the junk character "^S" from the below line using perl
Reference Data Not Recognised ^S Where a value is provided by the consuming system, which is not reco
Thanks,
M.Mohan (1 Reply)
Discussion started by: mohan_xunil
1 Replies
3. Shell Programming and Scripting
Hi,
Is there anyway to find the junk characters in a file.Consider the file has data as given below:
123|abc^M|Doctor^C #record 1
234|def|Med #record 2
345|dfg^C|Wrong^V #record 3
The junk characters are highlighted and this is a pipe delimited file.
Is there anyway to... (20 Replies)
Discussion started by: ashwin3086
20 Replies
4. Shell Programming and Scripting
Hi,
I have a file with data as given below
$cat file1
123|abc|345
345|def|567
The first record is good record. The second record has an invisible junk character like \032.
I was replace all the occurences of that invisible character with #.
I want to do this for a set of... (16 Replies)
Discussion started by: ashwin3086
16 Replies
5. Shell Programming and Scripting
Urgently ur help is needed.
Actually my req is i have an input file, that input file may have junk characters (^M, ^Z) etc...
eg:
cat file
name abc^Z addres
name2 msdmskd^Z address2
I want to validate the record and display where exactly this junk character resides.
I want to... (3 Replies)
Discussion started by: help_scr_seeker
3 Replies
6. Shell Programming and Scripting
I have script which send a mail with top output. The script look like
$ cat health.sh
#!/bin/sh
maillist="email address"
rm /home/rtq1/file
top -n 1 | head 15 > file
cat file | mailx -s "Daily Health Report from `hostname` ..." "${maillist}"
But now i am getting some junk characters along... (1 Reply)
Discussion started by: Renjesh
1 Replies
7. UNIX for Dummies Questions & Answers
Hi
I have to remove the junk characters from my file. Please help..
File content :
CURITY_CODE_GSD) FROM� DL_CB_SOD_EOD_VALUATION WHERE� ASOF (1 Reply)
Discussion started by: arukuku
1 Replies
8. Solaris
Hi,
I rebooted a Solaris 11 box and after that date stamp is coming in junk in almost all directories.
root@tstilp05 # ls -l
total 112
drwxrwxr-x 9 root sys 19 juin 1 03:10 adm
drwxr-xr-x 6 root sys 6 sept. 19 2012 ai
drwxr-xr-x 3 root bin ... (3 Replies)
Discussion started by: solaris_1977
3 Replies
9. Shell Programming and Scripting
I am using flatfile, in that flat file we are getting the junk chars
1)I21001f<82>^Me<85>!h49 Service Charge
2) I21001f‚
e...!h49 Service Charge
please tell me how to remove all junk chars in unix scripts. (1 Reply)
Discussion started by: Talari
1 Replies
10. UNIX for Dummies Questions & Answers
I would like to remove all characters starting with "%" and ending with ")" in the 4th field - please help!!
1412007819.864 /device/services/heartbeatxx 204 0.547%!i(int=0) 0.434 0.112
1412007819.866 /device/services/heartbeatxx 204 0.547%!i(int=1) 0.423 0.123... (10 Replies)
Discussion started by: snemuk14
10 Replies
CUT(1) User Commands CUT(1)
NAME
cut - remove sections from each line of files
SYNOPSIS
cut OPTION... [FILE]...
DESCRIPTION
Print selected parts of lines from each FILE to standard output.
Mandatory arguments to long options are mandatory for short options too.
-b, --bytes=LIST
select only these bytes
-c, --characters=LIST
select only these characters
-d, --delimiter=DELIM
use DELIM instead of TAB for field delimiter
-f, --fields=LIST
select only these fields; also print any line that contains no delimiter character, unless the -s option is specified
-n (ignored)
--complement
complement the set of selected bytes, characters or fields
-s, --only-delimited
do not print lines not containing delimiters
--output-delimiter=STRING
use STRING as the output delimiter the default is to use the input delimiter
--help display this help and exit
--version
output version information and exit
Use one, and only one of -b, -c or -f. Each LIST is made up of one range, or many ranges separated by commas. Selected input is written
in the same order that it is read, and is written exactly once. Each range is one of:
N N'th byte, character or field, counted from 1
N- from N'th byte, character or field, to end of line
N-M from N'th to M'th (included) byte, character or field
-M from first to M'th (included) byte, character or field
With no FILE, or when FILE is -, read standard input.
AUTHOR
Written by David M. Ihnat, David MacKenzie, and Jim Meyering.
REPORTING BUGS
Report cut bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
Report cut translation bugs to <http://translationproject.org/team/>
COPYRIGHT
Copyright (C) 2011 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
SEE ALSO
The full documentation for cut is maintained as a Texinfo manual. If the info and cut programs are properly installed at your site, the
command
info coreutils 'cut invocation'
should give you access to the complete manual.
GNU coreutils 8.12.197-032bb September 2011 CUT(1)