01-31-2008
Invalid Characters in the file.
I am working on AIX. We ftp files to a database. The flat files are having thousands of records and each record is having some 50 to 60 characters(there are fields having certain character length). In addition to some valid ascii characters some invalid characters like Å, å, Ä, ä or pipes creep in which datawarehouse rejects to load in.
Example: AcuM-^?a 051706 ;
above is a field in the record which is having special characters like -,^ and ? , which should not have been there.
The record separator is a new line and there is no field seperator.
How can I remove these invalid or special characters to creep in the records?
Please help me to find the logic in the shell sripting..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am working on AIX. We ftp files to a database. The flat files are having thousands of records and each record is having some 50 to 60 characters(there are fields having certain character length). In addition to some valid ascii characters some invalid characters like Å, å, Ä, ä or pipes creep in which... (5 Replies)
Discussion started by: kanu_pathak
5 Replies
2. Programming
This is a pretty straight-forward question. Within a program of mine, I have a string that's going to be used as a filename, but it might have some invalid characters in it that wouldn't be valid in a filename. If there are any invalid characters, I want to get rid of them and essentially squeeze... (4 Replies)
Discussion started by: cleopard
4 Replies
3. Shell Programming and Scripting
Hi,
I have to write s script to check an input file for invalid characters. In this script I have to find the exact line of the invalid character. If the input file contain 2 invalid character sat line 10 and 17, the script will show the value 10 and 17. Any help is appreciated. (3 Replies)
Discussion started by: beginner82
3 Replies
4. UNIX for Dummies Questions & Answers
there is a file is generated from my program due to undefined filename.
-rw-r--r-- 1 angie angie 8644055 Jun 22 09:17 Ô$ÿÿÿÿÿÆ
may i know how to delete this file..??? thanks in advance... :) (5 Replies)
Discussion started by: chxxangie
5 Replies
5. Shell Programming and Scripting
HI,
I have a source file which has the below data.
Tableid,table.txt
sourceid,1,2,3,4,5,6
targetid,1,2,3,4,5,6
Tableid,table
sourceid,1,2,3,4,5,6
targetid,1,2,3,4,5,6
Tableid,table.txt
sourceid,1,2,3,4,5,6
targetid,1,2,3,4,5,6
Tableid,table
sourceid,1,2,3,4,5,6
targetid,1,2,3,4,5,6... (6 Replies)
Discussion started by: shruthidwh
6 Replies
6. Shell Programming and Scripting
Hi All -
I'm building a script wherein it is design to remove characters that are not accepted on a non-unicode database. Examples are the following: ï,¿,½,Â,é, etc.
I can easily sed those characters one-by-one but I there's a problem when other unicode characters are found. Is there any way to... (1 Reply)
Discussion started by: Jin_
1 Replies
7. Shell Programming and Scripting
Hi All,
How to validate the 4th column,it is date column in the file, if it valid move to valid file else moved invalid file.
9f680174-cb87|20077337254|0|20120511|N
9f680174-cb88|20077337254|0|20120534|N
i want two file valid.txt and invalid.txt
Thanks, (7 Replies)
Discussion started by: bmk
7 Replies
8. UNIX for Dummies Questions & Answers
Hello,
Can any one help me in below query to search all the invalid characters that UNIX cannot recognize from a file. can we do anything with the help of grep command or any other commands.
Also, i am not sure what are the invalid characters present in the file.
Many thanks in advance.
... (6 Replies)
Discussion started by: schandru
6 Replies
9. Shell Programming and Scripting
My Input file is fixed length record ends with . as end of the line and the character length is 4156
Example:
12234XYZ TY^4253$+00000-00000...........
I need to check is there any control characters(like ^M,^Z)
The line will be splitted
awk
'{id=substr($0,1,5)
nm=substr($0,6,3)... (2 Replies)
Discussion started by: dineshaila
2 Replies
10. UNIX for Beginners Questions & Answers
Hello guys,
Here i am writing a script to check for a valid url from a file,i am getting the valid url & i print it in a file and i want to print the invalid url also.how to do that?
#here is my script
if
then
URL=$(grep -E -o... (2 Replies)
Discussion started by: Meeran Rizvi
2 Replies
LEARN ABOUT OPENDARWIN
cap_mkdb
CAP_MKDB(1) BSD General Commands Manual CAP_MKDB(1)
NAME
cap_mkdb -- create capability database
SYNOPSIS
cap_mkdb [-v] [-f outfile] file1 [file2 ...]
DESCRIPTION
Cap_mkdb builds a hashed database out of the getcap(3) logical database constructed by the concatenation of the specified files .
The database is named by the basename of the first file argument and the string ``.db''. The getcap(3) routines can access the database in
this form much more quickly than they can the original text file(s).
The ``tc'' capabilities of the records are expanded before the record is stored into the database.
The options as as follows:
-f outfile
Specify a different database basename.
-v Print out the number of capability records in the database.
FORMAT
Each record is stored in the database using two different types of keys.
The first type is a key which consists of the first capability of the record (not including the trailing colon (``:'')) with a data field
consisting of a special byte followed by the rest of the record. The special byte is either a 0 or 1, where a 0 means that the record is
okay, and a 1 means that there was a ``tc'' capability in the record that couldn't be expanded.
The second type is a key which consists of one of the names from the first capability of the record with a data field consisting a special
byte followed by the the first capability of the record. The special byte is a 2.
In normal operation names are looked up in the database, resulting in a key/data pair of the second type. The data field of this key/data
pair is used to look up a key/data pair of the first type which has the real data associated with the name.
RETURN VALUE
The cap_mkdb utility exits 0 on success and >0 if an error occurs.
SEE ALSO
dbopen(3), getcap(3), termcap(5)
BSD
June 2, 2019 BSD