Sponsored Content
Top Forums UNIX for Dummies Questions & Answers To get the invalid characters from a file Post 302747347 by schandru on Friday 21st of December 2012 06:04:20 AM
Old 12-21-2012
Hi,
Today i faced a problem in which the file at Unix had the characters like � and it caused the file processing failure at downstream end which is Mainframe. and it somehow identified by the Mainframe and notified to me, so i removed the above mentioned by going to the particular line and removed the characters and it got processed successfully.

Could you please help me to find these characters like non-standard characters in a file?

Hope now this makes sense and clear..!

Chandru.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Invalid Characters in the file.

I am working on AIX. We ftp files to a database. The flat files are having thousands of records and each record is having some 50 to 60 characters(there are fields having certain character length). In addition to some valid ascii characters some invalid characters like Å, å, Ä, ä or pipes creep in which... (5 Replies)
Discussion started by: kanu_pathak
5 Replies

2. UNIX for Dummies Questions & Answers

Invalid Characters in the file.

I am working on AIX. We ftp files to a database. The flat files are having thousands of records and each record is having some 50 to 60 characters(there are fields having certain character length). In addition to some valid ascii characters some invalid characters like Å, å, Ä, ä or pipes creep in which... (15 Replies)
Discussion started by: kanu_pathak
15 Replies

3. Programming

string with invalid characters

This is a pretty straight-forward question. Within a program of mine, I have a string that's going to be used as a filename, but it might have some invalid characters in it that wouldn't be valid in a filename. If there are any invalid characters, I want to get rid of them and essentially squeeze... (4 Replies)
Discussion started by: cleopard
4 Replies

4. Shell Programming and Scripting

writing shell script to find line of invalid characters

Hi, I have to write s script to check an input file for invalid characters. In this script I have to find the exact line of the invalid character. If the input file contain 2 invalid character sat line 10 and 17, the script will show the value 10 and 17. Any help is appreciated. (3 Replies)
Discussion started by: beginner82
3 Replies

5. UNIX for Dummies Questions & Answers

to delete an invalid file

there is a file is generated from my program due to undefined filename. -rw-r--r-- 1 angie angie 8644055 Jun 22 09:17 Ô$ÿÿÿÿÿÆ may i know how to delete this file..??? thanks in advance... :) (5 Replies)
Discussion started by: chxxangie
5 Replies

6. Shell Programming and Scripting

Capturing the invalid records to error file

HI, I have a source file which has the below data. Tableid,table.txt sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6 Tableid,table sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6 Tableid,table.txt sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6 Tableid,table sourceid,1,2,3,4,5,6 targetid,1,2,3,4,5,6... (6 Replies)
Discussion started by: shruthidwh
6 Replies

7. Shell Programming and Scripting

Remove invalid database characters on a file

Hi All - I'm building a script wherein it is design to remove characters that are not accepted on a non-unicode database. Examples are the following: ï,¿,½,Â,é, etc. I can easily sed those characters one-by-one but I there's a problem when other unicode characters are found. Is there any way to... (1 Reply)
Discussion started by: Jin_
1 Replies

8. Shell Programming and Scripting

Valid and invalid date in the file

Hi All, How to validate the 4th column,it is date column in the file, if it valid move to valid file else moved invalid file. 9f680174-cb87|20077337254|0|20120511|N 9f680174-cb88|20077337254|0|20120534|N i want two file valid.txt and invalid.txt Thanks, (7 Replies)
Discussion started by: bmk
7 Replies

9. Shell Programming and Scripting

How to get the Invalid records from a file using awk?

My Input file is fixed length record ends with . as end of the line and the character length is 4156 Example: 12234XYZ TY^4253$+00000-00000........... I need to check is there any control characters(like ^M,^Z) The line will be splitted awk '{id=substr($0,1,5) nm=substr($0,6,3)... (2 Replies)
Discussion started by: dineshaila
2 Replies

10. UNIX for Beginners Questions & Answers

Search for the invalid url in a file

Hello guys, Here i am writing a script to check for a valid url from a file,i am getting the valid url & i print it in a file and i want to print the invalid url also.how to do that? #here is my script if then URL=$(grep -E -o... (2 Replies)
Discussion started by: Meeran Rizvi
2 Replies
newform(1)						      General Commands Manual							newform(1)

NAME
newform - change or reformat a text file SYNOPSIS
[file]... DESCRIPTION
reads lines from the named files, or standard input if no input file is named, and reproduces the lines on standard output. Lines are reformatted in accordance with command line options in effect. Command line options can appear in any order, can be repeated, and can be intermingled with the optional files. Command line options are processed in the order specified. This means that option sequences such as yield results different from Options are applied to all files on the command line. Options recognizes the following options: Same as except characters are appended to the end of a line. Truncate n characters from the beginning of the line when the line length is greater than the effective line length (see The default is to truncate the number of characters necessary to obtain the effective line length. The default value is used when with no n is used. This option can be used to delete the sequence numbers from a COBOL program as follows: The must be used to set the effective line length shorter than any existing line in the file so that the option is activated. Change the prefix/append character to k. The default character for k is a space. Same as except that characters are truncated from the end of the line. Write the tab specification format line on the standard output before any other lines are output. The tab specification format line which is printed will correspond to the format specified in the option. If no option is specified, the line which is printed contains the default specification of Input tab specification: expands tabs to spaces, according to the tab specifications given. The tabspec recognizes all tab specification forms described in tabs(1). In addition, tabspec can be in which assumes that the tab specification is to be found in the first line read from the standard input (see fspec(4)). If no tabspec is given, tabspec defaults to A tabspec of expects no tabs; if any are found, they are treated as Set the effective line length to n characters. If n is not entered, defaults to 72. The default line length without the option is 80 characters. Note that tabs and backspaces are treated as single characters (use to expand tabs to spaces). Output tab specification: replaces spaces with tabs, according to the tab specifications given. The tab specifications are the same as for If no tabspec is given, tabspec defaults to A tabspec of means that no spaces will be converted to tabs on output. Prefix n characters (see to the beginning of a line when the line length is less than the effective line length. The default is to prefix the number of characters necessary to obtain the effective line length. Shear off leading characters on each line up to the first tab and place up to 8 of the sheared characters at the end of the line. If more than 8 characters (not counting the first tab) are sheared, the eighth character is replaced by a and any characters to the right of it are discarded. The first tab is always discarded. An error message and program exit occur if this option is used on a file without a tab on each line. The characters sheared off are saved internally until all other options specified are applied to that line. The characters are then added at the end of the processed line. For example, to convert a file with leading digits, one or more tabs, and text on each line, to a file beginning with the text, all tabs after the first expanded to spaces, padded with spaces out to column 72 (or truncated to column 72), and the leading digits placed starting at column 73, the command would be: RETURN VALUE
returns one of the following values: No errors encountered. An error occurred. DIAGNOSTICS
All diagnostics are fatal. was called with a bad option. There was no tab on one line. Self-explanatory. A line exceeds 512 characters after being expanded in the internal work buffer. A tab specification is incorrectly formatted, or specified tab stops are not ascending. A tabspec read from a file (or standard input) must not contain a tabspec referencing another file (or standard input). WARNINGS
normally only keeps track of physical characters; however, for the and options, keeps track of backspaces in order to line up tabs in the appropriate logical columns. does not prompt the user if a tabspec is to be read from the standard input (by use of or If the option is used, and the last option specified was and was preceded by either a or a the tab specification format line will be incor- rect. SEE ALSO
csplit(1), tabs(1), fspec(4). newform(1)
All times are GMT -4. The time now is 02:54 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy