UNIX for Dummies Questions & Answers

How do I get rid of null characters/strings in a unix text file. I need to use this in a script. When I use dev/null it puts null characters in my file and makes it larger. Please help

What exactly do you mean, what are you using /dev/null for?

When I use it as a null device that exactly how it behaves:

Code:

user1@balrog ~
$ touch emptyfile

user1@balrog ~
$ ls -l emptyfile 
-rw-r--r--  1 user1 staff 0 Aug 30 01:15 emptyfile

user1@balrog ~
$ cp /dev/null emptyfile 

user1@balrog ~
$ ls -l emptyfile 
-rw-r--r--  1 user1 staff 0 Aug 30 01:15 emptyfile

user1@balrog ~
$ cat /dev/null > emptyfile 

user1@balrog ~
$ ls -l emptyfile 
-rw-r--r--  1 user1 staff 0 Aug 30 01:15 emptyfile

To remove nulls from a file...

Code:

sed 's/\x0//g' file1 > file2

Hi,

We have an input datafile coming in from a different source. When this file is being processed by the Unix code on a AIX machine, it give a different record count and fails reconciliation with the expected couts. In short the Binary Nulls are getting treated differently when we are doing a processing of file. The main thing change is when we redirect the data from the input data file to another temporary file to remove the last line, which is a control record.

In this process the file that gets created as a totally different record length as the binary nulls are being treated differently by the unix.

Please advice.

MVS ignores nulls, UNIX does not - for example. You will have to change the "binary nulls" to spaces

Code:

sed 's/\x0/ /g' file1 > file2

But this means that we are modifying inpur data, which is really not possible. If a binary null has come in a number field position then it will throw an error at oracle level.
But if we replace the binary Nulls with space then it will throw an error for invalid data.
And also if binary nulls is in a varchar field which is a unique constraint then space will be taken as a value and the wrong data will be entered.

I dont think replacing them with the space is not a good idea.

I saw another thread talkingabout the following command

perl -pi -e '$_ = "" if ($. == 1);' filename

All I need is to remove the last line of control record from the input file and go ahead.

Please let me know if this approach is right. Do you foresee any negative consequence?

Thx
Arch