Shell Programming and Scripting

Hi,

i am facing issue while reading files in unix, which has ^J at end of every line. I want to know how to replace it in the file.

Example: infile.txt has below data
-----------------
20101030^J
110

Now in the above file i want unix command to replace the ^J character . so that i will use in my script for source files to replace the '^J' character.

I am New bee to Control Characters. Dont know how to deal with them. Please help me in fixing this.

Thanks very much in Advance.
KK

You can try

Code:

sed 's/\^J//g' inputfile

But the sed command not modifying the source file.. it is displaying the data without ^J character . but not modifying the source file.

This is a strange request because ctrl/j is line-feed is the standard unix line text file terminator.

What software are you using to see this character(s)?
Can you display the file in Hex or Octal so we can see the character(s)?

What do you get for this "sed" enquiry which is designed to make control characters visible:

Code:

sed -n l infile.txt

Sed will not work properly unless we have a correctly formatted text file.

Output it to some other file

Code:

sed 's/\^J//g' inputfile > outputfile

When dealing with characters, that are not easy to type in an editor, I usually find the "offending" character with od. od can be found on most *nixes, and supports multiple ways to display it's input.

Here are a few examples. First I created some arbitary data:

Code:

perl -e 'print chr($_) for (1..10);' > some_data

And the outputs of od for that file:

Hexadecimal 2-byte output:

Code:

$ od -h < some_data
0000000 0201 0403 0605 0807 0a09
0000012

Hexadecimal 2-byte output with the characters next to it:

Code:

$ od -hc < some_data
0000000 0201 0403 0605 0807 0a09
        001 002 003 004 005 006  \a  \b  \t  \n
0000012

Octal 2-byte output with the characters:

Code:

$ od -oc < some_data
0000000 001001 002003 003005 004007 005011
        001 002 003 004 005 006  \a  \b  \t  \n
0000012

Octal output with the characters:

Code:

$ od -bc < some_data
0000000 001 002 003 004 005 006 007 010 011 012
        001 002 003 004 005 006  \a  \b  \t  \n
0000012

Then just use sed or something similar to remove/translate the characters.

EDIT: Added last example.

I have done the file ftp and place it in AIX box. Now i am surprised with the source file... it is not showing any special character in the file when i opened through vi. but when i have given the command tail -1 input.txt.. it is not giving the last line , it is giving all the two lines from the source file.

file inut.txt:
-----------
20101021
1023

Below is the output after giving the command tail -1 input.txt:
20101021
1023

it is displaying all the lines from the source file . I am shocked tat why it is happening even though there is no special character visible in the file.
How to fix this. and identify the special character.
Please suggest how to fix it

Thanks In Advance
kk