Sed append newline character

Gaurang033 · #1 (**permalink**) 06-26-2008

Hi All,

I am new to Shell scripting.. I have a task to parse the text file into csv format. more then half the things has done.

But the problem is when I use the sed command in shell script. it appends newline character at the end of the line. and so when I open the file in CSV it's format is not as what I want..

I have the test file like
input.txt
test-1;1;1;100Mb/s
test-2;1;2;
test-1;2;2;
test-2;2;3;
test-2;3;4;

and I want to replace the data in the third column with some text ( it's in other file, info.txt) . And for that I am using the following shell script

info.txt
Result 1 Pass
Result 2 Fail
Result 3 Unknown
Result 4 Crash

Shell Script

Code:

exec < info.txt 
while read line; do 
	echo $line  > tmp 
	fnd=`awk '{print $2}' tmp`
	rep=`awk '{print $3}' tmp`
	echo "c=$fnd d=$rep"
        sed -i 's/test-[1-9],[1-9],'$fnd'/&'$rep'/' input.txt   
        sed -i 's/'$fnd$rep'/'$rep'/' input.txt
	          
done

After the above code I wish the input.txt file to be like below
test-1,1,Pass,100Mb/s
test-3,1,Fail,200Mb/s
test-1,3,Unknow,150Mb/s
test-1,3,Crash,150Mb/s

But when I open it in the text editor I found it like below
It appends extra newline character after the pattern find and replace

test-1,1,Pass
,100Mb/s
test-2,1,Fail
,
test-1,2,Fail
,
test-2,2,Unknown
,
test-2,3,Crash
Crash

And so when I export it to CSV file the format just ruins...

Please Please help me.
Thnx in advance ...

chihung · #2 (**permalink**) 06-26-2008

Below code is able to solve your problem. I create tmp file which define the mapping array to be included in the BEGIN block in awk and it will be dynamically included in the second awk command to replace the 3rd field based on the mapping.

Code:

$ cat a.sh
#! /bin/sh

if [ $# -ne 2 ]; then
        echo "Usage: $0 <input.txt> <info.txt>"
        exit 1
fi


tmpfile=".tmpfile-$$"
trap "rm -f $tmpfile" 0 1 2 3 15


awk '
{
        printf("map[%d]=\"%s\";",$2,$3)
}' $2 > $tmpfile


awk '
BEGIN {FS=";";OFS=";";'`cat $tmpfile`'}
{
        print $1,$2,map[$3],$4
}' $1

$ ./a.sh input.txt info.txt
test-1;1;Pass;100Mb/s
test-2;1;Fail;
test-1;2;Fail;
test-2;2;Unknown;
test-2;3;Crash;

Gaurang033 · #3 (**permalink**) 06-26-2008

Quote:

Originally Posted by chihung

Below code is able to solve your problem. I create tmp file which define the mapping array to be included in the BEGIN block in awk and it will be dynamically included in the second awk command to replace the 3rd field based on the mapping.

Code:

$ cat a.sh
#! /bin/sh

if [ $# -ne 2 ]; then
        echo "Usage: $0 <input.txt> <info.txt>"
        exit 1
fi


tmpfile=".tmpfile-$$"
trap "rm -f $tmpfile" 0 1 2 3 15


awk '
{
        printf("map[%d]=\"%s\";",$2,$3)
}' $2 > $tmpfile


awk '
BEGIN {FS=";";OFS=";";'`cat $tmpfile`'}
{
        print $1,$2,map[$3],$4
}' $1

$ ./a.sh input.txt info.txt
test-1;1;Pass;100Mb/s
test-2;1;Fail;
test-1;2;Fail;
test-2;2;Unknown;
test-2;3;Crash;

hey the solution is nice.. and hope it will work.. but the problem is that I don't have only the data I have displayed here in the input.txt file...
There are other much data above it too...

By the way I have find out where the problem lies...
actually when I read the file I get the newline character in the variable ....
So can u tell me how to remove new line character...

I tried like below but it's not working.

Code:

rep = `echo ${rep} | tr -d '\n'`

I use ^n and ^m instead of \n but they are not working as well

chihung · #4 (**permalink**) 06-26-2008

It shouldn't assign the newline to the variable

You can do a 'od -c' on the input file to see whether there is any 'special character'

Alternatively, just do one more round of pipe to avoid the problem

./a.sh input.txt info.txt | egrep -v '^$'

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
newline character	trichyselva	UNIX for Dummies Questions & Answers	9	10-31-2008 01:47 PM
Append newline at the file end	KrishnaSaran	Shell Programming and Scripting	1	06-19-2008 05:36 AM
append newline to existing variables	finalight	Shell Programming and Scripting	1	05-21-2008 05:07 AM
Newline character not working for ksh	sanikv	Shell Programming and Scripting	11	06-23-2005 01:34 PM
append newline to files with incomplete last line	ziyi	UNIX for Dummies Questions & Answers	1	04-14-2004 09:00 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread:


	Powered by