Visit Our UNIX and Linux User Community


Sed append newline character


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sed append newline character
# 1  
Old 06-26-2008
Sed append newline character

Hi All,

I am new to Shell scripting.. I have a task to parse the text file into csv format. more then half the things has done.

But the problem is when I use the sed command in shell script. it appends newline character at the end of the line. and so when I open the file in CSV it's format is not as what I want..

I have the test file like
input.txt
test-1;1;1;100Mb/s
test-2;1;2;
test-1;2;2;
test-2;2;3;
test-2;3;4;

and I want to replace the data in the third column with some text ( it's in other file, info.txt) . And for that I am using the following shell script

info.txt
Result 1 Pass
Result 2 Fail
Result 3 Unknown
Result 4 Crash

Shell Script
Code:
exec < info.txt 
while read line; do 
	echo $line  > tmp 
	fnd=`awk '{print $2}' tmp`
	rep=`awk '{print $3}' tmp`
	echo "c=$fnd d=$rep"
        sed -i 's/test-[1-9],[1-9],'$fnd'/&'$rep'/' input.txt   
        sed -i 's/'$fnd$rep'/'$rep'/' input.txt
	          
done

After the above code I wish the input.txt file to be like below
test-1,1,Pass,100Mb/s
test-3,1,Fail,200Mb/s
test-1,3,Unknow,150Mb/s
test-1,3,Crash,150Mb/s

But when I open it in the text editor I found it like below Smilie
It appends extra newline character after the pattern find and replace


test-1,1,Pass
,100Mb/s
test-2,1,Fail
,
test-1,2,Fail
,
test-2,2,Unknown
,
test-2,3,Crash
Crash

And so when I export it to CSV file the format just ruins...


Please Please help me.
Thnx in advance ...

Last edited by Gaurang033; 06-26-2008 at 04:24 AM.. Reason: A minor mistake in the script posted
# 2  
Old 06-26-2008
Below code is able to solve your problem. I create tmp file which define the mapping array to be included in the BEGIN block in awk and it will be dynamically included in the second awk command to replace the 3rd field based on the mapping.

Code:
$ cat a.sh
#! /bin/sh

if [ $# -ne 2 ]; then
        echo "Usage: $0 <input.txt> <info.txt>"
        exit 1
fi


tmpfile=".tmpfile-$$"
trap "rm -f $tmpfile" 0 1 2 3 15


awk '
{
        printf("map[%d]=\"%s\";",$2,$3)
}' $2 > $tmpfile


awk '
BEGIN {FS=";";OFS=";";'`cat $tmpfile`'}
{
        print $1,$2,map[$3],$4
}' $1

$ ./a.sh input.txt info.txt
test-1;1;Pass;100Mb/s
test-2;1;Fail;
test-1;2;Fail;
test-2;2;Unknown;
test-2;3;Crash;

# 3  
Old 06-26-2008
Quote:
Originally Posted by chihung
Below code is able to solve your problem. I create tmp file which define the mapping array to be included in the BEGIN block in awk and it will be dynamically included in the second awk command to replace the 3rd field based on the mapping.

Code:
$ cat a.sh
#! /bin/sh

if [ $# -ne 2 ]; then
        echo "Usage: $0 <input.txt> <info.txt>"
        exit 1
fi


tmpfile=".tmpfile-$$"
trap "rm -f $tmpfile" 0 1 2 3 15


awk '
{
        printf("map[%d]=\"%s\";",$2,$3)
}' $2 > $tmpfile


awk '
BEGIN {FS=";";OFS=";";'`cat $tmpfile`'}
{
        print $1,$2,map[$3],$4
}' $1

$ ./a.sh input.txt info.txt
test-1;1;Pass;100Mb/s
test-2;1;Fail;
test-1;2;Fail;
test-2;2;Unknown;
test-2;3;Crash;

hey the solution is nice.. and hope it will work.. but the problem is that I don't have only the data I have displayed here in the input.txt file...
There are other much data above it too...

By the way I have find out where the problem lies...
actually when I read the file I get the newline character in the variable ....
So can u tell me how to remove new line character...

I tried like below but it's not working. Smilie
Code:
rep = `echo ${rep} | tr -d '\n'`

I use ^n and ^m instead of \n but they are not working as well

# 4  
Old 06-26-2008
It shouldn't assign the newline to the variable

You can do a 'od -c' on the input file to see whether there is any 'special character'

Alternatively, just do one more round of pipe to avoid the problem

./a.sh input.txt info.txt | egrep -v '^$'

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace newline with comma and append quotes

Hi, I have below requirement. Apple Orange Banana Required O/p in bash 'Apple,Orange,Banana' Can you please help. Please wrap your samples, codes in CODE TAGS as per forum rules. (3 Replies)
Discussion started by: Rtk
3 Replies

2. Shell Programming and Scripting

How to remove newline character if it is the only character in the entire file.?

I have a file which comes every day and the file data look's as below. Vi abc.txt a|b|c|d\n a|g|h|j\n Some times we receive the file with only a new line character in the file like vi abc.txt \n (8 Replies)
Discussion started by: rak Kundra
8 Replies

3. UNIX for Dummies Questions & Answers

newline character in a variable

variable="unix\nlinux" echo $variable expected output: unix linux :wall: can i do that ?? thanks in advance!! (3 Replies)
Discussion started by: sathish92
3 Replies

4. Shell Programming and Scripting

Why SED can't see the last newline character?

Removed. My question does not make sense. and SED does see the last newline character. But I still have a question: How to remove the last newline character(the newline character at the end of last line) using SED? ---------- Post updated 05-01-11 at 10:51 AM ---------- Previous update was... (7 Replies)
Discussion started by: kevintse
7 Replies

5. Shell Programming and Scripting

any savant ? using AWK/SED to remove newline character between two strings : conditional removal

I'd like to remove (do a pattern or precise replacement - this I can handle in SED using Regex ) ---AFTER THE 1ST Occurrence ( i.e. on the 2nd occurrence - from the 2nd to fourth occurance ) of a specific string : type 1 -- After the 1st occurrence of 1 string1 till the 1st occurrence of... (4 Replies)
Discussion started by: sieger007
4 Replies

6. UNIX for Dummies Questions & Answers

newline character

hi, I want to print the below lines "Message from bac logistics The Confirmation File has not been received." When i give like this in the code "Message from bac logistics\n The Confirmation File has not been received." It is giving only Message from bac logistics\n The... (9 Replies)
Discussion started by: trichyselva
9 Replies

7. Shell Programming and Scripting

Append newline at the file end

Hi All, Is there any way to append a newline character at the end of a file(coma-separated file), through shell script? I need to check whether newline character exists at the end of a file, if it does not then append it. Regards, Krishna (1 Reply)
Discussion started by: KrishnaSaran
1 Replies

8. Shell Programming and Scripting

append newline to existing variables

firstly, i check is the variable empty or not, if so vesselNameList=`echo $vesselName` if not vesselNameList="${vesselNameList}""\n"`echo "$vesselName"` and it produces this result BUNGA TERATAI 3 5055\ JADE TRADER 143W\ MOL SPLENDOR 0307A BUNGA TERATAI 3 5055\ JADE... (1 Reply)
Discussion started by: finalight
1 Replies

9. Shell Programming and Scripting

Using SED to append character to each line

Hey - my first post here, and I'm a total SED newb. I've looked around for previous help on this, but have so far been unsuccessful. I have a program (AMStracker for OS X) that outputs data in the terminal. Output is in this form: . . . 3 0 -75 3 0 -76 3 0 -77 ... (4 Replies)
Discussion started by: c0nn0r
4 Replies

10. UNIX for Dummies Questions & Answers

append newline to files with incomplete last line

Hi all, Is there any way I can check a file for the linefeed character at the end of the file, and append one only if it is missing (ie. Incomplete last line)? Need to do this because I need to write a script to process files FTP-ed over from various machines, which may or may not be... (1 Reply)
Discussion started by: ziyi
1 Replies

Featured Tech Videos