Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Script for removing newline character from file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script for removing newline character from file
# 1  
Old 04-24-2014
Script for removing newline character from file
Hi below is my file.
cat input.dat
Code:
101,abhilash,1000
102,prave
en,2000
103,partha,4
000
10
4,naresh,5000

(its just a example file)

and my output should be:
Code:
101,abhilash,1000
102,praveen,2000
103,partha,4000
104,naresh,5000

below is my code
Code:
cat input.dat |tr -d '\n' > output_file
perl -ne 's/,/++$i % 3 ? "," : "\n"/ge; print' output_file > final_file

.

but this code is not giving exact result.

Let me know any perfect approach.

Thanks in advance.
Last edited by Scrutinizer; 04-25-2014 at 05:25 AM.. Reason: code tags
# 2  
Old 04-24-2014
Totally longhand using OSX 10.7.5, default bash terminal...
This works for your highly limited file information and assumes the last 3 characters of each line ends with 000
Code:
#!/bin/sh
# nl_err.sh
echo '101,abhilash,1000
102,prave
en,2000
103,partha,4
000
10
4,naresh,5000' > /tmp/sample1.txt
n=0
newtext=""
# Corrupt file...
cat < /tmp/sample1.txt
tr -d '\n' < /tmp/sample1.txt > /tmp/sample2.txt
# Flat file...
text=$(cat < /tmp/sample2.txt)
echo ""
echo "$text"
echo ""
echo "Start the corrections..."
echo ""
# Start of conversion...
while true
do
	if [ "${text:$n:3}" == "000" ]
	then
		newtext=$newtext${text:$n:3}$'\n'
		n=$[ ( $n + 3 ) ]
	else
		newtext=$newtext${text:$n:1}
		n=$[ ( $n + 1 ) ]
	fi
	if [ $n -gt ${#text} ]
	then
		break
	fi
done
# End of conversion...
printf "$newtext" > /tmp/sample3.txt
# Final file...
cat < /tmp/sample3.txt
exit 0

Results:-
Code:
Last login: Thu Apr 24 18:57:04 on ttys000
AMIGA:barrywalker~> chmod 755 nl_err.sh
AMIGA:barrywalker~> ./nl_err.sh
101,abhilash,1000
102,prave
en,2000
103,partha,4
000
10
4,naresh,5000

101,abhilash,1000102,praveen,2000103,partha,4000104,naresh,5000

Start the corrections...

101,abhilash,1000
102,praveen,2000
103,partha,4000
104,naresh,5000
AMIGA:barrywalker~> _

# 3  
Old 04-24-2014
Code:
cat input.dat |tr -d '\n' | sed 's/000/000\n/g' > output_file

Last edited by Scrutinizer; 04-25-2014 at 05:27 AM..
# 4  
Old 04-24-2014
Previous solutions assume that the last field always has 000.
The following solution compares the number of fields, of the current line and the previous line:
Code:
awk -F, '(pNF+NF-1==cols) {print p0 $0; pNF=0; p0=""; next} (pNF==cols) {print p0} {pNF=NF; p0=$0} END {if (pNF==cols) print p0}' cols=3 input.dat

This User Gave Thanks to MadeInGermany For This Post:
abhilash_nakka
# 5  
Old 04-25-2014
Thanks guys for your reply.
but there is no such limitation tht always last field ends with '000'.
its just an example.
the last field may be a description or alpha-numeric.

so can't go with such pattern.

"objective is every line should have 5 fields."
# 6  
Old 04-25-2014
every line having 5 fileds is fine...but for the below data, how can we assume if the '10' in the 3rd line goes to above line or below?
Code:
103,partha,4
000
10
4,naresh,5000

This User Gave Thanks to SriniShoo For This Post:
abhilash_nakka
# 7  
Old 04-25-2014
Thanks "MadeInGermany" dude.
Its working perfect.
i had chekcd for multiple scenarios and its given perfect results.

one small additional requirement.
if line is devided into morethan 2 lines then its eliminating 1st line.
letme show u an example.

actual line:
Code:
1,abhilash,Trainy,India

in the source file its exist as:
Code:
1,abhilash,Tra
iny,I
ndia

with above code the result shown as
Code:
[E20044773]:TgtFiles> awk -F, '(pNF+NF-1==cols) {print p0 $0; pNF=0; p0=""; next} (pNF==cols) {print p0} {pNF=NF; p0=$0} END {if (pNF==cols) print p0}' cols=4 input.dat
 
1,abhilash,Trainy,I



any suggestions pls?

Thanks in advance


---------- Post updated at 12:52 AM ---------- Previous update was at 12:49 AM ----------

---------- Post updated at 03:15 AM ---------- Previous update was at 12:52 AM ----------

Hi Srini , i got your point but i could not see that issue with below code.

Code:
awk -F« '(pNF+NF-1==cols) {print p0 $0; pNF=0; p0=""; next} (pNF==cols) {print p0} {pNF=NF; p0=$0} END {if (pNF==cols) print p0}' cols=24 input.dat


now the issue is if the line splited into more than 2 lines then its only merging current and just previous line. not to all .
example i shown in previous thread.

can u help here?
Last edited by Scrutinizer; 04-25-2014 at 05:28 AM.. Reason: not related; mod: CODE tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove newline character from column spread over multiple lines in a file

Hi, I came across one issue recently where output from one of the columns of the table from where i am creating input file has newline characters hence, record in the file is spread over multiple lines. Fields in the file are separated by pipe (|) delimiter. As header will never have newline... (4 Replies)
Discussion started by: Prathmesh
4 Replies

2. Shell Programming and Scripting

How to remove newline character if it is the only character in the entire file.?

I have a file which comes every day and the file data look's as below. Vi abc.txt a|b|c|d\n a|g|h|j\n Some times we receive the file with only a new line character in the file like vi abc.txt \n (8 Replies)
Discussion started by: rak Kundra
8 Replies

3. Shell Programming and Scripting

Warning while sorting : A newline character was added to the end of file

Hi, I am trying to sort a csv file which has say 10 lines each line having a row size that is upto 30183 no. of COLUMNS (Row length = 30183). There is a LINE FEED (LF) at the end of each line. When I try to sort this file say, based on the second FIELD using the below command, sort -t ',' +1... (5 Replies)
Discussion started by: DHeisenberg
5 Replies

4. Shell Programming and Scripting

Appending newline character End of File

Hi Gurus, Need help. I'm a beginner in Unix. I have a requirement, need to add or append newline (\n) character in file. Sample Data: 1|Main|Test|~# 2|Main|Hello|~# 3|Main|Unix|~# 4|Main|File|~#Output: 1|Main|Test|~# 2|Main|Hello|~# 3|Main|Unix|~# 4|Main|File|~#\n -- append only... (13 Replies)
Discussion started by: Gouri Solleti
13 Replies

5. Shell Programming and Scripting

How to add newline character at end of file?

Hi All, I have following piece of code in UNIX C Shell script and I want to add one more command which can add newline at the end of file only if there is no newline character exists. foreach file (`ls $dd_PLAYCARD_EDI_IN`) if ( -f $dd_PLAYCARD_EDI_IN/${file} ) then cat -n... (4 Replies)
Discussion started by: jnrohit2k
4 Replies

6. Shell Programming and Scripting

Echo to file using SH without adding newline character

Hello! I am able to do this in bash, using: echo -ne HELLO > file.txt and then, 'HELLO' is written into file.txt without the newline character to be added in the end of the file. How is this possible to be done using sh instead of bash? If I try something similar is SH, then inside... (3 Replies)
Discussion started by: hakermania
3 Replies

7. Shell Programming and Scripting

How to check newline character in file?

Hi All, I have file with only one record,always be only one record. as like below. if that line contains newline end of the line..no need to add, if not just add the new line character. END OF FILE. ROW COUNT: 7 Please help me.. Thanks, (9 Replies)
Discussion started by: bmk
9 Replies

8. Shell Programming and Scripting

How to avoid Newline character in generated text file?

Hi All, Just need small help in resolving the special new line character in generated output file. In one of my shell script I am using following lines to get the spool file (i.e. sfile.txt) and AAA_XXXX_p111_n222.txt AAA_YYYY_p111_n222.txt Here assuming v_pnum="p111" v_nid="n222" ... (1 Reply)
Discussion started by: shekharjchandra
1 Replies

9. Shell Programming and Scripting

To remove the newline character while appending into a file

Hi All, We append the output of a file's size in a file. But a newline character is appended after the variable. Pls help how to clear this. filesize=`ls -l test.txt | awk `{print $5}'` echo File size of test.txt is $filesize bytes >> logfile.txt The output we got is, File size of... (4 Replies)
Discussion started by: amio
4 Replies

10. Shell Programming and Scripting

Removing Embedded Newline from Delimited File

Hey there - a bit of background on what I'm trying to accomplish, first off. I am trying to load the data from a pipe delimited file into a database. The loading tool that I use cannot handle embedded newline characters within a field, so I need to scrub them out. Solutions that I have tried... (7 Replies)
Discussion started by: bbetteridge
7 Replies
Login or Register to Ask a Question