Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Getting an unexpected newline in my while loop line-by-line feed

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting an unexpected newline in my while loop line-by-line feed
# 1  
Old 12-09-2018
Getting an unexpected newline in my while loop line-by-line feed
Hi,
I'm trying to get a line returned as is from the below input.csv file in Bash in Linux, and somehow I get an unexpected newline in the middle of my input.

Here's a sample line in input.csv

Code:
$> more input.csv
TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA H|BROWN

And here's a very basic while loop to read the contents of input.csv:

Code:
INPUT=`cat input.csv`
for read in $INPUT; do
	echo "current line: " $read
done

Here is the output:

Code:
current line:  TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA
current line:  H|BROWN

I noticed that when I remove ' H' from the input (if the input.csv line reads
Code:
TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA|BROWN),

then the whole line gets printed fine. So there's something fishy with the ' H' format that's driving me nuts.

Thanks.
# 2  
Old 12-10-2018
It's easy to see from your sample input that the issue is the space between the A and the H. You have not written your script in any way that it knows the input file is a CSV file, so your post is a bit confusing.

What did you expect the output to be, exactly, based on the script you posted?
# 3  
Old 12-10-2018
Some more comments:
- it's not wise (though not an error) to use read for a variable name as it is a bash (builtin) command.
- it might be worthwhile to read about the IFS variable in bash.
- why the detour over cat and a variable? Why a for loop at all?
- .csv stands for "comma separated values" - pipes as separators are possible, but quite unusual.
# 4  
Old 12-10-2018
Quote:
Originally Posted by ChicagoBlues
I'm trying to get a line returned as is from the below input.csv file in Bash in Linux, and somehow I get an unexpected newline in the middle of my input.
In addition to what RudiC so succinctly analysed: you fell victim to something that is called "field splitting" in the shell. I suggest you look up the concept, it will make the meaning of the suggested IFS variable much clearer.

I hope this helps.

bakunin
# 5  
Old 12-10-2018
Thanks guys, I will look into 'field splitting' concept and figure out the IFS command.

--- Post updated at 10:49 AM ---

Here is what works for me:

Code:
printf '%s\n' "$INPUT" |
  while IFS= read -r line; do printf '%s\n' "$line"; done

Thanks again.
# 6  
Old 12-10-2018
Quote:
Originally Posted by ChicagoBlues
Here is what works for me:

Code:
printf '%s\n' "$INPUT" |
  while IFS= read -r line; do printf '%s\n' "$line"; done

Easier and without detours:

Code:
while IFS= read line ; do
     printf "%s\n" "$line"     # change this to the actual processing code
done < /path/to/your/file

For "field splitting" you might want to start here: Help with file compare and move script. See post #6 in this thread

I hope this helps.

bakunin
Last edited by bakunin; 12-10-2018 at 11:46 AM..
# 7  
Old 12-10-2018
Code:
INPUT=`cat input.csv`
for read in $INPUT; do
	echo "current line: " $read
done

need to quote variable:
Code:
INPUT=`cat input.csv`
for read in "$INPUT"; do
	echo "current line: " $read
done

if more than one line
Code:
mapfile -t < <(cat input.csv)
for read in "${MAPFILE[@]}"; do
        echo "current line: " $read
done

Last edited by nezabudka; 12-10-2018 at 01:15 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove line feed in data

Please use code tags for sample data Hi I have a file where there are line feeds in the data. I am not able to read the file from an application. I exported this data from Access database and many columns contain line feed. My data looks like this abcd,efgh,ijkl,mnop abcd,ef... (7 Replies)
Discussion started by: dnat
7 Replies

2. Shell Programming and Scripting

Want to remove a line feed depending on number of tabs in a line

Hi! I have been struggling with a large file that has stray end of line characters. I am working on a Mac (Lion). I mention this only because I have been mucking around with fixing my problem using sed, and I have learned far more than I wanted to know about Unix and Mac eol characters. I... (1 Reply)
Discussion started by: user999991
1 Replies

3. Shell Programming and Scripting

awk for replacing line feed

Hello all, I have data like "1"|"My_name"|"My_Email"|"My_Last"|My_other" "2"|"My_name"|"My_Email"|"My_Last"|My_other" "3"|"My_name"|"My_Email"|" "|My_other" "1"|"My_name"|"My_Email"|"My_Last"|My_other" Need output like "1"|"My_name"|"My_Email"|"My_Last"|My_other"... (10 Replies)
Discussion started by: lokaish23
10 Replies

4. UNIX for Dummies Questions & Answers

Adding a line feed to each line

Hi folks, I did a search on this and didn't really find what fit my needs. I have a file with fixed record length of 436 bytes. I want to add a line feed x"0A" to the end x"0A" of each record(in byte 437). What's the easiest way to accomplish this? Thanks. (5 Replies)
Discussion started by: keeferb
5 Replies

5. Shell Programming and Scripting

Get the 1st 99 characters and add new line feed at the end of the line

I have a file with varying record length in it. I need to reformat this file so that each line will have a length of 100 characters (99 characters + the line feed). AU * A01 EXPENSE 6990370000 CWF SUBC TRAVEL & MISC MY * A02 RESALE 6990788000 Y... (3 Replies)
Discussion started by: udelalv
3 Replies

6. Shell Programming and Scripting

line feed printing issue

I am have an issue with the carrige or line feed chars showing up on info converted to a pdf file from a Orcale tool. Any direction would be appericated. ex. John doe 12435 1 232344 1 Jane doe 12435 1 424343 1 when should be like this John doe... (1 Reply)
Discussion started by: sherrod6970
1 Replies

7. Shell Programming and Scripting

for loop not working - syntax error at line 6: `end of file' unexpected

I have a file called test.dat which contains a b I have written a shell script called test.sh for i in `cat test.dat` do echo $i done When i run this script using sh test.sh I get this message - test.sh: syntax error at line 6: `end of file' unexpected What is the... (3 Replies)
Discussion started by: debojyoty
3 Replies

8. Shell Programming and Scripting

replace last form feed with line feed

Hi I have a file with lots of line feeds and form feeds (page break). Need to replace last occurrence of form feed (created by - echo "\f" ) in the file with line feed. Please advise how can i achieve this. TIA Prvn (5 Replies)
Discussion started by: prvnrk
5 Replies

9. Shell Programming and Scripting

need script-remove line feed

hi all, i have csv file with three comma separated columns i/p file First_Name, Address, Last_Name XXX, "456 New albany \n newyork, Unitedstates \n 45322-33", YYY\n ZZZ, "654 rifle park \n toronto, canada \n 43L-w3b", RRR\n is there any way i can remove \n (newline) from the second... (1 Reply)
Discussion started by: gowrish
1 Replies

10. UNIX for Dummies Questions & Answers

file feed one line per argument

What tools can I use to accomplish this? I'm writing a shell script to analyze an inittab file. Here's a sample file: init:3:initdefault: ioin::sysinit:/sbin/ioinitrc >/dev/console 2>&1 tape::sysinit:/sbin/mtinit > /dev/console 2>&1 muxi::sysinit:/sbin/dasetup </dev/console >/dev/console... (10 Replies)
Discussion started by: jpprial
10 Replies
Login or Register to Ask a Question