Getting an unexpected newline in my while loop line-by-line feed


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Getting an unexpected newline in my while loop line-by-line feed
# 1  
Old 12-09-2018
Getting an unexpected newline in my while loop line-by-line feed

Hi,
I'm trying to get a line returned as is from the below input.csv file in Bash in Linux, and somehow I get an unexpected newline in the middle of my input.

Here's a sample line in input.csv

Code:
$> more input.csv
TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA H|BROWN

And here's a very basic while loop to read the contents of input.csv:

Code:
INPUT=`cat input.csv`
for read in $INPUT; do
	echo "current line: " $read
done

Here is the output:

Code:
current line:  TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA
current line:  H|BROWN

I noticed that when I remove ' H' from the input (if the input.csv line reads
Code:
TEST_SYSTEM,DUMMY@GMAIL.COM|JULIA|BROWN),

then the whole line gets printed fine. So there's something fishy with the ' H' format that's driving me nuts.

Thanks.
# 2  
Old 12-10-2018
It's easy to see from your sample input that the issue is the space between the A and the H. You have not written your script in any way that it knows the input file is a CSV file, so your post is a bit confusing.

What did you expect the output to be, exactly, based on the script you posted?
# 3  
Old 12-10-2018
Some more comments:
- it's not wise (though not an error) to use read for a variable name as it is a bash (builtin) command.
- it might be worthwhile to read about the IFS variable in bash.
- why the detour over cat and a variable? Why a for loop at all?
- .csv stands for "comma separated values" - pipes as separators are possible, but quite unusual.
# 4  
Old 12-10-2018
Quote:
Originally Posted by ChicagoBlues
I'm trying to get a line returned as is from the below input.csv file in Bash in Linux, and somehow I get an unexpected newline in the middle of my input.
In addition to what RudiC so succinctly analysed: you fell victim to something that is called "field splitting" in the shell. I suggest you look up the concept, it will make the meaning of the suggested IFS variable much clearer.

I hope this helps.

bakunin
# 5  
Old 12-10-2018
Thanks guys, I will look into 'field splitting' concept and figure out the IFS command.

--- Post updated at 10:49 AM ---

Here is what works for me:

Code:
printf '%s\n' "$INPUT" |
  while IFS= read -r line; do printf '%s\n' "$line"; done

Thanks again.
# 6  
Old 12-10-2018
Quote:
Originally Posted by ChicagoBlues
Here is what works for me:

Code:
printf '%s\n' "$INPUT" |
  while IFS= read -r line; do printf '%s\n' "$line"; done

Easier and without detours:

Code:
while IFS= read line ; do
     printf "%s\n" "$line"     # change this to the actual processing code
done < /path/to/your/file

For "field splitting" you might want to start here: Help with file compare and move script. See post #6 in this thread

I hope this helps.

bakunin

Last edited by bakunin; 12-10-2018 at 11:46 AM..
# 7  
Old 12-10-2018
Code:
INPUT=`cat input.csv`
for read in $INPUT; do
	echo "current line: " $read
done

need to quote variable:
Code:
INPUT=`cat input.csv`
for read in "$INPUT"; do
	echo "current line: " $read
done

if more than one line
Code:
mapfile -t < <(cat input.csv)
for read in "${MAPFILE[@]}"; do
        echo "current line: " $read
done


Last edited by nezabudka; 12-10-2018 at 01:15 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove line feed in data

Please use code tags for sample data Hi I have a file where there are line feeds in the data. I am not able to read the file from an application. I exported this data from Access database and many columns contain line feed. My data looks like this abcd,efgh,ijkl,mnop abcd,ef... (7 Replies)
Discussion started by: dnat
7 Replies

2. Shell Programming and Scripting

Want to remove a line feed depending on number of tabs in a line

Hi! I have been struggling with a large file that has stray end of line characters. I am working on a Mac (Lion). I mention this only because I have been mucking around with fixing my problem using sed, and I have learned far more than I wanted to know about Unix and Mac eol characters. I... (1 Reply)
Discussion started by: user999991
1 Replies

3. Shell Programming and Scripting

awk for replacing line feed

Hello all, I have data like "1"|"My_name"|"My_Email"|"My_Last"|My_other" "2"|"My_name"|"My_Email"|"My_Last"|My_other" "3"|"My_name"|"My_Email"|" "|My_other" "1"|"My_name"|"My_Email"|"My_Last"|My_other" Need output like "1"|"My_name"|"My_Email"|"My_Last"|My_other"... (10 Replies)
Discussion started by: lokaish23
10 Replies

4. UNIX for Dummies Questions & Answers

Adding a line feed to each line

Hi folks, I did a search on this and didn't really find what fit my needs. I have a file with fixed record length of 436 bytes. I want to add a line feed x"0A" to the end x"0A" of each record(in byte 437). What's the easiest way to accomplish this? Thanks. (5 Replies)
Discussion started by: keeferb
5 Replies

5. Shell Programming and Scripting

Get the 1st 99 characters and add new line feed at the end of the line

I have a file with varying record length in it. I need to reformat this file so that each line will have a length of 100 characters (99 characters + the line feed). AU * A01 EXPENSE 6990370000 CWF SUBC TRAVEL & MISC MY * A02 RESALE 6990788000 Y... (3 Replies)
Discussion started by: udelalv
3 Replies

6. Shell Programming and Scripting

line feed printing issue

I am have an issue with the carrige or line feed chars showing up on info converted to a pdf file from a Orcale tool. Any direction would be appericated. ex. John doe 12435 1 232344 1 Jane doe 12435 1 424343 1 when should be like this John doe... (1 Reply)
Discussion started by: sherrod6970
1 Replies

7. Shell Programming and Scripting

for loop not working - syntax error at line 6: `end of file' unexpected

I have a file called test.dat which contains a b I have written a shell script called test.sh for i in `cat test.dat` do echo $i done When i run this script using sh test.sh I get this message - test.sh: syntax error at line 6: `end of file' unexpected What is the... (3 Replies)
Discussion started by: debojyoty
3 Replies

8. Shell Programming and Scripting

replace last form feed with line feed

Hi I have a file with lots of line feeds and form feeds (page break). Need to replace last occurrence of form feed (created by - echo "\f" ) in the file with line feed. Please advise how can i achieve this. TIA Prvn (5 Replies)
Discussion started by: prvnrk
5 Replies

9. Shell Programming and Scripting

need script-remove line feed

hi all, i have csv file with three comma separated columns i/p file First_Name, Address, Last_Name XXX, "456 New albany \n newyork, Unitedstates \n 45322-33", YYY\n ZZZ, "654 rifle park \n toronto, canada \n 43L-w3b", RRR\n is there any way i can remove \n (newline) from the second... (1 Reply)
Discussion started by: gowrish
1 Replies

10. UNIX for Dummies Questions & Answers

file feed one line per argument

What tools can I use to accomplish this? I'm writing a shell script to analyze an inittab file. Here's a sample file: init:3:initdefault: ioin::sysinit:/sbin/ioinitrc >/dev/console 2>&1 tape::sysinit:/sbin/mtinit > /dev/console 2>&1 muxi::sysinit:/sbin/dasetup </dev/console >/dev/console... (10 Replies)
Discussion started by: jpprial
10 Replies
Login or Register to Ask a Question