Mac script - issue with tab delimited input file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Mac script - issue with tab delimited input file
# 1  
Old 08-18-2011
Mac script - issue with tab delimited input file

Hi:

I'm writing a script that will take source / destination pathnames and metadata information from a tab-delimited input file, and then perform various directory creation, file moving and renaming, and tagging of files.

I think I have what I need to do the file manipulation in the script - the problem I am having is with the input file, which is being created in Excel on my MacBook.

The basic loop I'm using in the script is:
Code:
#!/bin/bash

while read inputline
do
	echo $inputline
done < inputfile.txt

exit 0

My input file contains: Source path/filename, Destination path/filename, Date, and Keywords, separated by tabs.

When I save the file in Mac Excel, the lines are terminated with a ^M character on all except the last line, which brings me to my problem. The script above ignores the last line in the input file. Interestingly enough, when I create the exact same file in PC Excel, the final line terminator does come through, and the script handles it fine. Unfortunately (for me), I'm using my MacBook exclusively to develop the spreadsheet, and it looks like I'll have to live with the bug.

My question is - is there a way to modify the script above, so that it will take the last line as input (without the terminator), or can the script check for existence of a final line terminator and add one if needed before executing the loop? My script will be run numerous times against numerous copies of the spreadsheet, so remembering to add it manually isn't going to be very effective.

Thanks in advance for any help!

Last edited by pludi; 08-18-2011 at 04:13 PM..
# 2  
Old 08-18-2011
I suspect the last line not only has no ^M, but no newline as well. Not just bash read, but many shell utilities such as awk and sed can't be depended on to read a last line which has no ending newline.

Fortunately it's easy enough to add it.

Code:
cp input /tmp/$$
printf "\n" >> /tmp/$$

while read LINE
do
        # If we added an extra blank line, ignore it
        [ -z "$LINE" ] && continue
done < /tmp/$$
rm -f /tmp/$$

# 3  
Old 08-18-2011
Quote:
Originally Posted by Corona688
I suspect the last line not only has no ^M, but no newline as well. Not just bash read, but many shell utilities such as awk and sed can't be depended on to read a last line which has no ending newline.

Fortunately it's easy enough to add it.

Code:
cp input /tmp/$$
printf "\n" >> /tmp/$$

while read LINE
do
        # If we added an extra blank line, ignore it
        [ -z "$LINE" ] && continue
done < /tmp/$$
rm -f /tmp/$$

Ok - trying to unscramble this...
- first lines make a temp copy of the input file and put a newline character at the end
- next section loops through the file

I'm not clear on what the line '[ -z "$LINE" ] && continue' does. Is it jumping to the 'done' line, which would skip my processing code?

Thanks!
# 4  
Old 08-18-2011
Quote:
Originally Posted by GRIMESPACE
Ok - trying to unscramble this...
- first lines make a temp copy of the input file and put a newline character at the end
- next section loops through the file

I'm not clear on what the line '[ -z "$LINE" ] && continue' does. Is it jumping to the 'done' line, which would skip my processing code?
It would, but only if the line is completely, utterly blank anyway. That's what -z means.

The idea is, if we add an extra newline to a file which doesn't need one, that won't add an extra line -- and an invalid one, at that -- to your processing.
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 08-18-2011
Quote:
Originally Posted by Corona688
It would, but only if the line is completely, utterly blank anyway. That's what -z means.

The idea is, if we add an extra newline to a file which doesn't need one, that won't add an extra line -- and an invalid one, at that -- to your processing.
Just tried it - looks like exactly what I needed. Thank you!!!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace a column in tab delimited file with column in other tab delimited file,based on match

Hello Everyone.. I want to replace the retail col from FileI with cstp1 col from FileP if the strpno matches in both files FileP.txt ... (2 Replies)
Discussion started by: YogeshG
2 Replies

2. UNIX for Dummies Questions & Answers

Need to convert a pipe delimited text file to tab delimited

Hi, I have a rquirement in unix as below . I have a text file with me seperated by | symbol and i need to generate a excel file through unix commands/script so that each value will go to each column. ex: Input Text file: 1|A|apple 2|B|bottle excel file to be generated as output as... (9 Replies)
Discussion started by: raja kakitapall
9 Replies

3. UNIX for Dummies Questions & Answers

Need help with tab delimited file in unix

Hi, I need urgent help with a tab delimited file I am working on. This is the file : TTTT|YYYYYYY|jargon-journal|MP0000000UID||"j1, j2, j3" I need th following output: TTTT|YYYYYYY|jargon-journal|MP0000000UID||ji TTTT|YYYYYYY|jargon-journal|MP0000000UID||j2... (8 Replies)
Discussion started by: rayarnab
8 Replies

4. Shell Programming and Scripting

How to make tab delimited file to space delimited?

Hi How to make tab delimited file to space delimited? in put file: ABC kgy jkh ghj ash kjl o/p file: ABC kgy jkh ghj ash kjl Use code tags, thanks. (1 Reply)
Discussion started by: jagdishrout
1 Replies

5. Homework & Coursework Questions

Shell Script to read a tab delimited file and perform simple tasks

1. The problem statement, all variables and given/known data: Hello! I need help with this problem bash shell scripting that basically just reads the data in a tab delimited file and does the following below 1. Read in the data file Survey.txt and assign the column values to variables of... (6 Replies)
Discussion started by: jsmith6932
6 Replies

6. Shell Programming and Scripting

Help with converting Pipe delimited file to Tab Delimited

I have a file which was pipe delimited, I need to make it tab delimited. I tried with sed but no use cat file | sed 's/|//t/g' The above command substituted "/t" not tab in the place of pipe. Sample file: abc|123|2012-01-30|2012-04-28|xyz have to convert to: abc 123... (6 Replies)
Discussion started by: karumudi7
6 Replies

7. UNIX for Dummies Questions & Answers

tab delimited file that is not tab delimited.

Hi Forum I have a tab delimited file that opens well in Openoffice calc (excel). But when I perform any operation in command line, it reads the file incorrectly. When I 'save As' the same file in office as tab delimited then it works fine. The file that I think is tab delimited is actually... (8 Replies)
Discussion started by: imlearning
8 Replies

8. Shell Programming and Scripting

Retrieving values from tab-delimited file in unix script

Hi I am trying to retrieve values from a tab-delimited file.I am using while read record value=`echo $record | cut -f12` done Where 12 is the column no i want retieve and record is one line of the file. But it is returning the full record. Plz help (4 Replies)
Discussion started by: akashtcs
4 Replies

9. UNIX for Dummies Questions & Answers

Converting Space delimited file to Tab delimited file

Hi all, I have a file with single white space delimited values, I want to convert them to a tab delimited file. I tried sed, tr ... but nothing is working. Thanks, Rajeevan D (16 Replies)
Discussion started by: jeevs81
16 Replies

10. Shell Programming and Scripting

Converting Tab delimited file to Comma delimited file in Unix

Hi, Can anyone let me know on how to convert a Tab delimited file to Comma delimited file in Unix Thanks!! (22 Replies)
Discussion started by: charan81
22 Replies
Login or Register to Ask a Question