awk misreading txt file | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

awk misreading txt file

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 09-16-2008
ccox85 ccox85 is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 12 May 2009, 10:54 AM EDT
Posts: 27
Thanks: 0
Thanked 0 Times in 0 Posts
awk misreading txt file

Good morning,

I am having a problem that I have never had before using awk. I have a txt file that obviously has clear columns and records. When I open the file in textedit, or any other program, it looks as it should, but when I check it in awk by printing the whole thing, it considers everything to be on one line. The file was generated on a mac using OS 9, and I am trying to do the scripting work on a system using OS X, Leopard. I would like to add, however, that I have done this EXACT analysis before and there were no problems reading the text file -- I did a preliminary analysis of the data mid-experiment. Now, the experiment is over, and the files are appended with all the more recent trials, so I recollected them using a flash drive and moved them to my main work station for analysis.

I have tried running DOS to Linux on the files, I have copied the text into new files and saved them, even copied into word, saved as .doc, and then exported to txt. I am baffled, and starting to think it is something awry in my awk build. Does anyone have insight into this issue?

PS, if I convert the txt files to rtf files, and then read them with awk, they look ok, except at the end of each line there is a " \ ". I thought I would just use awk to print everything except for the backslash into a new file, but then nothing can read that new file properly EXCEPT awk. I am pretty irritated. Thank you!
Sponsored Links
    #2  
Old 09-16-2008
Franklin52's Avatar
Franklin52 Franklin52 is offline Forum Staff  
Moderator
 
Join Date: Feb 2007
Last Activity: 20 November 2014, 4:57 PM EST
Location: The Netherlands
Posts: 7,719
Thanks: 123
Thanked 538 Times in 508 Posts
Mac OS places a carriage return character at the end of each line of a text file, but Unix uses a line feed character. Convert the file to a unix format with:


Code:
awk '{gsub("\r", "\n"); print $0;}' macfile.txt > unixfile.txt

Regards
Sponsored Links
    #3  
Old 09-16-2008
ccox85 ccox85 is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 12 May 2009, 10:54 AM EDT
Posts: 27
Thanks: 0
Thanked 0 Times in 0 Posts
Wow, that was easy enough. Thank you so much! And here I was thinking that the mac formatting was the same as Unix... good life lesson.

And just to save myself the frustration of trying and failing, to put it back into a mac-readable format, I would just reverse that:


Code:
awk '{gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txt

correct? Thank you! You definitely saved the day.


PS: Mac can read unix format. Ha, now I see my confusion come full circle.

Last edited by ccox85; 09-16-2008 at 01:01 PM.. Reason: Realization
    #4  
Old 09-16-2008
avronius avronius is offline
Registered User
 
Join Date: Apr 2008
Last Activity: 22 May 2013, 8:38 AM EDT
Location: Calgary
Posts: 317
Thanks: 0
Thanked 0 Times in 0 Posts
If you ever have a question about hidden characters, there's always "vi"
open a file with vi, and then type:


Code:
:set list

This will display any hidden characters (including new line, linefeed, etc.)
To turn off the "display", type:

Code:
:set nolist

Sponsored Links
    #5  
Old 09-16-2008
Franklin52's Avatar
Franklin52 Franklin52 is offline Forum Staff  
Moderator
 
Join Date: Feb 2007
Last Activity: 20 November 2014, 4:57 PM EST
Location: The Netherlands
Posts: 7,719
Thanks: 123
Thanked 538 Times in 508 Posts
Quote:
Originally Posted by ccox85 View Post
And just to save myself the frustration of trying and failing, to put it back into a mac-readable format, I would just reverse that:


Code:
awk '{gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txt

correct?
That's correct.
Sponsored Links
    #6  
Old 09-16-2008
dennis.jacob's Avatar
dennis.jacob dennis.jacob is offline Forum Advisor  
Registered User
 
Join Date: Feb 2007
Last Activity: 30 March 2014, 9:33 AM EDT
Location: Singapore/Cochin
Posts: 871
Thanks: 0
Thanked 12 Times in 11 Posts
Quote:
Originally Posted by avronius View Post
If you ever have a question about hidden characters, there's always "vi"
open a file with vi, and then type:


Code:
:set list

This will display any hidden characters (including new line, linefeed, etc.)
To turn off the "display", type:

Code:
:set nolist


Or you can use

Quote:
vi -b filename
Sponsored Links
    #7  
Old 09-17-2008
ccox85 ccox85 is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 12 May 2009, 10:54 AM EDT
Posts: 27
Thanks: 0
Thanked 0 Times in 0 Posts
Thank you everyone!

With all this help, I completed the project today! I definitely need to get vi up and running on my mac. Doesn't feel very professional working in textedit
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Before I delete any file in Unix, How can I check no open file handle is pointing to that file? kchinnam Solaris 12 10-06-2010 03:30 PM
Reading file names from a file and executing the relative file from shell script anushilrai Shell Programming and Scripting 4 03-10-2006 05:25 AM



All times are GMT -4. The time now is 10:53 PM.