Unix/Linux Go Back    


Shell Programming and Scripting Unix shell scripting - KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and shell scripts and shell scripting languages here.

awk misreading txt file

Shell Programming and Scripting


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 09-16-2008
ccox85 ccox85 is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 12 May 2009, 10:54 AM EDT
Posts: 27
Thanks: 0
Thanked 0 Times in 0 Posts
awk misreading txt file

Good morning,

I am having a problem that I have never had before using awk. I have a txt file that obviously has clear columns and records. When I open the file in textedit, or any other program, it looks as it should, but when I check it in awk by printing the whole thing, it considers everything to be on one line. The file was generated on a mac using OS 9, and I am trying to do the scripting work on a system using OS X, Leopard. I would like to add, however, that I have done this EXACT analysis before and there were no problems reading the text file -- I did a preliminary analysis of the data mid-experiment. Now, the experiment is over, and the files are appended with all the more recent trials, so I recollected them using a flash drive and moved them to my main work station for analysis.

I have tried running DOS to Linux on the files, I have copied the text into new files and saved them, even copied into word, saved as .doc, and then exported to txt. I am baffled, and starting to think it is something awry in my awk build. Does anyone have insight into this issue?

PS, if I convert the txt files to rtf files, and then read them with awk, they look ok, except at the end of each line there is a " \ ". I thought I would just use awk to print everything except for the backslash into a new file, but then nothing can read that new file properly EXCEPT awk. I am pretty irritated. Thank you!
Sponsored Links
    #2  
Old Unix and Linux 09-16-2008
Franklin52's Unix or Linux Image
Franklin52 Franklin52 is offline Forum Staff  
Moderator
 
Join Date: Feb 2007
Last Activity: 28 September 2016, 5:30 PM EDT
Location: The Netherlands
Posts: 7,746
Thanks: 139
Thanked 558 Times in 520 Posts
Mac OS places a carriage return character at the end of each line of a text file, but Unix uses a line feed character. Convert the file to a unix format with:


Code:
awk '{gsub("\r", "\n"); print $0;}' macfile.txt > unixfile.txt

Regards
Sponsored Links
    #3  
Old Unix and Linux 09-16-2008
ccox85 ccox85 is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 12 May 2009, 10:54 AM EDT
Posts: 27
Thanks: 0
Thanked 0 Times in 0 Posts
Wow, that was easy enough. Thank you so much! And here I was thinking that the mac formatting was the same as Unix... good life lesson.

And just to save myself the frustration of trying and failing, to put it back into a mac-readable format, I would just reverse that:


Code:
awk '{gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txt

correct? Thank you! You definitely saved the day.


PS: Mac can read unix format. Ha, now I see my confusion come full circle.

Last edited by ccox85; 09-16-2008 at 12:01 PM.. Reason: Realization
    #4  
Old Unix and Linux 09-16-2008
avronius avronius is offline
Registered User
 
Join Date: Apr 2008
Last Activity: 22 May 2013, 8:38 AM EDT
Location: Calgary
Posts: 317
Thanks: 0
Thanked 0 Times in 0 Posts
If you ever have a question about hidden characters, there's always "vi"
open a file with vi, and then type:


Code:
:set list

This will display any hidden characters (including new line, linefeed, etc.)
To turn off the "display", type:

Code:
:set nolist

Sponsored Links
    #5  
Old Unix and Linux 09-16-2008
Franklin52's Unix or Linux Image
Franklin52 Franklin52 is offline Forum Staff  
Moderator
 
Join Date: Feb 2007
Last Activity: 28 September 2016, 5:30 PM EDT
Location: The Netherlands
Posts: 7,746
Thanks: 139
Thanked 558 Times in 520 Posts
Quote:
Originally Posted by ccox85 View Post
And just to save myself the frustration of trying and failing, to put it back into a mac-readable format, I would just reverse that:


Code:
awk '{gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txt

correct?
That's correct.
Sponsored Links
    #6  
Old Unix and Linux 09-16-2008
dennis.jacob's Unix or Linux Image
dennis.jacob dennis.jacob is offline Forum Advisor  
Registered User
 
Join Date: Feb 2007
Last Activity: 30 March 2014, 9:33 AM EDT
Location: Singapore/Cochin
Posts: 871
Thanks: 0
Thanked 12 Times in 11 Posts
Quote:
Originally Posted by avronius View Post
If you ever have a question about hidden characters, there's always "vi"
open a file with vi, and then type:


Code:
:set list

This will display any hidden characters (including new line, linefeed, etc.)
To turn off the "display", type:

Code:
:set nolist


Or you can use

Quote:
vi -b filename
Sponsored Links
    #7  
Old Unix and Linux 09-17-2008
ccox85 ccox85 is offline
Registered User
 
Join Date: Dec 2007
Last Activity: 12 May 2009, 10:54 AM EDT
Posts: 27
Thanks: 0
Thanked 0 Times in 0 Posts
Thank you everyone!

With all this help, I completed the project today! I definitely need to get vi up and running on my mac. Doesn't feel very professional working in textedit Linux
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Before I delete any file in Unix, How can I check no open file handle is pointing to that file? kchinnam Solaris 12 10-06-2010 02:30 PM
Reading file names from a file and executing the relative file from shell script anushilrai Shell Programming and Scripting 4 03-10-2006 04:25 AM



All times are GMT -4. The time now is 04:20 AM.