awk misreading txt file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk misreading txt file
# 1  
Old 09-16-2008
awk misreading txt file

Good morning,

I am having a problem that I have never had before using awk. I have a txt file that obviously has clear columns and records. When I open the file in textedit, or any other program, it looks as it should, but when I check it in awk by printing the whole thing, it considers everything to be on one line. The file was generated on a mac using OS 9, and I am trying to do the scripting work on a system using OS X, Leopard. I would like to add, however, that I have done this EXACT analysis before and there were no problems reading the text file -- I did a preliminary analysis of the data mid-experiment. Now, the experiment is over, and the files are appended with all the more recent trials, so I recollected them using a flash drive and moved them to my main work station for analysis.

I have tried running DOS to Linux on the files, I have copied the text into new files and saved them, even copied into word, saved as .doc, and then exported to txt. I am baffled, and starting to think it is something awry in my awk build. Does anyone have insight into this issue?

PS, if I convert the txt files to rtf files, and then read them with awk, they look ok, except at the end of each line there is a " \ ". I thought I would just use awk to print everything except for the backslash into a new file, but then nothing can read that new file properly EXCEPT awk. I am pretty irritated. Thank you!
# 2  
Old 09-16-2008
Mac OS places a carriage return character at the end of each line of a text file, but Unix uses a line feed character. Convert the file to a unix format with:

Code:
awk '{gsub("\r", "\n"); print $0;}' macfile.txt > unixfile.txt

Regards
# 3  
Old 09-16-2008
Wow, that was easy enough. Thank you so much! And here I was thinking that the mac formatting was the same as Unix... good life lesson.

And just to save myself the frustration of trying and failing, to put it back into a mac-readable format, I would just reverse that:

Code:
awk '{gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txt

correct? Thank you! You definitely saved the day.


PS: Mac can read unix format. Ha, now I see my confusion come full circle.

Last edited by ccox85; 09-16-2008 at 01:01 PM.. Reason: Realization
# 4  
Old 09-16-2008
If you ever have a question about hidden characters, there's always "vi"
open a file with vi, and then type:

Code:
:set list

This will display any hidden characters (including new line, linefeed, etc.)
To turn off the "display", type:
Code:
:set nolist

# 5  
Old 09-16-2008
Quote:
Originally Posted by ccox85
And just to save myself the frustration of trying and failing, to put it back into a mac-readable format, I would just reverse that:

Code:
awk '{gsub("\n", "\r"); print $0;}' unixfile.txt > macfile.txt

correct?
That's correct.
# 6  
Old 09-16-2008
Quote:
Originally Posted by avronius
If you ever have a question about hidden characters, there's always "vi"
open a file with vi, and then type:

Code:
:set list

This will display any hidden characters (including new line, linefeed, etc.)
To turn off the "display", type:
Code:
:set nolist


Or you can use

Quote:
vi -b filename
# 7  
Old 09-17-2008
Thank you everyone!

With all this help, I completed the project today! I definitely need to get vi up and running on my mac. Doesn't feel very professional working in textedit Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Awk, sed, shell all words in INPUT.txt find in column1 of TABLE.txt and replce with column2 in

Hi dears i have text file like this: INPUT.txt 001_1_173 j nuh ]az 001_1_174 j ]esma. nuh ]/.xori . . . and have another text like this TABLE.txt j j nuh word1... (6 Replies)
Discussion started by: alii
6 Replies

2. Shell Programming and Scripting

Desired output.txt for reading txt file using awk?

Dear all, I have a huge txt file (DATA.txt) with the following content . From this txt file, I want the following output using some shell script. Any help is greatly appreciated. Greetings, emily DATA.txt (snippet of the huge text file) 407202849... (2 Replies)
Discussion started by: emily
2 Replies

3. Shell Programming and Scripting

Converting txt file into CSV using awk or sed

Hello folks I have a txt file of information about journal articles from different fields. I need to convert this information into a format that is easier for computers to manipulate for some research that I'm doing on how articles are cited. The file has some header information and then details... (8 Replies)
Discussion started by: ksk
8 Replies

4. Shell Programming and Scripting

awk to print value from txt file to csv

Hi, I want to print two columns from a .txt file to a .csv file using awk. data in text file: Application -------------------------------------------------- ----------- OS Related Issues 1 EMEA Solutions ... (8 Replies)
Discussion started by: prashu_g
8 Replies

5. Shell Programming and Scripting

awk append fileA.txt to growing file B.txt

This is appending a column. My question is fairly simple. I have a program generating data in a form like so: 1 20 2 22 3 23 4 12 5 43 For ever iteration I'm generating this data. I have the basic idea with cut -f 2 fileA.txt | paste -d >> FileB.txt ???? I want FileB.txt to grow, and... (4 Replies)
Discussion started by: theawknewbie
4 Replies

6. Shell Programming and Scripting

Help with selecting column with awk for a txt file generated by excel

I am new to scripting/programming, so I apologize for any novice questions. I have a tab delimited text file that was saved from excel xls file. I am trying to select only the third column using awk command. My command line is as below: cat test.txt | awk '{print $3}' However, above... (8 Replies)
Discussion started by: SangLad
8 Replies

7. Shell Programming and Scripting

Using csh / awk / sed to compare database sizes in a txt file

Hello, I have an output file showing database sizes across the 3 environments that I use (LIVE, TEST & DEVELOPMENT). I am trying to write a script that lets me know if the size of a db on one environment is different to its corresponding db on the other environments. Here is an example... (4 Replies)
Discussion started by: stevie_g
4 Replies

8. Shell Programming and Scripting

Select some lines from a txt file and create a new file with awk

Hi there, I have a text file with several colums separated by "|;#" I need to search the file extracting all columns starting with the value of "1" or "2" saving in a separate file just the first 7 columns of each row maching the criteria, with replacement of the saparators in the nearly created... (4 Replies)
Discussion started by: capnino
4 Replies

9. Shell Programming and Scripting

AWK CSV to TXT format, TXT file not in a correct column format

HI guys, I have created a script to read 1 column in a csv file and then place it in text file. However, when i checked out the text file, it is not in a column format... Example: CSV file contains name,age aa,11 bb,22 cc,33 After using awk to get first column TXT file... (1 Reply)
Discussion started by: mdap
1 Replies

10. UNIX for Dummies Questions & Answers

Binary txt file received when i use uuencode to send txt file as attachment

Hi, I have already read a lot of posts on sending attachments in unix...but none of them were of help for my problem...so here goes.. i wanna attach a text file and send to a mail id..used the following code : uuencode "$File1" "$File1" ;|mail -s "$Mail_sub" abc@abc.com it works... (2 Replies)
Discussion started by: ash22
2 Replies
Login or Register to Ask a Question