Confused about redirecting output from awk.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Confused about redirecting output from awk.
# 1  
Old 05-19-2008
Confused about redirecting output from awk.

I have a simple script written in awk whose purpose is to go through some php files and replace some strings. Naturally, I want the changes to be written out to the files. The script looks generally as follows:
Code:
{ 
gsub(/'replacethis'/, "with this");  # a bunch of these

print $0 > FILENAME
}

This does the replacements just fine, and writes them out to the original file, as desired. However, for files over 4096 bytes, the file is truncated to 4096 bytes. Additionally, if I change the 'print $0 > FILENAME' line to just 'print $0', it outputs the entire file into the terminal. The problem only appears to exist when writing into a file. Why is this, and what can I do about it?

Oh also, I forgot to mention that I need to do the output redirection inside of the awk script, because I run it as follows, as I want it run on every .php file.
Code:
find . -name "*.php" | xargs awk  -f fixdb.awk

# 2  
Old 05-20-2008
Quote:
Originally Posted by face1
... Why is this, and what can I do about it? ...
You need to use temporary files, otherwise you're going to hit OS limits:

Code:
find . -name "*.php" | xargs -i awk '{f=FILENAME; gsub(/replacethis/, "with this"); print > f".tmp"} END{printf ("mv  %s %s\n", f".tmp", f ) | "sh"}' '{}'

Insert as many gsub functions in the awk statement as needed.
# 3  
Old 05-20-2008
Hey, thanks! I haven't gotten a chance to try it yet, but I assume it works. Could you please explain why it works, and mine doesn't? I would think that as the program parses every line, it would write out the line to the new file, so as long as the line isn't more than 4096 bytes long.

EDIT: Okay, I see that your command is different in that it calls the awk script once for each file, rather than all of the files at once, and is thus able to use a relevant END block to rename the .tmp files to the original file. But why I can I not replace the text in one go, as I was trying to do?

Last edited by face1; 05-20-2008 at 03:15 PM..
# 4  
Old 05-20-2008
Actually, scratch my last reply. I'm using basically the command suggested to me above, but I'm still having the same issue. The files are still getting cut off.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Confused about redirecting stderr

I know that mmmmm 2> error.txt will send the error message to the specified file instead of the screen. However, I have seen >&2 in some scripts, and I can't get it to do anything. A source said it sends stdout and stderr to a file. What file? Ubuntu 18.04.2; Xfce 4.12.3;... (11 Replies)
Discussion started by: Xubuntu56
11 Replies

2. Shell Programming and Scripting

Redirecting output using if with awk

I have this line were I am selecting some fields from one file and creating a new file for the selected data. awk -F "\"*,\"*" '{print $1"," $2}' folderone/list.txt > folderone/qlist.txt This works, but then from this new file I want it to create a new file where it separates data: $2 >5 it... (2 Replies)
Discussion started by: rluna
2 Replies

3. Shell Programming and Scripting

Redirecting output to file

Hi, I have created script which redirect the output to file.I am able to get the output in file but not in the format. Output :Content of the log which have 10 -15 lines. Actal :Line1 ..Line 2Line3 Line4 Line 5 Expected:Line1 Line 2 Line3 Please... (7 Replies)
Discussion started by: karthik771
7 Replies

4. Shell Programming and Scripting

Awk: Print Error While Redirecting output in multiple Files

Hi, I have a following code in which I am unable to redirect to multiple files. Can anybody please help with some corrections awk -F, '{ if ( substr($1,26,2)=="02" && substr($1,184,14)=="MTSCC_VALFIRST") { array1++ array2++ array3++ } else if (substr($1,26,2)=="03" &&... (4 Replies)
Discussion started by: siramitsharma
4 Replies

5. Shell Programming and Scripting

Redirecting the output

For example, if we run the below command, symcfg list -thin -pool , results in an output most of the times and if the out is generated i'm able to redirect the output to a file. but sometimes it doesnt result any output and even though the output is being redirected, i can see "No Thin Pools "... (2 Replies)
Discussion started by: web2moha
2 Replies

6. Shell Programming and Scripting

redirecting output using if-then-else in awk

I am trying to filter records based on number of "|", delimiter in my application. If number of "|" is greater than 14 its a bad record, else its a good record. I have to redirect output to two different files based on the if-then-else evaluation in AWK. if number of “|” in file_0 > 14 ... (2 Replies)
Discussion started by: siteregsam
2 Replies

7. UNIX for Dummies Questions & Answers

redirecting script output

Hello, I am interested in taking the output from a script i wrote and using it as input to a different script i wrote. So for example i want to take the output from program2 and use it as a parameter for program1. I didnt think i could use the >> symbols because i think that is just for .txt... (4 Replies)
Discussion started by: GmGeubt
4 Replies

8. Shell Programming and Scripting

Redirecting to different output files with awk.

Well, it didn't take me long to get stumped again. I assure you that I'm not mentally deficient, just new to scripting. So, here's the gist. I want to redirect output from awk based off of which branch of an if-else statement under which it falls. #!/bin/bash #some variables... (2 Replies)
Discussion started by: mikesimone
2 Replies

9. Shell Programming and Scripting

Redirecting OUTPUT

Hi, I want to move the output of a command/script to a file as well as to to be displayed on stdout. Can anybody help me in this. Thanks in advace .. -Chanakya M (1 Reply)
Discussion started by: Chanakya.m
1 Replies

10. UNIX for Dummies Questions & Answers

confused on ls -l output

Here is my output from ls -l: drwxr-xr-x 2 username username 4096 2005-12-27 09:51 bin drwxr-xr-x 2 username username 4096 2006-01-30 13:03 Desktop drwxr-xr-x 1 root root 4096 2006-01-30 09:13 somedirectory drwxr-xr-x 6 username username 4096 2005-12-15 08:09 mystuff drwxr-xr-x ... (2 Replies)
Discussion started by: outtacontrol
2 Replies
Login or Register to Ask a Question