Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

awk and redirecting to files

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers awk and redirecting to files
# 1  
Old 05-21-2008
awk and redirecting to files
Hello,
I have a set of data (comma seperated) that I want to save to multiple files according to one of the fields in the data set.

I can easily do this with the following script:

BEGIN { FS = OFS = ","}
NF {print $0 >> ($2 "" ".csv")}

It works perfectly on a set of dummy data I have set up but when i go to use the actual set i get the following error

gawk: cmd. line:2: (FILENAME=shouldwork.txt FNR=2) fatal: expression for `>' redirection has null string value

I've tried using if value is null then skip etc but to no avail.

Any suggestions would be greatly appreciated.
# 2  
Old 05-21-2008
If you open many files and don't close them, the program would run out of available open files.
Try to close a file before you open a new file.

Regards
# 3  
Old 05-21-2008
Thanks, but I don't have any of the associated files open when I run the script.

It works on one set of data typed out in notepad but fails to work on the required set of data. It's as if there is an error/null value in the data that i can't see using notepad etc but awk does when using this command

First few lines of the set of data as pasted from word pad. Final goal is to extract data to files eg. A140 - Address 00.csv that has all fields associated with this line.

249,A140 - Address 00,1,1,NULL,2008-04-01 17:01:30.857,0
250,A142 - Address 01,1,NULL,1,2008-04-01 17:01:30.950,0
250,A142 - Address 01,1,NULL,1,2008-04-01 17:01:31.013,81
251,A142A - Address 02,1,NULL,1,2008-04-01 17:01:31.043,0
251,A142A - Address 02,1,NULL,1,2008-04-01 17:01:31.060,81
252,A140 - Address 03,1,1,NULL,2008-04-01 17:01:31.123,0
253,A142 - Address 04,1,NULL,1,2008-04-01 17:01:31.247,0
253,A142 - Address 04,1,NULL,1,2008-04-01 17:01:31.263,81
254,E208 1,1,1,1,2008-04-01 17:01:31.873,0
254,E208 1,1,1,1,2008-04-01 17:01:31.890,81
255,E208 2,1,1,1,2008-04-01 17:01:31.950,0
255,E208 2,1,1,1,2008-04-01 17:01:31.950,81


Regards
# 4  
Old 05-22-2008
I don't think that you have posted your actual code because it would not generate the error. Even if $2 was null, the constant ".csv" would prevent that error message.
# 5  
Old 05-22-2008
If you redirect the output to many files you have to close those files, try something like this (not tested):

Code:
awk 'BEGIN { FS = OFS = ","}
NF && f != $2{close(f "" ".csv");f=$2}
NF {print $0 >> (f "" ".csv")}' file

Regards
# 6  
Old 05-22-2008
Cheers guys thanks for the information but still hitting the same error message, using Franklin52 suggestion:

gawk: cmd. line:3: (FILENAME=SampleFile.csv FNR=1) fatal: expression for `>>' redirection has null string value

Ygor, this is the actual code that i had in my script file, and typed the above directily into Msys.
# 7  
Old 05-23-2008
I have been doing abit more digging around and have obtained a new set of data and the same problem exists

Original data from notepad

423,111X,1,NULL,1,2008-04-0X,0
423,111X,1,NULL,1,2008-04-0X,39
423,111X,1,NULL,1,2008-04-0Y,38
423,111X,1,NULL,1,2008-04-0Z,0
423,111X,1,NULL,1,2008-04-0AA,25

Printing the same lines using awk results in
$ awk ' FNR <= 5 { print $0 }' GCTestData.csv
ÿþ423,111X,1,NULL,1,2008-04-01,0
423,111X,1,NULL,1,2008-04-01,39
423,111X,1,NULL,1,2008-04-03,28
423,111X,1,NULL,1,2008-04-08,0
423,111X,1,NULL,1,2008-04-09,25

As can be seen ÿþ is added to the first line. Where did it come from and is this the possible cause of my problem?
Any help would be greatly appreciated
Thanking in advance
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Redirecting output using if with awk

I have this line were I am selecting some fields from one file and creating a new file for the selected data. awk -F "\"*,\"*" '{print $1"," $2}' folderone/list.txt > folderone/qlist.txt This works, but then from this new file I want it to create a new file where it separates data: $2 >5 it... (2 Replies)
Discussion started by: rluna
2 Replies

2. Shell Programming and Scripting

Awk: Print Error While Redirecting output in multiple Files

Hi, I have a following code in which I am unable to redirect to multiple files. Can anybody please help with some corrections awk -F, '{ if ( substr($1,26,2)=="02" && substr($1,184,14)=="MTSCC_VALFIRST") { array1++ array2++ array3++ } else if (substr($1,26,2)=="03" &&... (4 Replies)
Discussion started by: siramitsharma
4 Replies

3. Shell Programming and Scripting

awk script issue redirecting to multiple files after matching pattern

Hi All I am having one awk and sed requirement for the below problem. I tried multiple options in my sed or awk and right output is not coming out. Problem Description ############################################################### I am having a big file say file having repeated... (4 Replies)
Discussion started by: kshitij
4 Replies

4. Shell Programming and Scripting

redirecting output using if-then-else in awk

I am trying to filter records based on number of "|", delimiter in my application. If number of "|" is greater than 14 its a bad record, else its a good record. I have to redirect output to two different files based on the if-then-else evaluation in AWK. if number of “|” in file_0 > 14 ... (2 Replies)
Discussion started by: siteregsam
2 Replies

5. Shell Programming and Scripting

Redirecting to different output files with awk.

Well, it didn't take me long to get stumped again. I assure you that I'm not mentally deficient, just new to scripting. So, here's the gist. I want to redirect output from awk based off of which branch of an if-else statement under which it falls. #!/bin/bash #some variables... (2 Replies)
Discussion started by: mikesimone
2 Replies

6. Shell Programming and Scripting

Printing and redirecting files

Hello, I am curious. If I have a line of code run in bash VAR=" It is a nice shiny day " and I would like to be able to print this with the newlines in tact, is there a way? Are the newlines actually there but stripped by echo $VAR or echo -e $VAR? In Python you can:>>>VAR = """... (4 Replies)
Discussion started by: Narnie
4 Replies

7. Shell Programming and Scripting

Redirecting columns from multiple files

Hi all, I have two years worth of daily files (e.g. 090107.dat) and I'd like to write a BASH script to extract the same 2 columns from each of these files and copy them all to one separate file. Could anyone please point me in the right direction as I'm new to shell scripting? It would... (3 Replies)
Discussion started by: larrymuli
3 Replies

8. Shell Programming and Scripting

Confused about redirecting output from awk.

I have a simple script written in awk whose purpose is to go through some php files and replace some strings. Naturally, I want the changes to be written out to the files. The script looks generally as follows: { gsub(/'replacethis'/, "with this"); # a bunch of these print $0 > FILENAME }... (3 Replies)
Discussion started by: face1
3 Replies

9. Shell Programming and Scripting

Redirecting list of files to cp

I am trying to take a list of filenames and copy those files from one directory into another. Is there a way I can do a 'more' of the file with the filenames and redirect or pipe the output into a 'cp' command. I am using a bash shell. I apologise in advance for what is probably a very simple... (2 Replies)
Discussion started by: markp
2 Replies

10. Shell Programming and Scripting

using redirecting from awk

Well this is what im doing, im writing a script that you pass 3 variables into. Filename, delimiter or "FS in AWK", and a string of columbs you want to keep 1,2,4,5... Just modifing a data file and rewriting with a different extension. My problem atm is using awk to seperate the "columb String"... (2 Replies)
Discussion started by: ugh
2 Replies
Login or Register to Ask a Question