The Format_trial.csv.awk script;

Code:

#!/bin/gawk -f


# This file is to restructure the picarro data into the correct .csv columns for R


# create a header with same headings as variable in table
# also set other variables before parsing data
BEGIN   {
        OFS="," 	# tells awk that the output separator is a comma
        ORS=""  	# tells awk to not print newline after each print command so all records are
					# on the same line until we want a new line "\n"			
		getline}	# removes 1st line of input file ie header so we can replace it with correct one

# rearrange the yyyy-mm-dd | hh:mm:ss date and time to single date column of yyyy/mm/dd hh:mm:ss needed for openair
{print substr($1,9,2) "/" substr($1,6,2) "/" substr($1,1,4) " " substr($2,1,5)}		

# print the rest of variables as columns 
{print (" ", $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20)
}


$1 $2 != prev {
    	
	{print "\n"}	# newline after each 5 seconds of data has been parsed
  
; prev=$1 $2}

Yes this awk script works on one file, it creates a combined .csv file with the correct format. The problem is trying to input all the files at once (with one command) as opposed to having to call in every days file.

Yes the output (for the 'new' combined script) is just an empty 'june.csv' file, maybe it's not recognising the june.dat as the input?

The desired data file is just a data frame containing 20 variables. The awk file was essentially used to combine the data and time columns into one column with the correct format I need to input into an R package.

Cheers guys!

---------- Post updated at 01:59 PM ---------- Previous update was at 01:52 PM ----------

Ah I see where I went wrong. I was using 'IN_ALL' to call in the file and then 'IN_all' to produce the file! My how I'm learning how precise programming can be!

Another quick thing guys, this programme is going to be run everyday to create the .csv file. However, whenever I run it, the data is just added to the old data, so I essentially have the same results twice if I run the programme again, and 3 times if I run it again and so on. Is there anyway to tell awk to write over the old file?