Sponsored Content
Top Forums Shell Programming and Scripting Sed or awk script to remove text / or perform calculations from large CSV files Post 302326310 by ahmad.diab on Wednesday 17th of June 2009 03:49:09 PM
Old 06-17-2009
Quote:
Originally Posted by metronomadic
Thanks Ahmad. I tried the awk code (which I think needs an extra } to close out the for loop?), but I think that might be calculating something else. I am trying to get the average flow (column three) for each minute (or each 15 minute span) of each day. I am not sure I understand the code, but from the output it looks like it is gathering each days worth of records, and dividing them by the number of days?

I don't mean to be a bother, but can you tell me if this is what is going on?
sorry Smilie kindly add the bold string below:-

awk -F"," ' BEGIN{c=0} {a[$1]+=$2;b[$1]+=$3;c++} END{for (i in a) {print "Total", a[i]/c,b[i]/c} ' file.txt
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to perform calculations using numbers greater than 2150000000.

Could someone tell me how to perform calculations using numbers greater than 2150000000 in Korn Shell? When I tried to do it it gave me the wrong answer. e.g. I have a ksh file with the contents below: --------------------------------- #!/bin/ksh SUM=`expr 2150000000 + 2` PRODUCT=`expr... (3 Replies)
Discussion started by: stevefox
3 Replies

2. Shell Programming and Scripting

calculations in awk/sed

Hi friends, I am in a little confusion. I have a comma separated file contains hugh records. say, 88562848,21-JAN-08,2741079, -1188,-7433,TESTING 88558314,21-JAN-08,2741189, -1273,-7976,TESTING I want 4th and 5th field to be multiplied by 100. like, 88562848,21-JAN-08,2741079,... (3 Replies)
Discussion started by: clx
3 Replies

3. Shell Programming and Scripting

Remove text from a csv file using sed

I am trying to remove the ita from this file: "1234ita","john","smith" "56789ita","jim","thomas" the command i am using is: sed '/^ita/d' infile.csv > outfile.csv it is running but doing nothing, very rarely use sed so trying to learn it any help would be appreciated (2 Replies)
Discussion started by: Pablo_beezo
2 Replies

4. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

5. Shell Programming and Scripting

Need help with a script to process a CSV file using SED and AWK

I get a CSV file every day with 2 columns and multiple rows ex: date1,date2 ( both the fields are varchar fields) This data has to be updated in a table which is being done manually and i want to automate that. 1. I have to select all the data from the prod table( 2 columns { date1,date2}) into... (4 Replies)
Discussion started by: kkb
4 Replies

6. Programming

awk script to convert a text file into csv format

hi...... thanks for allowing me to start a discussion i am collecting usb usage details of all users and convert it into csv files so that i can export it into some database.. the input text file is as follows:- USB History Dump by nabiy (c)2008 (1) --- Kingston DataTraveler 130 USB... (2 Replies)
Discussion started by: certteam
2 Replies

7. Shell Programming and Scripting

shell script to take input from a text file and perform check on each servers and copy files

HI all, I want to script where all the server names will be in a text file like server1 server2 server3 . and the script should take servernames from a text file and perform copy of files if the files are not present on those servers.after which it should take next servername till the end of... (0 Replies)
Discussion started by: joseph.dmello
0 Replies

8. Emergency UNIX and Linux Support

Help to make awk script more efficient for large files

Hello, Error awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt What it is It is a unix shell script that contains an awk program as well as... (4 Replies)
Discussion started by: script_op2a
4 Replies

9. Shell Programming and Scripting

awk script to perform an action similar to vlookup between two csv files in UNIX

Hi, I am new to awk/unix and am trying to put together an awk script to perform an action similar to vlookup between the two csv files. Here are the contents of the two files: File 1: Date,ParentID,Number,Area,Volume,Dimensions 2014-01-01,ABC,247,83430.33,857.84,8110.76... (9 Replies)
Discussion started by: Prit Siv
9 Replies

10. Shell Programming and Scripting

Filter on one column and then perform conditional calculations on another column with a Linux script

Hi, I have a file (stats.txt) with columns like in the example below. Destination IP address, timestamp, TCP packet sequence number and packet length. destIP time seqNo packetLength 1.2.3.4 0.01 123 500 1.2.3.5 0.03 44 1500 1.3.2.5 0.08 44 1500 1.2.3.4 0.44... (12 Replies)
Discussion started by: Zooma
12 Replies
GMMKTIME(3)								 1							       GMMKTIME(3)

gmmktime - Get Unix timestamp for a GMT date

SYNOPSIS
int gmmktime ([int $hour = gmdate("H")], [int $minute = gmdate("i")], [int $second = gmdate("s")], [int $month = gmdate("n")], [int $day = gmdate("j")], [int $year = gmdate("Y")], [int $is_dst = -1]) DESCRIPTION
Identical to mktime(3) except the passed parameters represents a GMT date. gmmktime(3) internally uses mktime(3) so only times valid in derived local time can be used. Like mktime(3), arguments may be left out in order from right to left, with any omitted arguments being set to the current corresponding GMT value. PARAMETERS
o $hour - The number of the hour relative to the start of the day determined by $month, $day and $year. Negative values reference the hour before midnight of the day in question. Values greater than 23 reference the appropriate hour in the following day(s). o $minute - The number of the minute relative to the start of the $hour. Negative values reference the minute in the previous hour. Values greater than 59 reference the appropriate minute in the following hour(s). o $second - The number of seconds relative to the start of the $minute. Negative values reference the second in the previous minute. Values greater than 59 reference the appropriate second in the following minute(s). o $month - The number of the month relative to the end of the previous year. Values 1 to 12 reference the normal calendar months of the year in question. Values less than 1 (including negative values) reference the months in the previous year in reverse order, so 0 is December, -1 is November, etc. Values greater than 12 reference the appropriate month in the following year(s). o $day - The number of the day relative to the end of the previous month. Values 1 to 28, 29, 30 or 31 (depending upon the month) refer- ence the normal days in the relevant month. Values less than 1 (including negative values) reference the days in the previous month, so 0 is the last day of the previous month, -1 is the day before that, etc. Values greater than the number of days in the relevant month reference the appropriate day in the following month(s). o $year - The year o $is_dst - Parameters always represent a GMT date so $is_dst doesn't influence the result. Note This parameter has been removed in PHP 7.0.0. RETURN VALUES
Returns a integer Unix timestamp. CHANGELOG
+--------+---------------------------------------------------+ |Version | | | | | | | Description | | | | +--------+---------------------------------------------------+ | 7.0.0 | | | | | | | $is_dst parameter has been removed. | | | | | 5.1.0 | | | | | | | As of PHP 5.1.0, the $is_dst parameter became | | | deprecated. As a result, the new timezone han- | | | dling features should be used instead. | | | | +--------+---------------------------------------------------+ EXAMPLES
Example #1 gmmktime(3) basic example <?php // Prints: July 1, 2000 is on a Saturday echo "July 1, 2000 is on a " . date("l", gmmktime(0, 0, 0, 7, 1, 2000)); ?> SEE ALSO
mktime(3), date(3), time(3). PHP Documentation Group GMMKTIME(3)
All times are GMT -4. The time now is 02:37 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy