Sponsored Content
Top Forums Shell Programming and Scripting Sed or awk script to remove text / or perform calculations from large CSV files Post 302326310 by ahmad.diab on Wednesday 17th of June 2009 03:49:09 PM
Old 06-17-2009
Quote:
Originally Posted by metronomadic
Thanks Ahmad. I tried the awk code (which I think needs an extra } to close out the for loop?), but I think that might be calculating something else. I am trying to get the average flow (column three) for each minute (or each 15 minute span) of each day. I am not sure I understand the code, but from the output it looks like it is gathering each days worth of records, and dividing them by the number of days?

I don't mean to be a bother, but can you tell me if this is what is going on?
sorry Smilie kindly add the bold string below:-

awk -F"," ' BEGIN{c=0} {a[$1]+=$2;b[$1]+=$3;c++} END{for (i in a) {print "Total", a[i]/c,b[i]/c} ' file.txt
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to perform calculations using numbers greater than 2150000000.

Could someone tell me how to perform calculations using numbers greater than 2150000000 in Korn Shell? When I tried to do it it gave me the wrong answer. e.g. I have a ksh file with the contents below: --------------------------------- #!/bin/ksh SUM=`expr 2150000000 + 2` PRODUCT=`expr... (3 Replies)
Discussion started by: stevefox
3 Replies

2. Shell Programming and Scripting

calculations in awk/sed

Hi friends, I am in a little confusion. I have a comma separated file contains hugh records. say, 88562848,21-JAN-08,2741079, -1188,-7433,TESTING 88558314,21-JAN-08,2741189, -1273,-7976,TESTING I want 4th and 5th field to be multiplied by 100. like, 88562848,21-JAN-08,2741079,... (3 Replies)
Discussion started by: clx
3 Replies

3. Shell Programming and Scripting

Remove text from a csv file using sed

I am trying to remove the ita from this file: "1234ita","john","smith" "56789ita","jim","thomas" the command i am using is: sed '/^ita/d' infile.csv > outfile.csv it is running but doing nothing, very rarely use sed so trying to learn it any help would be appreciated (2 Replies)
Discussion started by: Pablo_beezo
2 Replies

4. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

5. Shell Programming and Scripting

Need help with a script to process a CSV file using SED and AWK

I get a CSV file every day with 2 columns and multiple rows ex: date1,date2 ( both the fields are varchar fields) This data has to be updated in a table which is being done manually and i want to automate that. 1. I have to select all the data from the prod table( 2 columns { date1,date2}) into... (4 Replies)
Discussion started by: kkb
4 Replies

6. Programming

awk script to convert a text file into csv format

hi...... thanks for allowing me to start a discussion i am collecting usb usage details of all users and convert it into csv files so that i can export it into some database.. the input text file is as follows:- USB History Dump by nabiy (c)2008 (1) --- Kingston DataTraveler 130 USB... (2 Replies)
Discussion started by: certteam
2 Replies

7. Shell Programming and Scripting

shell script to take input from a text file and perform check on each servers and copy files

HI all, I want to script where all the server names will be in a text file like server1 server2 server3 . and the script should take servernames from a text file and perform copy of files if the files are not present on those servers.after which it should take next servername till the end of... (0 Replies)
Discussion started by: joseph.dmello
0 Replies

8. Emergency UNIX and Linux Support

Help to make awk script more efficient for large files

Hello, Error awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt What it is It is a unix shell script that contains an awk program as well as... (4 Replies)
Discussion started by: script_op2a
4 Replies

9. Shell Programming and Scripting

awk script to perform an action similar to vlookup between two csv files in UNIX

Hi, I am new to awk/unix and am trying to put together an awk script to perform an action similar to vlookup between the two csv files. Here are the contents of the two files: File 1: Date,ParentID,Number,Area,Volume,Dimensions 2014-01-01,ABC,247,83430.33,857.84,8110.76... (9 Replies)
Discussion started by: Prit Siv
9 Replies

10. Shell Programming and Scripting

Filter on one column and then perform conditional calculations on another column with a Linux script

Hi, I have a file (stats.txt) with columns like in the example below. Destination IP address, timestamp, TCP packet sequence number and packet length. destIP time seqNo packetLength 1.2.3.4 0.01 123 500 1.2.3.5 0.03 44 1500 1.3.2.5 0.08 44 1500 1.2.3.4 0.44... (12 Replies)
Discussion started by: Zooma
12 Replies
PARSE_TIME(3)						   BSD Library Functions Manual 					     PARSE_TIME(3)

NAME
parse_time, print_time_table, unparse_time, unparse_time_approx, -- parse and unparse time intervals LIBRARY
The roken library (libroken, -lroken) SYNOPSIS
#include <parse_time.h> int parse_time(const char *timespec, const char *def_unit); void print_time_table(FILE *f); size_t unparse_time(int seconds, char *buf, size_t len); size_t unparse_time_approx(int seconds, char *buf, size_t len); DESCRIPTION
The parse_time() function converts a the period of time specified in into a number of seconds. The timespec can be any number of <number unit> pairs separated by comma and whitespace. The number can be negative. Number without explicit units are taken as being def_unit. The unparse_time() and unparse_time_approx() does the opposite of parse_time(), that is they take a number of seconds and express that as human readable string. unparse_time produces an exact time, while unparse_time_approx restricts the result to only include one units. print_time_table() prints a descriptive list of available units on the passed file descriptor. The possible units include: second, s minute, m hour, h day week seven days month 30 days year 365 days Units names can be arbitrarily abbreviated (as long as they are unique). RETURN VALUES
parse_time() returns the number of seconds that represents the expression in timespec or -1 on error. unparse_time() and unparse_time_approx() return the number of characters written to buf. if the return value is greater than or equal to the len argument, the string was too short and some of the printed characters were discarded. EXAMPLES
#include <stdio.h> #include <parse_time.h> int main(int argc, char **argv) { int i; int result; char buf[128]; print_time_table(stdout); for (i = 1; i < argc; i++) { result = parse_time(argv[i], "second"); if(result == -1) { fprintf(stderr, "%s: parse error ", argv[i]); continue; } printf("-- "); printf("parse_time = %d ", result); unparse_time(result, buf, sizeof(buf)); printf("unparse_time = %s ", buf); unparse_time_approx(result, buf, sizeof(buf)); printf("unparse_time_approx = %s ", buf); } return 0; } $ ./a.out "1 minute 30 seconds" "90 s" "1 y -1 s" 1 year = 365 days 1 month = 30 days 1 week = 7 days 1 day = 24 hours 1 hour = 60 minutes 1 minute = 60 seconds 1 second -- parse_time = 90 unparse_time = 1 minute 30 seconds unparse_time_approx = 1 minute -- parse_time = 90 unparse_time = 1 minute 30 seconds unparse_time_approx = 1 minute -- parse_time = 31535999 unparse_time = 12 months 4 days 23 hours 59 minutes 59 seconds unparse_time_approx = 12 months BUGS
Since parse_time() returns -1 on error there is no way to parse "minus one second". Currently "s" at the end of units is ignored. This is a hack for English plural forms. If these functions are ever localised, this scheme will have to change. HEIMDAL
October 31, 2004 HEIMDAL
All times are GMT -4. The time now is 11:14 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy