Sponsored Content
Full Discussion: Data processing using awk
Top Forums Shell Programming and Scripting Data processing using awk Post 302755597 by shadyuk on Sunday 13th of January 2013 09:07:19 PM
Old 01-13-2013
Both very clever indeed. Output is correct if theres only one column. However, a little more assistance please. If the csv is as follow, how do I get it to convert the bitrate columns (data1, data2, data3) without having to parse the file first?

Code:
timestamp,data1,data2,data3
08/01/13 16:31:00,1M07,786,896K
08/01/13 16:31:10,1K23,2K44,4M34
08/01/13 16:31:20,345,2M84,437K

Don - Your method only outputs if the data is in the first column.
Chubler - Your method outputs the wrong data.

Code:
timestamp,wan1,wan2,wan3
08/01/13 16:31:00,107,786,8960000000
08/01/13 16:31:10,123,244,434000000
08/01/13 16:31:20,345,284,4370000000

Additionally, I'd like to be able to call the script and give it an input file from the shell prompt rather than explicitly calling the file in the script. For example:

Code:
$ awkscript infile

Thanks again.

Last edited by Scrutinizer; 01-14-2013 at 08:52 AM.. Reason: quote tags -> code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

data processing

hi i am having a file of following kind: 20015#67143645#143123#4214 62014#67143148#67143159#456 15432#67143568#00143862#4632 54112#67143752#0067143657#143 54623#67143357#167215#34531 65446#67143785#143598#7456 75642#67143546#156146#845 24464#67143465#172532#6544... (5 Replies)
Discussion started by: rochitsharma
5 Replies

2. UNIX for Dummies Questions & Answers

Data File Processing Help

I need to read contents of directory and create a list of data files that match a certain pattern and process by renaming it and calling a existing .ksh script then archiving off to file another directory. Any suggestions or samples u could point me to on using .ksh perl or other to process... (5 Replies)
Discussion started by: mavsman
5 Replies

3. Shell Programming and Scripting

How should i know that the process is still processing data

I have some process . How should i know that the process is still processing data or got hanged even though it is showing that it is running in background I know of a command called truss. how should i use this command and determine 1) process is still processing data 2) process got hanged... (7 Replies)
Discussion started by: ali560045
7 Replies

4. Shell Programming and Scripting

Help with data processing, maybe awk

I have a file, first 5 columns are very normal, like "1107",106027,71400,"Y","BIOLOGY",, however, the 6th columns, the user can put comments, anything, just any characters, like new line, double quote, single quote, whatever from the keyboard, like"Please load my previous SOM597G course content in... (3 Replies)
Discussion started by: freelong
3 Replies

5. Shell Programming and Scripting

awk script processing data from 2 files

Hi! I have 2 files containing data that I need to process at the same time, I have problems in reading a different number of lines from the different files. Here is an explanation of what I need to do (possibly with an awk script). File "samples.txt" contains data in the format: time_instant... (6 Replies)
Discussion started by: Alice236
6 Replies

6. Programming

Data processing

Hello guys! I have some issue in how to processing some data. I have some files with 3 columns. The 1st column is a name of my sample. The 2nd column is a numerical sequence (very big sequence) starting from "1". And the 3rd column is a feature of each line, represented for a number (completely... (2 Replies)
Discussion started by: bfantinatti
2 Replies

7. UNIX for Dummies Questions & Answers

Genomic data processing

Dear fellow members, I've just joined the forum and am a newbie to shell scripting and programming. I'm stuck on the following problem. I'm working with large scale genomic data and need to do some analyses on it. Essentially it is text processing problem, so please don't mind the scientific... (0 Replies)
Discussion started by: mvaishnav
0 Replies

8. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

9. Shell Programming and Scripting

awk processing of variable number of fields data file

Hy! I need to post-process some data files which have variable (and periodic) number of fields. For example, I need to square (data -> data*data) the folowing data file: -5.34281E-28 -3.69822E-29 8.19128E-29 9.55444E-29 8.16494E-29 6.23125E-29 4.42106E-29 2.94592E-29 1.84841E-29 ... (5 Replies)
Discussion started by: radudownload
5 Replies

10. Shell Programming and Scripting

Data Processing

I have below Data *************************************************** ********************BEGINNING-1******************** directive url is : https://coursera-eu.mokar.com/directives/96df29ff-176a-35f7-8b1b-4ce483d15762 Src urls are :... (8 Replies)
Discussion started by: nikhil jain
8 Replies
kst(1)								       kst2								    kst(1)

NAME
kst -- plots scientific data SYNOPSIS
kst kst [ -F DATASOURCE ] [ -f STARTFRAME ] [ -n NUMFRAMES ] [ -s FRAMESPERSAMPLE [ -a ] ] kstfile kst datasource [ -f STARTFRAME ] [ -n NUMFRAMES ] [ -s FRAMESPERSAMPLE [ -a ] ] [ -P PLOTNAME ] [ -A] [ -d ] [ -l ] [ -b ] [ -x FIELD ] [ -e FIELD ] [ -r RATE ] [ -y FIELD ] [ -p FIELD ] [ -h FIELD ] [ -z FIELD ] [ --png filename ] [ --print filename [ --landscape | --portrait ] [ --Letter | --A4 ] ] DESCRIPTION
kst is a graphical data plotting tool, which includes the ability to plot live data (contiuously update the plot as new data becomes avail- able). kst may be run entirely from within its GUI, in which case no command line arguments are needed. kst has a powerful wizard for easily set- ting up new plots. In the second invocation, kst loads pre-saved plot specifications from the file kstfile and optionally may override some of the settings in that file. In the third invocation, kst displays plots of the data as directed in the default manner. Additional datasources may be specified on the command line with options for that datasource following it. Command line options generally apply to all subsequent options unless later overridden. OPTIONS
-A plot all curves in separate plots -a average frames (simple mean) when plotting with -s. --A4 use A4 sized paper for printing. Requires --print. -b plot as bargraphs. -d use points when plotting curves. -e FIELD use FIELD as the magnitude of error bars applied to the next plot. -F datasource override the datasource indicated in the kstfile and instead use datasource. -f STARTFRAME begin plotting at STARTFRAME frames into the data. Set to -1 to start at NUMFRAMES from the end of the data. -h FIELD plot FIELD as a histogram. -l use lines when plotting curves. --landscape use landscape orientation for printing. Requires --print. --Letter use US Letter sized paper for printing. Requires --print. -n NUMFRAMES plot at most NUMFRAMES frames of data. Set to -1 to indicate all of the data. -P PLOTNAME plot all plots in the plot named PLOTNAME. -p FIELD plot FIELD as a power spectrum plot. --png filename render to a portable network graphics (png) image with file name filename and quit. --portrait use portrait orientation for printing. Requires --print. --print filename print the plot to file filename and quit. -s FRAMESPERSAMPLE plot every FRAMESPERSAMPLE frames when plotting. -x FIELD use FIELD as the X-axis vector. -y FIELD plot FIELD as an XY plot. -z FIELD plot FIELD as an image. EXAMPLES
Plot all data in column 2 from data.dat. kst data.dat -y 2 Same as above, except only read 20 lines, starting at line 10. kst data.dat -f 10 -n 20 -y 2 also read col 1. One plot per curve. kst data.dat -f 10 -n 20 -y 1 -y 2 Read col 1 from data2.dat and col 1 from data.dat kst data.dat -f 10 -n 20 -y 2 data2.dat -y 1 Same as above, except read 40 lines starting at 30 in data2.dat kst data.dat -f 10 -n 20 -y 2 data2.dat -f 30 -n 40 -y 1 Specify the X vector and error bars: Plot x = col 1 and Y = col 2 and error flags = col 3 from data.dat kst data.dat -x 1 -e 3 -y 2 Get the X vector from data1.dat, and the Y vector from data2.dat. kst data1.dat -x 1 data2.dat -y 1 Placement: Plot column 2 and column 3 in plot P1 and column 4 in plot P2 kst data.dat -P P1 -y 2 -y 3 -P P2 -y 4 BUGS
Please report bugs to either the kst mailing list at kst@kde.org or to the kde bugzilla under the kst product at http://bugs.kde.org/. If you can run kst, you can easily report bugs by selecting Bug Report Wizard from the Help menu. AUTHOR
Matthew Truch <matt@truch.net> LICENSE
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the `COPYING.DOC' file as part of the kst distribution. Version 2.0.3 22 February 2011 kst(1)
All times are GMT -4. The time now is 12:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy