Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

AWK Data Cleaning

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers AWK Data Cleaning
# 15  
Old 10-06-2010
This is the code I am using:

Code:
BEGIN {
FS="\t";RS="\n";
}
{if ( NR == 1) { printf "%s\t%s\t%s\n", $3, "MS", $7}}
{if ($8!=1 &&  $9!=1) { printf "%s\t%s\t%s\n", $3, NR-1, $7;
}
}

I get a constant millisecond count. Agama seemed to have gotten it, but that did not even work.
# 16  
Old 10-06-2010
Did you try my suggestion in post 14 and if so what is wrong with it?
This User Gave Thanks to Scrutinizer For This Post:
carmar87
# 17  
Old 10-06-2010
Is there any conditional to reset the millisecond counter when there is a new trial?
# 18  
Old 10-06-2010
You could reset the counter i each time $3 changes, for example:
Code:
awk -F"\t" 'NR==1{printf "%s\t\t%s\t%s\n", "Trial", "MS", "LPUPILSIZE";next}
            NF{if(p!=$3){i=0; p=$3}; i++;  if($8!=1 && $9!=1)printf "%s\t%s\t%s\n",$3,i,$7}' infile

# 19  
Old 10-06-2010
What does p! represent?
# 20  
Old 10-06-2010
There are SQL tools that will use a flat file as a db table, like JDBC HXTT under jisql, so you can just query it.
# 21  
Old 10-07-2010
p is just a variable the remembers the previous value. So in this case it is used to detect whether the value of $3 has changed. if(p!=$3) means if the previous value is unequal to the current value of $3...
 
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk --> math-operation in data-record and joining with second file data

Hi! I have a pretty complex job - at least for me! i have two csv-files with meassurement-data: fileA ...... (2 Replies)
Discussion started by: IMPe
2 Replies

2. Shell Programming and Scripting

Cleaning through perl or awk a Stemmer dictionary

Hello, I work under Windows Vista and I am compiling an open-source stemmer dictionary for English and eventually for other Indian languages. The Engine which I have written has spewed out all lemmatised/expanded forms of the words: Nouns, Adjectives, Adverbs etc. Each set of expanded forms is... (4 Replies)
Discussion started by: gimley
4 Replies

3. Shell Programming and Scripting

Cleaning output using awk

I have some small problem with my code. data.html <TD class="statuscol2">c</TD> <TD class="statuscol3">18</TD> <TD class="statuscol4"><SPAN TITLE="#04">test4</SPAN></TD> <TD... (4 Replies)
Discussion started by: Jotne
4 Replies

4. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

5. Shell Programming and Scripting

Cleaning AWK code

Hi I need some help to clean my code used to get city location. wget -q -O - http://www.ip2location.com/ | grep chkRegionCity | awk 'END { print }' | awk -F"" '{print $4}' It gives me the city but have a leading space. I am sure this could all be done by one single AWK Also if possible... (8 Replies)
Discussion started by: Jotne
8 Replies

6. Shell Programming and Scripting

cleaning the file

Hi, I have a file with multiple rows. each row has 8 columns. Column 8 has entries separated by commas. I want to exclude all the rows in which column 8 has more than 3 commas. 1234#0/1 - ABC_1234 3 ATGCATGCATGC HHHIIIGIHVF 1 49:T>C,60:T>C,78:C>A,76:G>T,65:T>G Thanks, Diya (3 Replies)
Discussion started by: Diya123
3 Replies

7. Shell Programming and Scripting

File cleaning

HI , I am getting the source data as below. Source Data CDR_Data,,,,, F1,F2,F3,F4,F5,F6 5,5,6,7,8,7 6,6,g,,, 7,7,76,,, 8,8,gt,,, 9,9,df ,d,d,d ,,,,, (4 Replies)
Discussion started by: wangkc
4 Replies

8. Shell Programming and Scripting

Data Cleaning in a file

Hi , I have the below source data, I need to clean the data in 2nd,4th,5th columns. Source Data RECORD,CASH_TRANS,BEJING,AUG2011/CASH_TRANS,Y/N150/CASH_TRANS,N/201108 RECORD,CASH_TRANS,INDIA,AUG2011/CASH_TRANS,Y/NC110/CASH_TRANS,N/201108... (7 Replies)
Discussion started by: mora
7 Replies

9. AIX

doing some spring cleaning....

USERS="me you jim joe sue" for user in ${USERS}; do rmuser -p $user usrdir=`cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` rm -fr `cat /etc/passwd|grep $user|awk -F":" '{ print $6 }'` echo Deleting: $user '\t' REMOVING: $usrdir done This is for AIX ONLY!!! but easily ported to... (0 Replies)
Discussion started by: Optimus_P
0 Replies
Login or Register to Ask a Question