Sponsored Content
Top Forums Shell Programming and Scripting Selection of records y time and offset. Post 302990596 by jiam912 on Sunday 29th of January 2017 03:42:16 AM
Old 01-29-2017
Selection of records y time and offset.

Gents,

I have an csv file which contends value GPS time for each record and x, y coordinates.. The purpose is to identify which vps does not match with the law parameters in the relation time and offset.
In the graphic attached you can see example of 3 vps which are showed in red color and falls in the area not allowed. All records marked as blue are ok.

Then the thing is to sort the records by GPS time .. ( I did this part in my script ). calculate the time from GPS time as HHMMSS. ( done ).. now we need to duplicate say 10 time each record and write in front the next ones ( example analysis_25.csv ).
Here is the time to calculate the offset (column 18) and time (column 19) in the file analysis_25.csv..
Using the Law parameters we need to compute the Allowed_SlipTime ( column 19 ) and with this value we can define witch records are bad ( bad_records_25.csv) ...
In the graphic analysis_25.png we can see how the law works by time and offset
Values for law are:
Code:
Slip time(s))  18        18         6           0           0            0
Offset(m)       0      2000      6000       12000       12001        24000

Max time (s) 25
Max offset (m) 15000

Here my script till now.

Code:
awk 'NR>1{print $0}' $25.csv | awk -F, '{
                       line=substr($10,1,5)
                point=substr($11,1,5)
                ffid=$2
                sw=$5
                tape=$6;\
                x=$49;\
                y=$50;\
                tb=$46
                printf("%4d %5d %5d %4d %4d %10.1f %10.1f %s\n",tape,line,point,ffid,sw,x,y,tb)}' | sort -k8n |
                awk '{v=substr($8,16,1); gsub(/2/,"0",v);print substr($0,1,64) v}' > toto            
awk '\
        function gpstojdtime(gpstime) {\
            gpss = (gpstime/1000000)+10800;\
            jtmp = (gpss-((2017-1980)*365+10-6)*86400-18)/86400;\
            jday = int(jtmp);\
            htmp = 24*(jtmp - jday);\
            hour = int(htmp);\
            mtmp = 60*((htmp - hour));\
            minu = int(mtmp);\
            stmp = 60*(mtmp - minu);\
            secd = (stmp);\
            return hour*10000+minu*100+secd;\
        }\
         {\
            {\
                tape=$1;\
                splin=$2;\
                spnum=$3;\
                sprec=$4;\
                sw=$5;\
                x=$6;\
                y=$7;\
                spgps=$8;\
                gpt=$8;\
                printf"%4s %5d %5d %4s %4s %10.1f %10.1f %10.3f %s\n",\
                tape,splin,spnum,sprec,sw,x,y,gpstojdtime(spgps),gpt\
            };\
        }' toto | head -20

The support data is attached :

Code:
25.csv = Input data

analysis_25.png = Graphic results
analysis_25.csv = Data results - Desired output file 
bad_records_25.csv = Bad records results - Desired output file 

Appreciate your help
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perform selection and deletion at the same time

I am performing some operation like: SQL Statement below: UPDATE table2 SET column1_table2 = ( SELECT column1_table1 FROM table1 WHERE column2_table1 = column2_table2 LIMIT 1) LIMIT 2; So what I'm trying to do is to find a corresponding value and then I'm adding it into table2. Now how... (2 Replies)
Discussion started by: Legend986
2 Replies

2. Shell Programming and Scripting

deleting multiple records from a huge file at one time

I have a very big file of 5gb size and there are about 50 million records in there. I have to delete the records based on recrord number that I know fromoutside with out opening the file. The record numbers are very random like 5000678, 7890005 etc. Can somebody let me know how i can... (5 Replies)
Discussion started by: dsravan
5 Replies

3. Programming

Negative Offset

Function: int fcntl(int fd, int cmd, struct flock * lock) Data Type: struct flock This structure is used with the fcntl function to describe a file lock. It has these members: off_t l_start This specifies the offset of the start of the region to which the lock applies, and... (1 Reply)
Discussion started by: DNAx86
1 Replies

4. Shell Programming and Scripting

awk scripting - matching records and summing up time

Hello. I just found out about awk, and it appears that this could handle the problem I'm having right now. I first stumbled on the thread How to extract first and last line of different record from a file, and that problem is almost similar to mine. In my case, an ASCII file will contain the... (0 Replies)
Discussion started by: Gonik
0 Replies

5. Shell Programming and Scripting

Scroll records from database, one at a time

Hi, I need to come up with a site that will display all the records in the database, but one at a time. Not sure how to go about it. Please pour in your suggestions. Thanks ---------- Post updated at 04:38 AM ---------- Previous update was at 12:52 AM ---------- Can... (3 Replies)
Discussion started by: sh_kk
3 Replies

6. Shell Programming and Scripting

Find records using epoch time.

How do i find the record which has been edit the last 10 minutes? from a.txt which has last field is epoch time updated. 10/17/2012 1:47 PM||||||In Use|chicken||1350005487 10/17/2012 2:53 PM||||||Available|chicken||13500000 10/17/2012 3:20 PM||||||In Use|cat||1351000000 10/17/2012 3:22... (2 Replies)
Discussion started by: sabercats
2 Replies

7. UNIX for Advanced & Expert Users

Grep --byte-offset not returning the offset (Grep version 2.5.1)

Hi, I am trying to get the position of a repeated string in a line using grep -b -o "pattern" In my server I am using GNU grep version 2.14 and the code is working fine. However when I am deploying the same code in a different server which is using GNU grep version 2.5.1 the code is not... (3 Replies)
Discussion started by: Subhamoy
3 Replies

8. UNIX for Dummies Questions & Answers

File name offset

Dear all, I want to offset the file numbers. can you please make some awk code or linux code for the same. Example: input file names ANI_WFMASS_PIST00001.gif ANI_WFMASS_PIST00002.gif . . . ANI_WFMASS_PIST0000n.gif offset --> 30 ANI_WFMASS_PIST00031.gif ANI_WFMASS_PIST00032.gif... (14 Replies)
Discussion started by: kri321shna
14 Replies

9. Shell Programming and Scripting

Grep 'time' in save records

Hi Team, Is there a way to grep time taken to save records. Its like there is one webpage where when I click save button taking so much time to save result. Therefore, I want to grep that time taken to save that record from file.log Thanks in advance. (1 Reply)
Discussion started by: TCS
1 Replies

10. Shell Programming and Scripting

Help with split a list of records into each line with 200 coordinate at a time

Input File: E 3359799 3360148 350 X D 3471287 3471607 321 X E 3359799 3360740 942 X E 4359790 4360039 250 X . . . Desired Output File: E 3359799 3359998 200 X E 3359999 3360148 150 X D 3471287 3471486 200 X D 3471487 3471607 121 X E 3359799 3359998 200 X E 3359999 3360198 200 X... (1 Reply)
Discussion started by: perl_beginner
1 Replies
All times are GMT -4. The time now is 05:04 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy