|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Data manipulation with Awk
Post 302347324 by Cham on Tuesday 25th of August 2009 11:32:10 AM
08-25-2009
I just noticed something bad with my data. Some points are doubled for nothing. Here's a list of Long/Lat coordinates (it's just a sample of a huge data file) :
So I now need to remove all the doubles (A', B', C' in the example above), while keeping the first (A, B, C). Take note that the doubles are always located immediately after the first.
So how can I remove them, using Awk ? I'm pretty sure this should be easy, but I'm really not a specialist of Awk.
Quote:
-0.1194983E+03 0.6000011E+02
-0.1194394E+03 0.6000006E+02
-0.1195464E+03 0.6000011E+02 <-------- A
-0.1195464E+03 0.6000011E+02 <-------- A'
-0.1195002E+03 0.6000021E+02
-0.1194394E+03 0.6000006E+02 # END
-0.1065872E+03 0.5324551E+02
-0.1100000E+03 0.6000000E+02
-0.1099966E+03 0.6000000E+02 <-------- B
-0.1099966E+03 0.6000000E+02 <-------- B'
-0.1090837E+03 0.6000000E+02 <-------- C
-0.1090837E+03 0.6000000E+02 <-------- C'
-0.1090004E+03 0.6000012E+02
....
-0.1194394E+03 0.6000006E+02 # END
-0.1194394E+03 0.6000006E+02
-0.1195464E+03 0.6000011E+02 <-------- A
-0.1195464E+03 0.6000011E+02 <-------- A'
-0.1195002E+03 0.6000021E+02
-0.1194394E+03 0.6000006E+02 # END
-0.1065872E+03 0.5324551E+02
-0.1100000E+03 0.6000000E+02
-0.1099966E+03 0.6000000E+02 <-------- B
-0.1099966E+03 0.6000000E+02 <-------- B'
-0.1090837E+03 0.6000000E+02 <-------- C
-0.1090837E+03 0.6000000E+02 <-------- C'
-0.1090004E+03 0.6000012E+02
....
-0.1194394E+03 0.6000006E+02 # END
So how can I remove them, using Awk ? I'm pretty sure this should be easy, but I'm really not a specialist of Awk.
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have a folder called {homedata}
Within this folder there are 12 subfolders 200601.......200612
Within each subfolder there are 8 sets of files
Each filename commences with A B C D E F G or H,
so {filename}* can be used.
I am trying to write a script which will from the top level
go... (1 Reply)
Discussion started by: grinder182533
1 Replies
2. UNIX for Dummies Questions & Answers
Hello
I am currently having problems in mapulating a certain file which contains vaious data.
Belos is a sample content
Event=<3190>
Client IP=<151.111.11.143> DNS=<abc.sbc.com> TransCount=<139>
Client IP=<150.222.133.163> DNS=<xyz.yuu.com> TransCount=<3734>
Event=<3120>
Client... (11 Replies)
Discussion started by: khestoi
11 Replies
3. Shell Programming and Scripting
HI all
i have a file consisting of following numbers
0000
0000
0000
0000
0000
1010
0000
0100
0000
0000
0000
1111
0000
1010
0000
0100 (3 Replies)
Discussion started by: vaibhavkorde
3 Replies
4. Shell Programming and Scripting
Hi,
I'm working on a data file with the following structure
val1,val2,flag
214.7332983,979.0259,1
12.87435571,205.7679,1
1.365976384,19.01616,1
44.08584096,205.7679,2
7.034721792,383.8778,2
189.5685503,979.0259,2
1.96352032,19.01616,2
where the field 'flag' identifies different... (10 Replies)
Discussion started by: Linoleum
10 Replies
5. Shell Programming and Scripting
Y,T,,H05,6,6,0,0 -> TH05_6
D,5,BT,B -> BT_KIOSK
P,KQC222 -> KQC222
G,B,2 -> BRANI_GATE_2
fileA
TPM658 Y,T,,H05,6,6,0,0
TPM110 D,5,BT,B
TPM136 P,KQC222
TPM180 P,BQC913
TPM575 Y,B,,T05,14,14,0,0
IPM760 G,B,2
TPM011
I need to use second column
$1,$2,$3,$4.....
if first char... (6 Replies)
Discussion started by: ment0smintz
6 Replies
6. Shell Programming and Scripting
Dear all
I have a dataset (in text format,delimited by tab) which have 100 variables (say, var0-var99) and more than 100,000 observations. I want to do the following:
1. for variable var0-var49, I want to add "00" in front of each data (for example, "1" would become "001")
2. for variable... (8 Replies)
Discussion started by: littlewenwen
8 Replies
7. UNIX for Dummies Questions & Answers
Hallo Team,
I need to manipulate existing data file. Have a look at current data and expected data:
Current Data:
27873517141
27873540000
27873515109
27873517140
27873540001
27873540000
27873501343
27873540000
27873517140
27873511292
27873645989
27873540000
27873540000... (7 Replies)
Discussion started by: kekanap
7 Replies
8. UNIX for Dummies Questions & Answers
Hallo Team,
I need your help. I would like to change field9 format to yyyy-mm-dd it should be for example 2013-11-16 instead of 20131116
0780112843,0873599381,E,ISOL,ZAR,0.0035,O,1,20131116,4373200,0.21
0733001720,0873516499,E,ISOL,ZAR,0.0035,O,1,20131116,4331600,0.21... (3 Replies)
Discussion started by: kekanap
3 Replies
9. UNIX for Dummies Questions & Answers
Dear Sir,
I have file input
RGR001|108.28|-2.86489|100-120|RANGGAR
RGR002|108.071|-2.69028|80-100|RANNGAR
RGR003|108.168|-2.97053|50-80|RANNGAR
RGR007|108.192722222|-2.766138889|0-50|RANGGARI want to create files by joining each rows with each rows below
Output as below
... (4 Replies)
Discussion started by: radius
4 Replies
10. Shell Programming and Scripting
Hello,
I have a huge set of data that needs to be reformatted.
Here is a simple example to explain the process.
I have number n=5 and a input with many numbers separated with comma:
... (11 Replies)
Discussion started by: liuzhencc
11 Replies
LEARN ABOUT DEBIAN
gr_plot_int
GR_PLOT_INT(1) User Commands GR_PLOT_INT(1) NAME
gr_plot_int.py - plot integer binary data using GNU Radio SYNOPSIS
gr_plot_int: [options] input_filenames DESCRIPTION
Takes a GNU Radio integer binary file and displays the samples versus time. You can set the block size to specify how many points to read in at a time and the start position in the file. By default, the system assumes a sample rate of 1, so in time, each sample is plotted ver- sus the sample number. To set a true time axis, set the sample rate (-R or --sample-rate) to the sample rate used when capturing the sam- ples. OPTIONS
-h, --help show this help message and exit -B BLOCK, --block=BLOCK Specify the block size [default=1000] -s START, --start=START Specify where to start in the file [default=0] -R SAMPLE_RATE, --sample-rate=SAMPLE_RATE Set the sampler rate of the data [default=1.0] SEE ALSO
gr_plot_char(1) gr_plot_const(1) gr_plot_fft_c(1) gr_plot_fft_f(1) gr_plot_float(1) gr_plot_int(1) gr_plot_iq(1) gr_plot_psd_c(1) gr_plot_psd_f(1) gr_plot_qt(1) gr_plot_short(1) gr_plot_int 3.5 December 2011 GR_PLOT_INT(1)