Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Filtering based on column values Post 302998250 by RudiC on Sunday 28th of May 2017 04:02:24 AM
Old 05-28-2017
You MUST be kidding! Incredible! That file does NOT contain any SF value other than 0 or 1 :


Code:
awk '{print $5, $6, $7}' FS="[,;=]" file
  
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 0 VRT
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 1 VRT
SF 0 1
SF 0 VRT
SF 1 VRT
SF 0 1
SF 1 VRT
SF 0 1
SF 0 1
SF 0 1
SF 1 VRT
SF 1 VRT
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 1 VRT
SF 0 1
SF 1 VRT
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 1 VRT
SF 0 1
SF 0 1
SF 0 1
SF 1 VRT
SF 0 1
SF 0 VRT
SF 0 1
SF 0 VRT
SF 0 VRT
SF 0 1
SF 0 1
SF 0 1
SF 0 1
SF 1 VRT

No surprise you have an empty output on all proposals! Unbelievable!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

2. Shell Programming and Scripting

How to pick values from column based on key values by usin AWK

Dear Guyz:) I have 2 different input files like this. I would like to pick the values or letters from the inputfile2 based on inputfile1 keys (A,F,N,X,Z). I have done similar task by using awk but in that case the inputfiles are similar like in inputfile2 (all keys in 1st column and values in... (16 Replies)
Discussion started by: repinementer
16 Replies

3. Shell Programming and Scripting

How to averaging column based on first column values

Hello I have file that consist of 2 columns of millions of entries timestamp and throughput I want to find the average (throughput ) for each equal timestamp before change it to proper format e.g : i want to average 2 coloumnd fot all 1308154800 values in column 1 and then print... (4 Replies)
Discussion started by: aadel
4 Replies

4. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

5. Shell Programming and Scripting

Filtering lines for column elements based on corresponding counts in another column

Hi, I have a file like this ACC 2 2 21 aaa AC 443 3 22 aaa GCT 76 1 33 xxx TCG 34 2 33 aaa ACGT 33 1 22 ggg TTC 99 3 44 wee CCA 33 2 33 ggg AAC 1 3 55 ddd TTG 10 1 22 ddd TTGC 98 3 22 ddd GCT 23 1 21 sds GTC 23 4 32 sds ACGT 32 2 33 vvv CGT 11 2 33 eee CCC 87 2 44... (1 Reply)
Discussion started by: polsum
1 Replies

6. Shell Programming and Scripting

Adding values of a column based on another column

Hello, I have a data such as this: ENSGALG00000000189 329 G A 4 2 0 ENSGALG00000000189 518 T C 5 1 0 ENSGALG00000000189 1104 G A 5 1 0 ENSGALG00000000187 3687 G T 5 1 0 ENSGALG00000000187 4533 A T 4 2 0 ENSGALG00000000233 5811 T C 4 2 0 ENSGALG00000000233 5998 C A 5 1 0 I want to... (3 Replies)
Discussion started by: Homa
3 Replies

7. Shell Programming and Scripting

Sum column values based in common identifier in 1st column.

Hi, I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column) The input is for example, after sorted: K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies

8. UNIX for Dummies Questions & Answers

Repositioning based on column values

Dear all ... I have a file which I want to change the structure based on the values in some columns and I would be grateful if you can help... one of my files looks like ... they all have ten rows 1,0,0 10,0,0 2,0,0 3,0,0 4,1,1 4,1,1 4,1,1 5,0,0 6,0,0 7,0,0 8,0.5,2 9,0.33,3 9,0.33,3... (1 Reply)
Discussion started by: A-V
1 Replies

9. Shell Programming and Scripting

Concatenate values in the first column based on the second column.

I have a file (myfile.txt) with contents like this: 1.txt apple is 3.txt apple is 5.txt apple is 2.txt apple is a 7.txt apple is a 8.txt apple is a fruit 4.txt orange not a fruit 6.txt zero isThe above file is already sorted using this command: sort -k2 myfile.txtMy objective is to get... (3 Replies)
Discussion started by: shoaibjameel123
3 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
gdal_grid(1)						      General Commands Manual						      gdal_grid(1)

NAME
gdal_grid - gdal_grid creates regular grid from the scattered data SYNOPSIS
gdal_grid [-ot {Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/ CInt16/CInt32/CFloat32/CFloat64}] [-of format] [-co "NAME=VALUE"] [-zfield field_name] [-a_srs srs_def] [-spat xmin ymin xmax ymax] [-clipsrc <xmin ymin xmax ymax>|WKT|datasource|spat_extent] [-clipsrcsql sql_statement] [-clipsrclayer layer] [-clipsrcwhere expression] [-l layername]* [-where expression] [-sql select_statement] [-txe xmin xmax] [-tye ymin ymax] [-outsize xsize ysize] [-a algorithm[:parameter1=value1]*] [-q] <src_datasource> <dst_filename> DESCRIPTION
This program creates regular grid (raster) from the scattered data read from the OGR datasource. Input data will be interpolated to fill grid nodes with values, you can choose from various interpolation methods. -ot type: For the output bands to be of the indicated data type. -of format: Select the output format. The default is GeoTIFF (GTiff). Use the short format name. -txe xmin xmax: Set georeferenced X extents of output file to be created. -tye ymin ymax: Set georeferenced Y extents of output file to be created. -outsize xsize ysize: Set the size of the output file in pixels and lines. -a_srs srs_def: Override the projection for the output file. The srs_def may be any of the usual GDAL/OGR forms, complete WKT, PROJ.4, EPSG:n or a file containing the WKT. -zfield field_name: Identifies an attribute field on the features to be used to get a Z value from. This value overrides Z value read from feature geometry record (naturally, if you have a Z value in geometry, otherwise you have no choice and should specify a field name containing Z value). -a [algorithm[:parameter1=value1][:parameter2=value2]...]: Set the interpolation algorithm or data metric name and (optionally) its parameters. See INTERPOLATION ALGORITHMS and DATA METRICS sections for further discussion of available options. -spat xmin ymin xmax ymax: Adds a spatial filter to select only features contained within the bounding box described by (xmin, ymin) - (xmax, ymax). -clipsrc [xmin ymin xmax ymax]|WKT|datasource|spat_extent: Adds a spatial filter to select only features contained within the specified bounding box (expressed in source SRS), WKT geometry (POLYGON or MULTIPOLYGON), from a datasource or to the spatial extent of the -spat option if you use the spat_extent keyword. When specifying a datasource, you will generally want to use it in combination of the -clipsrclayer, -clipsrcwhere or -clipsrcsql options. -clipsrcsql sql_statement: Select desired geometries using an SQL query instead. -clipsrclayer layername: Select the named layer from the source clip datasource. -clipsrcwhere expression: Restrict desired geometries based on attribute query. -l layername: Indicates the layer(s) from the datasource that will be used for input features. May be specified multiple times, but at least one layer name or a -sql option must be specified. -where expression: An optional SQL WHERE style query expression to be applied to select features to process from the input layer(s). -sql select_statement: An SQL statement to be evaluated against the datasource to produce a virtual layer of features to be processed. -co 'NAME=VALUE': Passes a creation option to the output format driver. Multiple -co options may be listed. See format specific documentation for legal creation options for each format. -q: Suppress progress monitor and other non-error output. src_datasource: Any OGR supported readable datasource. dst_filename: The GDAL supported output file. INTERPOLATION ALGORITHMS
There are number of interpolation algorithms to choose from. invdist Inverse distance to a power. This is default algorithm. It has following parameters: power: Weighting power (default 2.0). smoothing: Smoothing parameter (default 0.0). radius1: The first radius (X axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. radius2: The second radius (Y axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. angle: Angle of search ellipse rotation in degrees (counter clockwise, default 0.0). max_points: Maximum number of data points to use. Do not search for more points than this number. This is only used if search ellipse is set (both radiuses are non-zero). Zero means that all found points should be used. Default is 0. min_points: Minimum number of data points to use. If less amount of points found the grid node considered empty and will be filled with NODATA marker. This is only used if search ellipse is set (both radiuses are non-zero). Default is 0. nodata: NODATA marker to fill empty points (default 0.0). average Moving average algorithm. It has following parameters: radius1: The first radius (X axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. radius2: The second radius (Y axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. angle: Angle of search ellipse rotation in degrees (counter clockwise, default 0.0). min_points: Minimum number of data points to use. If less amount of points found the grid node considered empty and will be filled with NODATA marker. Default is 0. nodata: NODATA marker to fill empty points (default 0.0). Note, that it is essential to set search ellipse for moving average method. It is a window that will be averaged when computing grid nodes values. nearest Nearest neighbor algorithm. It has following parameters: radius1: The first radius (X axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. radius2: The second radius (Y axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. angle: Angle of search ellipse rotation in degrees (counter clockwise, default 0.0). nodata: NODATA marker to fill empty points (default 0.0). DATA METRICS
Besides the interpolation functionality gdal_grid can be used to compute some data metrics using the specified window and output grid geometry. These metrics are: minimum: Minimum value found in grid node search ellipse. maximum: Maximum value found in grid node search ellipse. range: A difference between the minimum and maximum values found in grid node search ellipse. count: A number of data points found in grid node search ellipse. average_distance: An average distance between the grid node (center of the search ellipse) and all of the data points found in grid node search ellipse. average_distance_pts: An average distance between the data points found in grid node search ellipse. The distance between each pair of points within ellipse is calculated and average of all distances is set as a grid node value. All the metrics have the same set of options: radius1: The first radius (X axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. radius2: The second radius (Y axis if rotation angle is 0) of search ellipse. Set this parameter to zero to use whole point array. Default is 0.0. angle: Angle of search ellipse rotation in degrees (counter clockwise, default 0.0). min_points: Minimum number of data points to use. If less amount of points found the grid node considered empty and will be filled with NODATA marker. This is only used if search ellipse is set (both radiuses are non-zero). Default is 0. nodata: NODATA marker to fill empty points (default 0.0). READING COMMA SEPARATED VALUES
Often you have a text file with a list of comma separated XYZ values to work with (so called CSV file). You can easily use that kind of data source in gdal_grid. All you need is create a virtual dataset header (VRT) for you CSV file and use it as input datasource for gdal_grid. You can find details on VRT format at Virtual Format description page. Here is a small example. Let we have a CSV file called dem.csv containing Easting,Northing,Elevation 86943.4,891957,139.13 87124.3,892075,135.01 86962.4,892321,182.04 87077.6,891995,135.01 For above data we will create dem.vrt header with the following content: <OGRVRTDataSource> <OGRVRTLayer name="dem"> <SrcDataSource>dem.csv</SrcDataSource> <GeometryType>wkbPoint</GeometryType> <GeometryField encoding="PointFromColumns" x="Easting" y="Northing" z="Elevation"/> </OGRVRTLayer> </OGRVRTDataSource> This description specifies so called 2.5D geometry with three coordinates X, Y and Z. Z value will be used for interpolation. Now you can use dem.vrt with all OGR programs (start with ogrinfo to test that everything works fine). The datasource will contain single layer called 'dem' filled with point features constructed from values in CSV file. Using this technique you can handle CSV files with more than three columns, switch columns, etc. If your CSV file does not contain column headers then it can be handled in the following way: <GeometryField encoding="PointFromColumns" x="field_1" y="field_2" z="field_3"/> Comma Separated Value description page contains details on CSV format supported by GDAL/OGR. EXAMPLE
The following would create raster TIFF file from VRT datasource described in READING COMMA SEPARATED VALUES section using the inverse distance to a power method. Values to interpolate will be read from Z value of geometry record. gdal_grid -a invdist:power=2.0:smoothing=1.0 -txe 85000 89000 -tye 894000 890000 -outsize 400 400 -of GTiff -ot Float64 -l dem dem.vrt dem.tiff The next command does the same thing as the previos one, but reads values to interpolate from the attribute field specified with -zfield option instead of geometry record. So in this case X and Y coordinates are being taken from geometry and Z is being taken from the 'Elevation' field. gdal_grid -zfield "Elevation" -a invdist:power=2.0:smoothing=1.0 -txe 85000 89000 -tye 894000 890000 -outsize 400 400 -of GTiff -ot Float64 -l dem dem.vrt dem.tiff AUTHORS
Andrey Kiselev dron@ak4719.spb.edu GDAL
Tue Sep 18 2012 gdal_grid(1)
All times are GMT -4. The time now is 02:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy