Sponsored Content
Top Forums Shell Programming and Scripting Filter duplicate block of text using SED Post 302222886 by dkumar91 on Thursday 7th of August 2008 09:27:38 PM
Old 08-07-2008
Hi,

Thanks for your help and time on this, It works great.

Deepak.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

using sed(?) to delete a block of text

hello people, i am trying to accomplish what i thought should be a simple task: find a token in a file and delete a number (let's say 25) of lines following the token. in sed, i can't figure out how to do a relative address (i.e. something like /token/25dd to delete 25 lines) and in gnu grep,... (3 Replies)
Discussion started by: toast
3 Replies

2. Shell Programming and Scripting

Filter/remove duplicate .dat file with certain criteria

I am a beginner in Unix. Though have been asked to write a script to filter(remove duplicates) data from a .dat file. File is very huge containig billions of records. contents of file looks like 30002157,40342424,OTC,mart_rec,100, ,0 30002157,40343369,OTC,mart_rec,95, ,0... (6 Replies)
Discussion started by: mukeshguliao
6 Replies

3. Shell Programming and Scripting

Filter or remove duplicate block of text without distinguishing marks or fields

Hello, Although I have found similar questions, I could not find advice that could help with our problem. The issue: We have several hundreds text files containing repeated blocks of text (I guess back at the time they were prepared like that to optmize printing). The block of texts... (13 Replies)
Discussion started by: samask
13 Replies

4. Shell Programming and Scripting

using sed/awk to replace a block of text in a file?

My apologies if this has been answered in a previous post. I've been doing a lot of searching, but I haven't been able to find what I was looking for. Specifically, I am wondering if I can utilize sed and/or awk to locate two strings in a file, and replace everything between those two strings... (12 Replies)
Discussion started by: kiddsupreme
12 Replies

5. Shell Programming and Scripting

Delete first block of text with sed/awk

Hello, guys! "filename" has blocks with three lines each in this fashion: 93909286 #verified has one bug 10909286 #unverified pending 10909286 #unverified pendingThe above example has duplicate blocks, and I have tried using sed to remove just one block... The... (2 Replies)
Discussion started by: teresaejunior
2 Replies

6. Shell Programming and Scripting

Block of text replacement using sed

Hi, I have a requirement in which i need to replace text as below - <stringProp name="Recipe">&lt;AddGroup Name=&quot;1001&quot; Path=&quot;ServiceAdministration/Controls/1001/ServiceSwitches&quot;&gt; &lt;Param Name=&quot;AttributeName&quot; Value=&quot;HeaderManipRspIngressRuleSet&quot; Type=&quot;String&quot; /&gt; &lt;Param Name=&quot;Value&quot;... (0 Replies)
Discussion started by: abhitanshu
0 Replies

7. UNIX for Dummies Questions & Answers

Filter records in a huge text file from a filter text file

Hi Folks, I have a text file with lots of rows with duplicates in the first column, i want to filter out records based on filter columns in a different filter text file. bash scripting is what i need. Data.txt Name OrderID Quantity Sam 123 300 Jay 342 498 Kev 78 2500 Sam 420 50 Vic 10... (3 Replies)
Discussion started by: tech_frk
3 Replies

8. Shell Programming and Scripting

Filter file to remove duplicate values in first column

Hello, I have a script that is generating a tab delimited output file. num Name PCA_A1 PCA_A2 PCA_A3 0 compound_00 -3.5054 -1.1207 -2.4372 1 compound_01 -2.2641 0.4287 -1.6120 3 compound_03 -1.3053 1.8495 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

9. Shell Programming and Scripting

Filter duplicate records from csv file with condition on one column

I have csv file with 30, 40 columns Pasting just three column for problem description I want to filter record if column 1 matches CN or DN then, check for values in column 2 if column contain 1235, 1235 then in column 3 values must be sequence of 2345, 2345 and if column 2 contains 6789, 6789... (5 Replies)
Discussion started by: as7951
5 Replies

10. Shell Programming and Scripting

CSV File:Filter duplicate records from column1 & another column having unique record

Hi Experts, I have csv file with 30, 40 columns Pasting just 2 column for problem description. Need to print error if below combination is not present in file check for column-1 (DocumentNumber) and filter columns where value in DocumentNumber field is same. For all such rows, the field... (7 Replies)
Discussion started by: as7951
7 Replies
FITCIRCLE(l)															      FITCIRCLE(l)

NAME
fitcircle - find mean position and pole of best-fit great [or small] circle to points on a sphere. SYNOPSIS
fitcircle [ xyfile ] -Lnorm [ -H[nrec] ] [ -S ] [ -V ] [ -: ] [ -bi[s][n] ] DESCRIPTION
fitcircle reads lon,lat [or lat,lon] values from the first two columns on standard input [or xyfile]. These are converted to cartesian three-vectors on the unit sphere. Then two locations are found: the mean of the input positions, and the pole to the great circle which best fits the input positions. The user may choose one or both of two possible solutions to this problem. The first is called -L1 and the second is called -L2. When the data are closely grouped along a great circle both solutions are similar. If the data have large dispersion, the pole to the great circle will be less well determined than the mean. Compare both solutions as a qualitative check. The -L1 solution is so called because it approximates the minimization of the sum of absolute values of cosines of angular distances. This solution finds the mean position as the Fisher average of the data, and the pole position as the Fisher average of the cross-products between the mean and the data. Averaging cross-products gives weight to points in proportion to their distance from the mean, analogous to the "leverage" of distant points in linear regression in the plane. The -L2 solution is so called because it approximates the minimization of the sum of squares of cosines of angular distances. It creates a 3 by 3 matrix of sums of squares of components of the data vectors. The eigenvectors of this matrix give the mean and pole locations. This method may be more subject to roundoff errors when there are thousands of data. The pole is given by the eigenvector corresponding to the smallest eigenvalue; it is the least-well represented factor in the data and is not easily estimated by either method. -L Specify the desired norm as 1 or 2, or use -L or -L3 to see both solutions. OPTIONS
xyfile ASCII [or binary, see -b] file containing lon,lat [lat,lon] values in the first 2 columns. If no file is specified, fitcircle will read from standard input. -H Input file(s) has Header record(s). Number of header records can be changed by editing your .gmtdefaults file. If used, GMT default is 1 header record. -S Attempt to fit a small circle instead of a great circle. The pole will be constrained to lie on the great circle connecting the pole of the best-fit great circle and the mean location of the data. -V Selects verbose mode, which will send progress reports to stderr [Default runs "silently"]. -: Toggles between (longitude,latitude) and (latitude,longitude) input/output. [Default is (longitude,latitude)]. Applies to geo- graphic coordinates only. -bi Selects binary input. Append s for single precision [Default is double]. Append n for the number of columns in the binary file(s). [Default is 2 input columns]. EXAMPLES
Suppose you have lon,lat,grav data along a twisty ship track in the file ship.xyg. You want to project this data onto a great circle and resample it in distance, in order to filter it or check its spectrum. Try: fitcircle ship.xyg -L2 project ship.xyg -Cox/oy -Tpx/py -S -pz | sample1d -S-100 -I1 > output.pg Here, ox/oy is the lon/lat of the mean from fitcircle, and px/py is the lon/lat of the pole. The file output.pg has distance, gravity data sampled every 1 km along the great circle which best fits ship.xyg SEE ALSO
gmt(1gmt), project(1gmt), sample1d(1gmt) 1 Jan 2004 FITCIRCLE(l)
All times are GMT -4. The time now is 03:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy