Sponsored Content
Top Forums Shell Programming and Scripting Processing result file based on a minimal value Post 302564355 by fozrun on Thursday 13th of October 2011 11:57:12 AM
Old 10-13-2011
Some further processing?

When I used a real world result file I came up with a couple of other problems. The file now has the right data but the classification produces extraneous data in the form of duplicated "unclassified" entries, and the sort order is out of whack.

Code:
ID	Name 	MA1_01.txt	MA_01.txt	MB_01.txt	MC_01.txt
0.1.44.6.26.2.17	uncultured	0.008904934	0.010950786	0.02002887	0.009102323	0.001050788
0.1.44.6.26.2.17.1	unclassified	0.008904934	0.010950786	0.02002887	0.009102323	0.001050788
0.1.44.6.26.2.17.1.1	unclassified	0.008904934	0.010950786	0.02002887	0.009102323	0.001050788
0.1.44.6.26.2.17.1.1.1	unclassified	0.008904934	0.010950786	0.02002887	0.009102323	0.001050788
0.1.44.6.26.2.8	Luteimonas	0.048856799	0.046508632	0.045110069	0.050219711	0.006654991
0.1.44.6.26.2.8.1	unclassified	0.048856799	0.046508632	0.045110069	0.050219711	0.006654991
0.1.44.6.26.2.8.1.1	unclassified	0.048856799	0.046508632	0.045110069	0.050219711	0.006654991
0.1.44.6.26.2.8.1.1.1	unclassified	0.048856799	0.046508632	0.045110069	0.050219711	0.006654991
0.1.44.6.8	Chromatiales	0.005535499	0.007729967	0.001082642	0.005963591	0.001050788
0.1.44.6.8.1	Chromatiaceae	0.005294826	0.007214635	0	0.005649718	0.001050788
0.1.44.6.8.1.8	Rheinheimera	0.004572804	0.006441639	0	0.005021971	0
0.1.44.6.8.1.8.1	unclassified	0.004572804	0.006441639	0	0.005021971	0
0.1.44.6.8.1.8.1.1	unclassified	0.004572804	0.006441639	0	0.005021971	0
0.1.44.6.8.1.8.1.1.1	unclassified	0.004572804	0.006441639	0	0.005021971	0
0.1.44.6.9	Enterobacteriales	0.007220217	0.012625612	0.018585348	0	0.122942207
0.1.44.6.9.1	Enterobacteriaceae	0.007220217	0.012625612	0.018585348	0	0.122942207
0.1.44.6.9.1.10	Enteric_Bacteria_cluster	0.006738869	0.011852615	0.017683147	0	0.116287215
0.1.44.6.9.1.10.1	Brenneria	0.004091456	0.005024478	0.008841573	0	0.070753065
0.1.44.6.9.1.10.1.1	unclassified	0.004091456	0.005024478	0.008841573	0	0.070753065
0.1.44.6.9.1.10.1.1.1	unclassified	0.004091456	0.005024478	0.008841573	0	0.070753065

I'd like to get rid of any lines that have unclassified in them and have the first column sorted based on the hierarchy of the numbers. So that for example 0.1.44.6.8 and following would come before 0.1.44.6.26 etc.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Time based Processing of the Scripts

Any one can tell me how can i execute the processes for every 10 min.Actually iam having 3 Processes for every 10 min i want to run these 3 Process,one process at every 10 min. If any of the process is busy i just want to execute the free one. first 10 min execute P1 next 10 min execute P2... (3 Replies)
Discussion started by: krk_555
3 Replies

2. Shell Programming and Scripting

printing in certain column based on some result

hii every one can anybody help me writing shell script with this, i have a file ... amit arun vivek and i want to read something from the user and print next to amit or arun in certain column.. like amit 23-wall street 2000 arun 34343 vivek 4758 is... (6 Replies)
Discussion started by: kumar_amit
6 Replies

3. Shell Programming and Scripting

Filter the column and print the result based on condition

Hi all This is my output of the some SQL Query TABLESPACE_NAME FILE_NAME TOTALSPACE FREESPACE USEDSPACE Free ------------------------- ------------------------------------------------------- ---------- --------- ---------... (2 Replies)
Discussion started by: jhon
2 Replies

4. Shell Programming and Scripting

How to processing the log file within certain dates based on the file name

Hi I am working on the script parsing specific message "TEST" from multiple file. The log file name looks like: N3.2009-11-26-03-05-02.console.log.tar.gz N4.2009-11-29-00-25-03.console.log.tar.gz N6.2009-12-01-10-05-02.console.log.tar.gz I am using the following command: zgrep -a --text... (1 Reply)
Discussion started by: shyork2001
1 Replies

5. Shell Programming and Scripting

Help with File processing - Adding predefined text to particular record based on condition

I am generating a output: Name Count_1 Count_2 abc 12 12 def 15 14 ghi 16 16 jkl 18 18 mno 7 5 I am sending the output in html email, I want to add the code: <font color="red"> NAME COLUMN record </font> for the Name... (8 Replies)
Discussion started by: karumudi7
8 Replies

6. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

7. Shell Programming and Scripting

Output block of lines in a file based on grep result

Hi I would appreciate your help with this. I have a output file from a command. It is broken based on initial of the users. Exmaple of iitials MN & SS. Under each section there is information pertaining to the user however each section can have different number of lines. MY challenge is to ... (5 Replies)
Discussion started by: mnassiri
5 Replies

8. Shell Programming and Scripting

How to sort grep result based on timestamp?

Hi, Trying to sort grep result based on timestamp of the filename. I have the following result and want to sort them on timestampgrep -i 'ERROR' *log*2013* s_m_xxx_xxx_xxx_xxx_xxxx.log.20130906092431:TRANSF_1_1_1> DBG_21216 Finished transformations for Source Qualifier . Total errors ... (5 Replies)
Discussion started by: bobbygsk
5 Replies

9. Shell Programming and Scripting

ksh : Building an array based on condition result

I want to build an Errorlog. I would like to build an array as I move through the if statements and print the array once all error conditions have been defined. The results need to be comma delimited. tsver will be static "1.9.6(2)" other vars $prit $lt $rt can have the same or a different... (1 Reply)
Discussion started by: popeye
1 Replies

10. Shell Programming and Scripting

awk to parse file and display result based on text

I am trying using awk to open an input file and check a column 2/field $2 and if there is a warning then that is displayed (variantchecker): G not found at position 459, found A instead. The attached Sample1.txt is that file. If in that column/field there is a black space, then the text after... (6 Replies)
Discussion started by: cmccabe
6 Replies
FITCIRCLE(l)															      FITCIRCLE(l)

NAME
fitcircle - find mean position and pole of best-fit great [or small] circle to points on a sphere. SYNOPSIS
fitcircle [ xyfile ] -Lnorm [ -H[nrec] ] [ -S ] [ -V ] [ -: ] [ -bi[s][n] ] DESCRIPTION
fitcircle reads lon,lat [or lat,lon] values from the first two columns on standard input [or xyfile]. These are converted to cartesian three-vectors on the unit sphere. Then two locations are found: the mean of the input positions, and the pole to the great circle which best fits the input positions. The user may choose one or both of two possible solutions to this problem. The first is called -L1 and the second is called -L2. When the data are closely grouped along a great circle both solutions are similar. If the data have large dispersion, the pole to the great circle will be less well determined than the mean. Compare both solutions as a qualitative check. The -L1 solution is so called because it approximates the minimization of the sum of absolute values of cosines of angular distances. This solution finds the mean position as the Fisher average of the data, and the pole position as the Fisher average of the cross-products between the mean and the data. Averaging cross-products gives weight to points in proportion to their distance from the mean, analogous to the "leverage" of distant points in linear regression in the plane. The -L2 solution is so called because it approximates the minimization of the sum of squares of cosines of angular distances. It creates a 3 by 3 matrix of sums of squares of components of the data vectors. The eigenvectors of this matrix give the mean and pole locations. This method may be more subject to roundoff errors when there are thousands of data. The pole is given by the eigenvector corresponding to the smallest eigenvalue; it is the least-well represented factor in the data and is not easily estimated by either method. -L Specify the desired norm as 1 or 2, or use -L or -L3 to see both solutions. OPTIONS
xyfile ASCII [or binary, see -b] file containing lon,lat [lat,lon] values in the first 2 columns. If no file is specified, fitcircle will read from standard input. -H Input file(s) has Header record(s). Number of header records can be changed by editing your .gmtdefaults file. If used, GMT default is 1 header record. -S Attempt to fit a small circle instead of a great circle. The pole will be constrained to lie on the great circle connecting the pole of the best-fit great circle and the mean location of the data. -V Selects verbose mode, which will send progress reports to stderr [Default runs "silently"]. -: Toggles between (longitude,latitude) and (latitude,longitude) input/output. [Default is (longitude,latitude)]. Applies to geo- graphic coordinates only. -bi Selects binary input. Append s for single precision [Default is double]. Append n for the number of columns in the binary file(s). [Default is 2 input columns]. EXAMPLES
Suppose you have lon,lat,grav data along a twisty ship track in the file ship.xyg. You want to project this data onto a great circle and resample it in distance, in order to filter it or check its spectrum. Try: fitcircle ship.xyg -L2 project ship.xyg -Cox/oy -Tpx/py -S -pz | sample1d -S-100 -I1 > output.pg Here, ox/oy is the lon/lat of the mean from fitcircle, and px/py is the lon/lat of the pole. The file output.pg has distance, gravity data sampled every 1 km along the great circle which best fits ship.xyg SEE ALSO
gmt(1gmt), project(1gmt), sample1d(1gmt) 1 Jan 2004 FITCIRCLE(l)
All times are GMT -4. The time now is 04:00 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy