Sponsored Content
Top Forums Shell Programming and Scripting extract data from a data matrix with filter criteria Post 302307588 by jeffm on Wednesday 15th of April 2009 09:23:21 PM
Old 04-15-2009
Fairly easy in Tcl. I'm not sure how you'd go about doing it in a "standard" shell script like Ksh, Bash, etc.

Tcl is pretty good for this because all variables are both strings and "lists" (sorta like arrays).

Code:
#! /usr/bin/tclsh

# open the data file for reading.
set fid [open /path/to/matrix.data r]

# loop through all lines in the file.
while { ![eof $fid] } {

        gets $fid row

        # Row are: ID, X1, X2, Y1, Y2
        # We don't care if the ID row is higer than
        # .5, so don't check it.
        set data [lrange $row 1 end]

        #data = X!, X2, Y1, Y2.
        # so foreach d $data will check the individual elements of the list.
        foreach d $data {

                if { $d >= 0.5 } {
                        set newdata $row
                }
        }

        # if we have a match from above, then
        # spit out the entire row.
        if { $newdata != "" } {
                puts stdout $newdata
                set newdata ""
        }
}
# close the file.
close $fid

So using your data above, the script produces:
Code:
[jeffm@stalin:~] tclsh script.tcl
 X1 X2 Y1 Y2
10855062 0.845956333 0.860396667 0.014440333 1.483899333
10808085 0.184618667 0.626846333 0.442227667 0.266913667

I'm sure the Perl people have a better way of doing it. Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

changing data into matrix form

Hi, I have a file whose structure is like this 7 7 1 2 3 4 5 1 3 4 8 6 1 4 5 6 0 2 6 8 3 8 2 5 7 8 0 5 7 9 4 1 3 8 0 2 2 3 5 6 8 basically first two row tell the number of rows and column but the data following them are not arranged in that format. now i want to create another... (1 Reply)
Discussion started by: g0600014
1 Replies

2. Shell Programming and Scripting

Merge 70 files into one data matrix

Hi, I have a list of 70 files in a directory and I need to merge the content of each file into one big matrix file (71 columns x 3060 rows). Each file has the following format only two columns per file: unique identifier1 randomtext1 randomtext1 a 5 b 3 c 6 d 3 e 2... (11 Replies)
Discussion started by: labrazil
11 Replies

3. Shell Programming and Scripting

Extract data based on match against one column data from a long list data

My input file: data_5 Ali 422 2.00E-45 102/253 140/253 24 data_3 Abu 202 60.00E-45 12/23 140/23 28 data_1 Ahmad 256 7.00E-45 120/235 140/235 22 data_4 Aman 365 8.00E-45 15/65 140/65 20 data_10 Jones 869 9.00E-45 65/253 140/253 18... (12 Replies)
Discussion started by: patrick87
12 Replies

4. Shell Programming and Scripting

Extract specific data content from a long list of data

My input: Data name: ABC001 Data length: 1000 Detail info Data Direction Start_time End_time Length 1 forward 10 100 90 1 forward 15 200 185 2 reverse 50 500 450 Data name: XFG110 Data length: 100 Detail info Data Direction Start_time End_time Length 1 forward 50 100 50 ... (11 Replies)
Discussion started by: patrick87
11 Replies

5. Shell Programming and Scripting

Invert Matrix of Data - Perl

I have columnar data in arrays perl, Example - @a = (1,2,3); @array1 = (A,B,C); @array2 = (D,E,F); @array3 = (I,R,T); I want the data to be formatted and printed as 1 A D I 2 B E F 3 C F T and so on... (8 Replies)
Discussion started by: dinjo_jo
8 Replies

6. Shell Programming and Scripting

Extract data based on specific search criteria

I have a huge file (about 2 millions records) contains data separated by “,” (comma). As part of the requirement, I can't change the format. The objective is to remove some of the records with the following condition. If the 23rd field on each line start with 302 , I need to remove that from the... (4 Replies)
Discussion started by: jaygamini
4 Replies

7. Ubuntu

How to convert full data matrix to linearised left data matrix?

Hi all, Is there a way to convert full data matrix to linearised left data matrix? e.g full data matrix Bh1 Bh2 Bh3 Bh4 Bh5 Bh6 Bh7 Bh1 0 0.241058 0.236129 0.244397 0.237479 0.240767 0.245245 Bh2 0.241058 0 0.240594 0.241931 0.241975 ... (8 Replies)
Discussion started by: evoll
8 Replies

8. Shell Programming and Scripting

Reformatting data in matrix form

Hi, Some assistance with respect to the following problem will be very helpful. I want to reformat my dataset in the following manner for subsequent analysis. I have first column values (which repeat for each value of 2nd column) which are names, the second column specifies position ad the... (1 Reply)
Discussion started by: newbie83
1 Replies

9. Shell Programming and Scripting

[Solved] Converting the data into matrix with 0's and 1's

I have a file that contains 2 columns tag,pos cat input_file tag pos atg 10 ata 16 agt 15 agg 19 atg 17 agg 14 I have used following command to sort the file based on second column sort -k 2 input_file tag pos atg 10 agg 14 agt 15 ata 16 agg 19 atg 17 (2 Replies)
Discussion started by: raj_k
2 Replies

10. Shell Programming and Scripting

Convert data into tabular matrix

Hi There, I want to convert the following data into tabular matrix, based on column 4th and 5th, and output the column 10th value chr1 2804449 2804450 NACpG_1 window1 + chr1 2804443 2804475 1 chr1 2804450 2804451 NACpG_1 window2 + chr1 2804443 ... (3 Replies)
Discussion started by: ChiragNepal
3 Replies
tclsh(1)							 Tcl Applications							  tclsh(1)

__________________________________________________________________________________________________________________________________________________

NAME
tclsh - Simple shell containing Tcl interpreter SYNOPSIS
tclsh ?-encoding name? ?fileName arg arg ...? _________________________________________________________________ DESCRIPTION
Tclsh is a shell-like application that reads Tcl commands from its standard input or from a file and evaluates them. If invoked with no arguments then it runs interactively, reading Tcl commands from standard input and printing command results and error messages to standard output. It runs until the exit command is invoked or until it reaches end-of-file on its standard input. If there exists a file .tclshrc (or tclshrc.tcl on the Windows platforms) in the home directory of the user, interactive tclsh evaluates the file as a Tcl script just before reading the first command from standard input. SCRIPT FILES
If tclsh is invoked with arguments then the first few arguments specify the name of a script file, and, optionally, the encoding of the | text data stored in that script file. Any additional arguments are made available to the script as variables (see below). Instead of reading commands from standard input tclsh will read Tcl commands from the named file; tclsh will exit when it reaches the end of the file. The end of the file may be marked either by the physical end of the medium, or by the character, "32" ("u001a", control-Z). If this character is present in the file, the tclsh application will read text up to but not including the character. An application that requires this character in the file may safely encode it as "32", "x1a", or "u001a"; or may generate it by use of commands such as for- mat or binary. There is no automatic evaluation of .tclshrc when the name of a script file is presented on the tclsh command line, but the script file can always source it if desired. If you create a Tcl script in a file whose first line is #!/usr/bin/tclsh then you can invoke the script file directly from your shell if you mark the file as executable. This assumes that tclsh has been installed in the default location in /usr/bin; if it is installed somewhere else then you will have to modify the above line to match. Many UNIX systems do not allow the #! line to exceed about 30 characters in length, so be sure that the tclsh executable can be accessed with a short file name. An even better approach is to start your script files with the following three lines: #!/bin/sh # the next line restarts using tclsh exec tclsh "$0" "$@" This approach has three advantages over the approach in the previous paragraph. First, the location of the tclsh binary does not have to be hard-wired into the script: it can be anywhere in your shell search path. Second, it gets around the 30-character file name limit in the previous approach. Third, this approach will work even if tclsh is itself a shell script (this is done on some systems in order to handle multiple architectures or operating systems: the tclsh script selects one of several binaries to run). The three lines cause both sh and tclsh to process the script, but the exec is only executed by sh. sh processes the script first; it treats the second line as a comment and executes the third line. The exec statement cause the shell to stop processing and instead to start up tclsh to reprocess the entire script. When tclsh starts up, it treats all three lines as comments, since the backslash at the end of the second line causes the third line to be treated as part of the comment on the second line. You should note that it is also common practice to install tclsh with its version number as part of the name. This has the advantage of allowing multiple versions of Tcl to exist on the same system at once, but also the disadvantage of making it harder to write scripts that start up uniformly across different versions of Tcl. VARIABLES
Tclsh sets the following Tcl variables: argc Contains a count of the number of arg arguments (0 if none), not including the name of the script file. argv Contains a Tcl list whose elements are the arg arguments, in order, or an empty string if there are no arg arguments. argv0 Contains fileName if it was specified. Otherwise, contains the name by which tclsh was invoked. tcl_interactive Contains 1 if tclsh is running interactively (no fileName was specified and standard input is a terminal-like device), 0 otherwise. PROMPTS
When tclsh is invoked interactively it normally prompts for each command with "% ". You can change the prompt by setting the variables tcl_prompt1 and tcl_prompt2. If variable tcl_prompt1 exists then it must consist of a Tcl script to output a prompt; instead of out- putting a prompt tclsh will evaluate the script in tcl_prompt1. The variable tcl_prompt2 is used in a similar way when a newline is typed but the current command is not yet complete; if tcl_prompt2 is not set then no prompt is output for incomplete commands. STANDARD CHANNELS
See Tcl_StandardChannels for more explanations. SEE ALSO
encoding(n), fconfigure(n), tclvars(n) KEYWORDS
argument, interpreter, prompt, script file, shell Tcl tclsh(1)
All times are GMT -4. The time now is 09:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy