Sponsored Content
Top Forums UNIX for Beginners Questions & Answers UNIX command to select the best edge values from a network file Post 303045006 by Sanchari on Tuesday 10th of March 2020 06:16:47 PM
Old 03-10-2020
UNIX command to select the best edge values from a network file

I have a tab-delimited data representing network data (undirected). Among the duplicated edges, I wanted to select those edges for which I have the higher absolute value of the log values.
I have written a code in python, but its taking a lot of time. I would be grateful if someone helps me with an awk command. Kindly note, the network is undirected, i.e. A--B and B--A are duplicate edges. My original file has a large number of columns, I have given a simplified test data

Test data



Code:
     Gene1    Gene2    Log
    AT1G01020    AT1G01010    1.682708
    AT1G01020    AT1G01010    -1.90043
    AT1G01020    AT1G01010    -1.832192
    AT1G01070    AT1G01060    -0.591932
    AT1G01070    AT1G01060    -1.204241
    AT1G01073    AT1G01070    0.790549
    AT1G01060    AT1G01070    1.214972

Expected Output

Code:
    AT1G01020    AT1G01010    -1.90043
    AT1G01070    AT1G01060    1.214972
    AT1G01073    AT1G01070    0.790549

Code:
gene_table=file1.readlines() # In the real file, j[12]=Gene1, j[13]=Gene2 and j[27]=log value
lfc=[]
for j in gene_table:
    j=j.split("\t")
    j[12]=j[12].strip()
    j[13]=j[13].strip()
    lfc=[]
    int_list=[]
    lfc.append(float(j[27]))
    int_list.append(j[0])
    dict_int={}
    for k in gene_table:
        k=k.split("\t")
        k[12]=k[12].strip()
        k[13]=k[13].strip()
        if (j[0]!=k[0]) and ((j[12]==k[12] and j[13]==k[13]) or (j[12]==k[13] and j[12]==k[13])):
            lfc.append(float(k[27]))
    dict_int=dict(zip(int_list, lfc))
    x=max(lfc, key=abs)
    #print x
    listOfKeys = [key  for (key, value) in dict_int.items() if value == x]
    print listOfKeys


Last edited by Scrutinizer; 03-11-2020 at 12:29 AM..
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to select a particular media from the printer with a UNIX command

Problem Overview: We have a scheduler that prints report on any of the network printer. Problem Statement: We need to find a UNIX command that picks up either A4, legal or letter size paper form the printer. I found out a command but it's not working on our environment. ... (3 Replies)
Discussion started by: HelpMeOUt
3 Replies

2. Shell Programming and Scripting

Select distinct values from a flat file

Hi , I have a similar problem. Please can anyone help me with a shell script or a perl. I have a flat file like this fruit country apple germany apple india banana pakistan banana saudi mango india I want to get a output like fruit country apple ... (7 Replies)
Discussion started by: smalya
7 Replies

3. Shell Programming and Scripting

Identify high values "ÿ" in a text file using Unix command

I have high values (such as ÿÿÿÿ) in a text file contained in an Unix AIX server. I need to identify all the records which are having these high values and also get the position/column number in the record structure if possible. Is there any Unix command by which this can be done to : 1.... (5 Replies)
Discussion started by: devina
5 Replies

4. Shell Programming and Scripting

Running a select script through UNIX and sending output to file

Hi, (Oracle, AIX) I have googled this and searched this forum, however I haven't had much luck with an answer and have tried several different things. Basically I have a SQL select statement which generates a whole load of UPDATE statements, I want to run the select statement via... (13 Replies)
Discussion started by: dbchud
13 Replies

5. Shell Programming and Scripting

Unix command to select first few characters and last character of a line

I have a huge file and I want to select first 10 charcters and last 2 characters of everyline and than will filter the unique line. I know, it must be easy bt I am new to unix scripting:) Ex. I have file as below and need to e3kbaird and last 2 characters. and than unique records. ... (3 Replies)
Discussion started by: Sanjeev Yadav
3 Replies

6. Shell Programming and Scripting

Comparing multiple network files (edge lists)

I want to compare 4 edge-lists to basically see if an edge is present in all 4 networks. The issue is that an edge A-B in one file can be present as B-A in another file. Example: Input 1: net1.txt A B 0.1 C D 0.65 D E 0.9 E A 0.7 Input 2: net2.txt A Z 0.1 C D 0.65 E D 0.9 E A... (1 Reply)
Discussion started by: Sanchari
1 Replies

7. Shell Programming and Scripting

UNIX command -Filter rows in fixed width file based on column values

Hi All, I am trying to select the rows in a fixed width file based on values in the columns. I want to select only the rows if column position 3-4 has the value AB I am using cut command to get the column values. Is it possible to check if cut -c3-4 = AB is true then select only that... (2 Replies)
Discussion started by: ashok.k
2 Replies

8. Shell Programming and Scripting

Split a content in a file with specific interval base on the delimited values using UNIX command

Hi All, we have a requirement to split a content in a text file every 5 rows and write in a new file . conditions: if 5th line falls between center of the statement . it should look upto after ";" files are below format: 1 UPDATE TABLE TEST1 SET VALUE ='AFDASDFAS' 2 WHERE... (3 Replies)
Discussion started by: KK230689
3 Replies
lfc2_python(3)							 Python Reference						    lfc2_python(3)

NAME
lfc2 - New python interface to the LFC client API lfc2thr - Thread enabled version of Python interface to the LFC SYNOPSIS
import lfc2 as lfc import lfc2thr as lfc lfc.init() DESCRIPTION
The lfc module permits you to access the LFC client interface from python programs. The lfc module is a swig wrapping of the standard C interface. For detailed descriptions of each function see the individual man page of each function. The lfcthr module is a version of the lfc module supporting multi-threaded Python clients. Its usage is similar to the usage of the lfc module except the obligatory initialisation call lfcthr.init() in the main program before threads are started. FUNCTIONS
lfc_aborttrans ( ) lfc_access ( string path, int amode ) -> bool access lfc_accessr ( string sfn, int amode ) -> bool access lfc_addreplica ( string guid, struct lfc_fileid *file_uniqueid, string server, string sfn, char status, char f_type, string poolname, string fs ) lfc_addreplicax ( string guid, struct lfc_fileid *file_uniqueid, string server, string sfn, char status, char f_type, string poolname, string fs, char r_type, string setname ) lfc_chdir ( string path ) lfc_chmod ( string path, mode_t mode ) lfc_chown ( string path, uid_t new_uid, gid_t new_gid ) lfc_client_setAuthorizationId ( uid_t uid, gid_t gid, string mech, string id ) lfc_client_setVOMS_data ( string voname, stringlist vomsdata ) lfc_closedir ( lfc_DIR dirp ) lfc_creatg ( string path, string guid, mode_t mode ) lfc_delcomment ( string path ) lfc_delete ( string path ) lfc_delfilesbyguid ( ListOfString guids, int force ) -> ListOfInt result lfc_delfilesbyname ( ListOfString names, intforce ) -> ListOfInt results lfc_delfilesbypattern ( string path, string pattern, int force ) -> ListOf struct lfc_filestatus results lfc_delreplica ( string guid, struct lfc_fileid *file_uniqueid, string sfn ) lfc_delreplicas ( ListOfString sfns, string se ) -> ListOfInt results lfc_delreplicasbysfn ( ListOfString sfns, ListOfString guids ) -> ListOfInt results lfc_endsess ( ) lfc_endtrans ( ) lfc_getacl ( string path, ) -> ListOf struct lfc_acl acls lfc_getcomment ( string path ) -> string comment lfc_getcwd ( ) -> string dir lfc_getifcevers ( ) -> string version lfc_getlinks ( string path, string guid ) -> ListOf struct lfc_linkinfo links lfc_getreplica ( string path, string guid, string se ) -> ListOf struct lfc_filereplica replicas lfc_getreplicas ( ListOfString guids, string se ) -> ListOf struct lfc_filereplicas replicas lfc_getreplicasl ( ListOfString paths, string se ) -> ListOf struct lfc_filereplicas replicas lfc_getreplicass ( ListOfString paths, ) -> ListOf struct lfc_filereplicas replicas lfc_getreplicax ( string path, string guid, string se ) -> ListOf struct lfc_filereplicax replicas lfc_lchown ( string path, uid_t new_uid, gid_t new_gid ) lfc_listreplicax ( string poolname, string server, string fs, int flags, lfc_list * listp ) -> struct lfc_filereplica replica lfc_listrepset ( string setname, int flags, lfc_list *listp ) -> struct lfc_filereplicax replica lfc_lstat ( string path ) -> struct lfc_filestat statbuf lfc_mkdir ( string path, mode_t mode ) lfc_mkdirg ( string path, string guid, mode_t mode ) lfc_modreplica ( string sfn, string setname, string poolname, string server ) lfc_modreplicax ( string sfn, string setname, string poolname, string server, string fs, char r_type ) lfc_opendir ( string path ) -> lfc_DIR dir lfc_opendirg ( string path, string guid ) -> lfc_DIR dir lfc_opendirxg ( string server, string path, string guid ) -> lfc_DIR dir lfc_ping ( string server ) -> string info lfc_readdir ( lfc_DIR dirp ) -> struct dirent dirent lfc_readdirc ( lfc_DIR dirp ) -> struct lfc_direncomm dirent lfc_readdirg ( lfc_DIR dirp ) -> struct lfc_direnstatg dirent lfc_readdirx ( lfc_DIR dirp ) -> struct lfc_direnstat dirent lfc_readdirxc ( lfc_DIR dirp ) -> struct lfc_direnstatc dirent lfc_readdirxp ( lfc_DIR dirp, string pattern, string se = None ) -> struct lfc_direnrep dirent lfc_readdirxr ( lfc_DIR dirp, string se = None ) -> struct lfc_direnrep dirent lfc_readlink ( string path ) -> string link lfc_registerfiles ( ListOf struct lfc_filereg files ) -> ListOfInt results lfc_rename ( string oldpath, string newpath ) lfc_rewinddir ( lfc_DIR dirp ) lfc_rmdir ( string path ) lfc_setacl ( string path, ListOf struct lfc_acl acls ) lfc_setatime ( string path, struct lfc_fileid file_uniqueid ) lfc_setcomment ( string path, string comment ) lfc_setfsize ( string path, struct lfc_fileid file_uniqueid, long filesize ) lfc_setfsizec ( string path, struct lfc_fileid file_uniqueid, long filesize, string csumtype, string csumvalue ) lfc_setfsizeg ( string guid, long filesize, string csumtype, string csumvalue ) lfc_setptime ( string sfn, long ptime ) lfc_setratime ( string sfn ) lfc_setrltime ( string sfn, long ltime ) lfc_setrstatus ( string sfn, char status ) lfc_setrtype ( string sfn, char type ) lfc_startsess ( string server, string comment ) lfc_starttrans ( string server, string comment ) lfc_stat ( string path ) -> struct lfc_filestat statbuf lfc_statg ( string path, string guid ) -> struct lfc_filestatg statbuf lfc_statr ( string sfn) ) -> struct lfc_filestatg statbuf lfc_symlink ( string target, string linkname ) lfc_umask ( mode_t cmask ) -> mode_t oldmask lfc_undelete ( string path ) lfc_unlink ( string path ) lfc_utime ( string path, struct utimbuf times ) lfc_entergrpmap ( gid_t gid, string groupname ) lfc_enterusrmap ( uid_t uid, string username ) lfc_getgrpbygid ( gid_t gid ) -> string group lfc_getgrpbygids ( ListOfgid_t gids ) -> ListOfString groups lfc_getgrpbynam ( string groupname ) -> gid_t gid lfc_getgrpmap ( ) -> ListOf struct lfc_groupinfo infos lfc_getidmap ( string username, ListOfString vo_roles ) -> uid_t uid, ListOf gid_t gids lfc_getusrbynam ( string username ) -> uid_t uid lfc_getusrbyuid ( uid_t uid ) -> string user lfc_getusrmap ( ) -> ListOf struct lfc_userinfo userinfo lfc_modifygrpmap ( gid_t gid, string newname ) lfc_modifyusrmap ( uid_t uid, string newname ) lfc_rmgrpmap ( gid_t gid, string groupname ) lfc_rmusrmap ( uid_t uid, string username ) STRUCTURES
Sometimes you need to specify a structure as an input argument (eg. struct lfc_acl, struct lfc_list, etc.). For that purpose, the module includes structure constructors, named exactly as the structure. Thus, to create struct lfc_acl or struct lfc_list, you have to do the following: lfcacl = lfc.lfc_acl() lfclist = lfc.lfc_list() If you want to see how to use it in a real case, please have a look at examples. ERRORS
As you can see, no error code is returned. When an error occurs, an exception is raised with a description of the error. EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # stat an existing entry in the LFC and print the GUID """ name = "/grid/dteam/my.test" try: stat = lfc.lfc_statg(name,"") except Exception: traceback.print_exc() sys.exit(1) guid = stat.guid print "The GUID for " + name + " is " + guid EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # setting and retrieving a comment on a file """ file = "/grid/dteam/my.test" comment = "MyComment" try: lfc.lfc_setcomment(file,comment) comment = lfc.lfc_getcomment(file) print "Read back comment " + comment except Exception: traceback.print_exc() sys.exit(1) EXAMPLE
#!/usr/bin/python """ # Using the lfc_readdirxr method """ import sys import traceback import lfc2 as lfc name = "/grid/dteam/my.test" try: dir = lfc.lfc_opendir(name) while 1: entry = lfc.lfc_readdirxr(dir) if entry == None: break print entry.d_name for i in range(entry.nbreplicas): print " ==> %s" % entry.rep[i].sfn lfc.lfc_closedir(dir) except TypeError, x: print " ==> None" except Exception: traceback.print_exc() sys.exit(1) EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # Using the lfc_getreplica method """ file = "/grid/dteam/my.test" try: list = lfc.lfc_getreplica(file, "", "") except Exception: traceback.print_exc() sys.exit(1) for i in list: print i.host print i.sfn EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # Using the lfc_getacl and lfc_setacl methods to add a user ACL """ file = "/grid/dteam/my.test" try: acls_list = lfc.lfc_getacl(file) except Exception: traceback.print_exc() sys.exit(1) for i in acls_list: print i.a_type print i.a_id print i.a_perm # When adding a first ACL for a given user, you also need to add the mask # When adding the second user ACL, it is not necessary anymore acl_user = lfc.lfc_acl() acl_mask = lfc.lfc_acl() acl_user.a_type = lfc.CNS_ACL_USER acl_user.a_id = 18701 # user id acl_user.a_perm = lfc.S_IROTH | lfc.S_IXOTH acl_mask.a_type = lfc.CNS_ACL_MASK acl_mask.a_id = 0 # no user id specified acl_mask.a_perm = lfc.S_IROTH | lfc.S_IXOTH acls_list.append(acl_user) acls_list.append(acl_mask) try: lfc.lfc_setacl(file, acls_list) except Exception: traceback.print_exc() sys.exit(1) EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # Using the lfc_getacl and lfc_setacl methods to remove a user ACL """ file = "/grid/dteam/my.test" try: acls_list = lfc.lfc_getacl(file) except Exception: traceback.print_exc() sys.exit(1) for i in acls_list: print i.a_type print i.a_id print i.a_perm del acls_list[1] # delete a given user ACL from the list of ACLs try: lfc.lfc_setacl(file, acls_list) except Exception: traceback.print_exc() sys.exit(1) EXAMPLE
#!/usr/bin/env python import lfc2thr as lfc import os from threading import Thread class slave(Thread): def __init__ (self): Thread.__init__(self) def run(self): .... result = lfc.lfc_getreplica("", guid, "") .... return if __name__ == '__main__': os.environ['LFC_HOST'] = 'my_lfc.cern.ch' # Threaded library initialisation lfc.init() .... # Start up of threads for i in xrange(totalNumberOfSlaves): slv = slave(i) slv.start() .... EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # Using the lfc_getusrmap method """ try: list = lfc.lfc_getusrmap() except Exception: traceback.print_exc() sys.exit(1) for i in list: print i.userid + " " + i.username EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # Using the lfc_getgrpmap method """ try: list = lfc.lfc_getgrpmap() except Exception: traceback.print_exc() sys.exit(1) for i in list: print i.gid + " " + i.groupname EXAMPLE
#!/usr/bin/python import sys import traceback import lfc2 as lfc """ # Using the lfc_getidmap method """ userdn = "user certificate DN" vo_roles = ["dteam"] try: uid, gids = lfc.lfc_getidmap (userdn, vo_roles) except Exception: traceback.print_exc() sys.exit(1) print "uid=", uid print "gids=", gids SEE ALSO
LFC C interface man pages LFC
$Date: 2010-12-14 08:31:05 +0100 (Tue, 14 Dec 2010) $ lfc2_python(3)
All times are GMT -4. The time now is 03:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy