Comparing a bigger network file with a smaller one


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing a bigger network file with a smaller one
# 1  
Old 08-02-2018
Comparing a bigger network file with a smaller one

Hello, I have two edgelists. One bigger list master.txt and a subset of that, child.txt. I want to print out all the edges in master.txt which is not there in child.txt. I have done it the Python way, but its taking way to much time as the number of edges are huge. (one thing is that A-B and B-A are same, that is there is no directionality)

master.txt

Code:
A    B
C    D
E    F

child.txt

Code:
B    A
C    D

output:
Code:
E    F

# 2  
Old 08-02-2018
# 3  
Old 08-02-2018
Well, no body posted any answers in that. I did using Python.
# 4  
Old 08-02-2018
No posts because you were asked to show your own efforts first. Even failed approaches give base to discussions / improvements / learnings.
# 5  
Old 08-02-2018
Quote:
Originally Posted by RudiC
No posts because you were asked to show your own efforts first. Even failed approaches give base to discussions / improvements / learnings.
I am just asking for help so that I can process my data faster. Efforts were did put in, but not using shell, but thanks for your interest to point things out
# 6  
Old 08-02-2018
What did you try?
# 7  
Old 08-03-2018
Quote:
Originally Posted by Corona688
What did you try?
python - Comparing two network edge lists - Stack Overflow
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

2. Shell Programming and Scripting

Comparing multiple network files (edge lists)

I want to compare 4 edge-lists to basically see if an edge is present in all 4 networks. The issue is that an edge A-B in one file can be present as B-A in another file. Example: Input 1: net1.txt A B 0.1 C D 0.65 D E 0.9 E A 0.7 Input 2: net2.txt A Z 0.1 C D 0.65 E D 0.9 E A... (1 Reply)
Discussion started by: Sanchari
1 Replies

3. Shell Programming and Scripting

Comparing floating number and replace the smaller one with awk?

Hi guys, I tried to write a script for replacing some number in a bunch of files. However, I failed because my limited bash scripting knowledge. Here I will explain the expected function: I have many data files need to be processed. Here shows one part of a file as an example: a list with... (7 Replies)
Discussion started by: liuzhencc
7 Replies

4. Shell Programming and Scripting

How to split a file into smaller files

Hi, I have a big text file with m columns and n rows. The format is like: STF123450001000200030004STF123450005000600070008STF123450009001000110012 STF234560345002208330154STF234590705620600070080STF234567804094562357688 STF356780001000200030004STF356780005000600070080STF356780800094562657687... (2 Replies)
Discussion started by: wintersnow2011
2 Replies

5. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

6. Shell Programming and Scripting

Lookup two values per line (from a second file) and write the smaller value to another file

Hello Unix Gurus, Please let me know if this is hard to understand and I apologize for my inability to explain better. I have a file "Foo" with the following structure CHR_A BP_A SNP_A CHR_B BP_B SNP_B R2 1 ... (3 Replies)
Discussion started by: genehunter
3 Replies

7. Programming

Can't create file bigger than 2GB with my application

Hi, I've created a simple application that is supposed to fill up a file with messages up to the size I pass as parameter. The problem is that once the file reaches the 2GB size, it stops growing. The flow of the application, for what is worth, is as follows: while ( bytes written <... (7 Replies)
Discussion started by: emitrax
7 Replies

8. Shell Programming and Scripting

splitting text file into smaller ones

Hello We have a text file with 400,000 lines and need to split into multiple files each with 5000 lines ( will result in 80 files) Got an idea of using head and tail commands to do that with a loop but looked not efficient. Please advise the simple and yet effective way to do it. TIA... (3 Replies)
Discussion started by: prvnrk
3 Replies

9. UNIX for Dummies Questions & Answers

Have problem transfer large file bigger 1GB

Hi folks, I have a big problem.... and need help from your experience/knowledge. I previously install and use FREEBSD 7.0 release on my storage/backup file server, for some reason, I can not transfer any files that is bigger than 1GB. If I transfer it to Freebsd file server, the system... (2 Replies)
Discussion started by: bsdme2
2 Replies
Login or Register to Ask a Question