Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Comparing a bigger network file with a smaller one
Top Forums Shell Programming and Scripting Comparing a bigger network file with a smaller one Post 303021010 by Sanchari on Thursday 2nd of August 2018 03:45:21 PM
Old 08-02-2018
Comparing a bigger network file with a smaller one
Hello, I have two edgelists. One bigger list master.txt and a subset of that, child.txt. I want to print out all the edges in master.txt which is not there in child.txt. I have done it the Python way, but its taking way to much time as the number of edges are huge. (one thing is that A-B and B-A are same, that is there is no directionality)

master.txt

Code:
A    B
C    D
E    F

child.txt

Code:
B    A
C    D

output:
Code:
E    F

 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Have problem transfer large file bigger 1GB

Hi folks, I have a big problem.... and need help from your experience/knowledge. I previously install and use FREEBSD 7.0 release on my storage/backup file server, for some reason, I can not transfer any files that is bigger than 1GB. If I transfer it to Freebsd file server, the system... (2 Replies)
Discussion started by: bsdme2
2 Replies

2. Shell Programming and Scripting

splitting text file into smaller ones

Hello We have a text file with 400,000 lines and need to split into multiple files each with 5000 lines ( will result in 80 files) Got an idea of using head and tail commands to do that with a loop but looked not efficient. Please advise the simple and yet effective way to do it. TIA... (3 Replies)
Discussion started by: prvnrk
3 Replies

3. Programming

Can't create file bigger than 2GB with my application

Hi, I've created a simple application that is supposed to fill up a file with messages up to the size I pass as parameter. The problem is that once the file reaches the 2GB size, it stops growing. The flow of the application, for what is worth, is as follows: while ( bytes written <... (7 Replies)
Discussion started by: emitrax
7 Replies

4. Shell Programming and Scripting

Lookup two values per line (from a second file) and write the smaller value to another file

Hello Unix Gurus, Please let me know if this is hard to understand and I apologize for my inability to explain better. I have a file "Foo" with the following structure CHR_A BP_A SNP_A CHR_B BP_B SNP_B R2 1 ... (3 Replies)
Discussion started by: genehunter
3 Replies

5. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

6. Shell Programming and Scripting

How to split a file into smaller files

Hi, I have a big text file with m columns and n rows. The format is like: STF123450001000200030004STF123450005000600070008STF123450009001000110012 STF234560345002208330154STF234590705620600070080STF234567804094562357688 STF356780001000200030004STF356780005000600070080STF356780800094562657687... (2 Replies)
Discussion started by: wintersnow2011
2 Replies

7. Shell Programming and Scripting

Comparing floating number and replace the smaller one with awk?

Hi guys, I tried to write a script for replacing some number in a bunch of files. However, I failed because my limited bash scripting knowledge. Here I will explain the expected function: I have many data files need to be processed. Here shows one part of a file as an example: a list with... (7 Replies)
Discussion started by: liuzhencc
7 Replies

8. Shell Programming and Scripting

Comparing multiple network files (edge lists)

I want to compare 4 edge-lists to basically see if an edge is present in all 4 networks. The issue is that an edge A-B in one file can be present as B-A in another file. Example: Input 1: net1.txt A B 0.1 C D 0.65 D E 0.9 E A 0.7 Input 2: net2.txt A Z 0.1 C D 0.65 E D 0.9 E A... (1 Reply)
Discussion started by: Sanchari
1 Replies

9. Shell Programming and Scripting

Splitting a text file into smaller files with awk, how to create a different name for each new file

Hello, I have some large text files that look like, putrescine Mrv1583 01041713302D 6 5 0 0 0 0 999 V2000 2.0928 -0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 5.6650 0.2063 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3.5217 ... (3 Replies)
Discussion started by: LMHmedchem
3 Replies

LEARN ABOUT DEBIAN

mpi_kmeans

MPI_KMEANS(1)						      General Commands Manual						     MPI_KMEANS(1)

NAME

       mpi_kmeans - K-Means clustering tool

SYNOPSIS

       mpi_kmeans [options]

DESCRIPTION

       mpi_kmeans  is a program that uses k-means clustering to produce a list of cluster centers. The resulting data can be used by mpi_assign(1)
       to assign points to those cluster centers.

OPTIONS

       A summary of options is included below.

   Generic Options:
       --help Produce help message

   Input/Output Options:
       --data file
	      Training file, one datum per line (default: "data.txt")

       --output file
	      Output file, one cluster center per line (default: "output.txt")

   K-Means Options:
       --k num
	      Number of clusters to generate (default: 100)

       --restarts num
	      Number of k-means restarts (default: 0 = single run)

       --maxiter num
	      Maximum number of k-means iterations (default: 0 = infinity)

EXAMPLES

	      mpi_kmeans --k 2 --data example.txt --output clusters.txt

SEE ALSO

       mpi_assign(1)

AUTHOR

       mpi_kmeans was written by Peter Gehler <peter.gehler@tuebingen.mpg.de>.

       This manual page was written by Christian Kastner <debian@kvr.at>, for the Debian project (and may be used by others).

								  April 11, 2011						     MPI_KMEANS(1)
All times are GMT -4. The time now is 01:42 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy