Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Best search technique
Top Forums Shell Programming and Scripting Best search technique Post 302408321 by chakrapani on Monday 29th of March 2010 07:53:48 AM
Old 03-29-2010
Best search technique - Need help
I have a snippet file with the shown below:

data file
Code:
1 2
1 3
1 3
4 2
3 2
2 1
2 2
5 1
3 2
3 2
2 3
1 4

Actual file has approx 50 Millions such lines with bigger number

Now I need to find a way so that I can pull out all uniq sets
example

Case satisfying conditions
3 2 is uniq
as first column has 3 and 2nd column is "2" only no other 2nd number
also 5 and 1 and it appears only once.

Cases does not work
1,2 etc WHERE 1 has other numbers in 2nd column.

Already tried:
1. Tried database mysql.. Does not work
2. grep and awk : very slow .. My script is running for more than 3 days
3. Sort column and comparing with scnd column ... Need help on any unix/linux tool to do this
4. comm commands also seems to be scared of too much data ...
5. Tried perl with BINMODE and reading a BLOCK etc etc ... Slower than grep and egrep.

Any ideas on how to get this details.....
Last edited by chakrapani; 03-29-2010 at 08:59 AM..
 

8 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Difference Technique's???

Is there any better way of doing this? I only want to find a status of a diff, ie diff the file and return to me whether it is different or not or non-existant. This example works, however I think it could be less messier: workd=`pwd`;find $workd -name "*.sum" | while read line ; do... (1 Reply)
Discussion started by: Shakey21
1 Replies

2. UNIX for Dummies Questions & Answers

FORK/EXEC technique

Hi! Can someone explain me exactly this technique? Why a process (PARENT) creates a copy of itself with FORK (CHILD)? What's the reason of this behaviour? Sorry, but I cannot understand the logic behind it. Thanks. (4 Replies)
Discussion started by: marshmallow
4 Replies

3. Shell Programming and Scripting

Perl - use search keywords from array and search a file and print 3rd field when matched

Hi , I have been trying to write a perl script to do this job. But i am not able to achieve the desired result. Below is my code. my $current_value=12345; my @users=("bob","ben","tom","harry"); open DBLIST,"<","/var/tmp/DBinfo"; my @input = <DBLIST>; foreach (@users) { my... (11 Replies)
Discussion started by: chidori
11 Replies

4. Shell Programming and Scripting

Password Obscuring Technique

Hi, We have a unix shell script which tries login to database. The user name and password to connect to database is stored in a file connection.sql. Now connection.sql has contents def ora_user =&1 CONNECT A_PROXY/abc123@DEV01 When on UNIX server we connect to database and set spool on... (7 Replies)
Discussion started by: Gangadhar Reddy
7 Replies

5. Linux

Best Compression technique ?

Hi all, I am working on a sample backup code, where i read the files per 7200 bytes and send it to server. Before sending to server, i compress each 7200 bytes using zlib compression algorithm using dictionary max length of 1.5 MB . I find zlib is slow. Can anyone recommend me a... (3 Replies)
Discussion started by: selvarajvss
3 Replies

6. Shell Programming and Scripting

Perl - start search by using search button or by pressing the enter key

#Build label and text box $main->Label( -text => "Input string below:" )->pack(); $main->Entry( -textvariable => \$text456 )->pack(); $main->Button( -text => "Search", -command => sub { errchk ($text456) ... (4 Replies)
Discussion started by: popeye
4 Replies

7. Shell Programming and Scripting

Search pattern on logfile and search for day/dates and skip duplicate lines if any

Hi, I've written a script to search for an Oracle ORA- error on a log file, print that line and the .trc file associated with it as well as the dateline of when I assumed the error occured. In most it is the first dateline previous to the error. Unfortunately, this is not a fool proof script.... (2 Replies)
Discussion started by: newbie_01
2 Replies

8. What is on Your Mind?

YouTube: Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search

Getting a bit more comfortable making quick YT videos in 4K, here is: Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search Console https://youtu.be/I6b9T2qcqFo (0 Replies)
Discussion started by: Neo
0 Replies

LEARN ABOUT CENTOS

platform::shell

platform::shell(n)					       Tcl Bundled Packages						platform::shell(n)

__________________________________________________________________________________________________________________________________________________

NAME

       platform::shell - System identification support code and utilities

SYNOPSIS

       package require platform::shell ?1.1.4?

       platform::shell::generic shell
       platform::shell::identify shell
       platform::shell::platform shell
_________________________________________________________________

DESCRIPTION

       The platform::shell package provides several utility commands useful for the identification of the architecture of a specific Tcl shell.

       This  package  allows the identification of the architecture of a specific Tcl shell different from the shell running the package. The only
       requirement is that the other shell (identified by its path), is actually executable on the current machine.

       While for most platform this means that the architecture of the interrogated shell is identical to the architecture of  the  running  shell
       this  is  not generally true. A counter example are all platforms which have 32 and 64 bit variants and where a 64bit system is able to run
       32bit code. For these running and interrogated shell may have different 32/64 bit settings and thus different identifiers.

       For applications like a code repository it is important to identify the architecture of the shell which will  actually  run  the  installed
       packages, versus the architecture of the shell running the repository software.

COMMANDS

       platform::shell::identify shell
	      This command does the same identification as platform::identify, for the specified Tcl shell, in contrast to the running shell.

       platform::shell::generic shell
	      This command does the same identification as platform::generic, for the specified Tcl shell, in contrast to the running shell.

       platform::shell::platform shell
	      This command returns the contents of tcl_platform(platform) for the specified Tcl shell.

KEYWORDS

       operating system, cpu architecture, platform, architecture

platform::shell 						       1.1.4							platform::shell(n)
All times are GMT -4. The time now is 12:40 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy