data file
Actual file has approx 50 Millions such lines with bigger number
Now I need to find a way so that I can pull out all uniq sets
example
Case satisfying conditions
3 2 is uniq
as first column has 3 and 2nd column is "2" only no other 2nd number
also 5 and 1 and it appears only once.
Cases does not work
1,2 etc WHERE 1 has other numbers in 2nd column.
Already tried:
1. Tried database mysql.. Does not work
2. grep and awk : very slow .. My script is running for more than 3 days
3. Sort column and comparing with scnd column ... Need help on any unix/linux tool to do this
4. comm commands also seems to be scared of too much data ...
5. Tried perl with BINMODE and reading a BLOCK etc etc ... Slower than grep and egrep.
Any ideas on how to get this details.....
Last edited by chakrapani; 03-29-2010 at 08:59 AM..
It's hard to tell without seeing the code you run ...
Please try the following command and let us know the timings:
Use gawk, nawk or /usr/xpg4/bin/awk on Solaris.
Ok what I need is the set to remain same so 1 2 and 1 4 should not be there as 2 and 4 changes but if we have same set all the time it is ok . example 3 2 and 5 1 etc in the original code.
What I was doing in my code was to to get uniq numbers and then grep and the numbers in the file and then see if scnd number appears only once using wc -l
It works but slow ..
Getting a bit more comfortable making quick YT videos in 4K, here is:
Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search Console
https://youtu.be/I6b9T2qcqFo (0 Replies)
Hi,
I've written a script to search for an Oracle ORA- error on a log file, print that line and the .trc file associated with it as well as the dateline of when I assumed the error occured. In most it is the first dateline previous to the error.
Unfortunately, this is not a fool proof script.... (2 Replies)
Hi all,
I am working on a sample backup code, where i read the files per 7200 bytes and send it to server. Before sending to server, i compress each 7200 bytes using zlib compression algorithm using dictionary max length of 1.5 MB . I find zlib is slow.
Can anyone recommend me a... (3 Replies)
Hi,
We have a unix shell script which tries login to database. The user name and password to connect to database is stored in a file connection.sql.
Now connection.sql has contents
def ora_user =&1
CONNECT A_PROXY/abc123@DEV01
When on UNIX server we connect to database and set spool on... (7 Replies)
Hi ,
I have been trying to write a perl script to do this job. But i am not able to achieve the desired result. Below is my code.
my $current_value=12345;
my @users=("bob","ben","tom","harry");
open DBLIST,"<","/var/tmp/DBinfo";
my @input = <DBLIST>;
foreach (@users)
{
my... (11 Replies)
Hi!
Can someone explain me exactly this technique? Why a process (PARENT) creates a copy of itself with FORK (CHILD)? What's the reason of this behaviour? Sorry, but I cannot understand the logic behind it.
Thanks. (4 Replies)
Is there any better way of doing this? I only want to find a status of a diff, ie diff the file and return to me whether it is different or not or non-existant.
This example works, however I think it could be less messier:
workd=`pwd`;find $workd -name "*.sum" | while read line ; do... (1 Reply)