File 1 contains a reference list of unique subscriber numbers ( 7 million entries in total)
File 2 contains a list of the subscriber numbers and their tariff (15 million entries in total). This file is in the production system and hasn't had old subscribers removed for some time so more than half of the entries need removed.
I created the following couple of lines to try to obtain the active 7 million subscriber numbers and tariffs from the behemoth 15 million list
While this is actually working, it's only producing around 20 entries per second, which will take days to complete.
I'm afraid I'm a complete noob and can't come up with anything more inventive. I'm sure awk/sed/perl or probably any other number of languages would be perfect for something like this.
I am trying to cat a file and then grep that file for a number. I can do it fine on other files but this particular file will not do anything. I tried running it on an older file from the same device but it is just not working. The file is nothing more than a flat file on a unix box. Here is just a... (3 Replies)
Hello,
So I sorted my file as I was supposed to:
sort -n -r -k 2 -k 1 file1 | uniq > file2
and when I wrote
> cat file2
in the command line, I got what I was expecting, but in the script itself
...
sort -n -r -k 2 -k 1 averages | uniq > temp
cat file2
It wrote a whole... (21 Replies)
Hi all,
I have a file like the following:
ID,
2,Andrew,0,1,2,3,4,2,5,6,7,7,9,3,4,5,34,3,2,1,5,6,78,89,8,7,6......................
4,James,0,6,7,0,5,6,4,7,8,9,6,46,6,3,2,5,6,87,0,341,0,5,2,5,6....................
END,
(there are more entires on each line but to keep it simple I've left... (10 Replies)
Hi all,
Here is my requirement
I have to search 'ORA' word in out.log file,if it is present then i need to send that file (out.log) content to some mail id.If 'ORA' word is not in that file then i need to send 'load succesful' message to some mail id.
The below the shell script is not... (5 Replies)
Hi All,
I'd like to do this
cat /etc/passwd
and grep -v on the /etc/shells list
I'd like to find all shell that doesn't exist on the /etc/passwd.
Is there an easy way without doing a egrep -v "/bin/sh|/bin/bash................"?
How do I use a file /etc/shells as my list for... (4 Replies)
I am not sure if using cat -n is the most efficient way to split a file into multiple files, one file per line in the source file.
I thought using cat -n would make it easy to process the file because it produces an output that numbers each line that I could then grep for with the regex "^ *$i".... (3 Replies)
Is there a way using grep or cat a file to create a new file based on whether the first 9 positions of each record is less than 399999999?
This is a fixed file format. (3 Replies)
Hello,
i need to search one word (snp1) from many files and copy the content of the columns of this word in new file.
example:
file 1:
SNP BP CHR P
snp1 1 3 0.01
snp2 2 2 0.05
.
.
file 2:
SNP BP CHR P
snp1 1 3 0.06
snp2 2 2 0.3
output... (6 Replies)
Hello
someone told me to use
OS=`awk '{print int($3)}' < /etc/redhat-release`
instead of
OS=cat /etc/redhat-release | `awk '{print int($3)}'`
any idea for the reason ? (5 Replies)
Hi Guys
This is my first post so I am not sure how things go here. I'm sorry if I'm breaking the rule or something. Feel free to correct me about that :)
So as I was saying...
I'd been trying to grep this folder containing 900,000 txt files but seems no luck. I get either "No such file... (6 Replies)
Discussion started by: Nexeu
6 Replies
LEARN ABOUT SUSE
dbd::file
DBD::File(3) User Contributed Perl Documentation DBD::File(3)NAME
DBD::File - Base class for writing DBI drivers
SYNOPSIS
This module is a base class for writing other DBDs.
It is not intended to function as a DBD itself.
If you want to access flatfiles, use DBD::AnyData, or DBD::CSV,
(both of which are subclasses of DBD::File).
DESCRIPTION
The DBD::File module is not a true DBI driver, but an abstract base class for deriving concrete DBI drivers from it. The implication is,
that these drivers work with plain files, for example CSV files or INI files. The module is based on the SQL::Statement module, a simple
SQL engine.
See DBI for details on DBI, SQL::Statement for details on SQL::Statement and DBD::CSV or DBD::IniFile for example drivers.
Metadata
The following attributes are handled by DBI itself and not by DBD::File, thus they all work like expected:
Active
ActiveKids
CachedKids
CompatMode (Not used)
InactiveDestroy
Kids
PrintError
RaiseError
Warn (Not used)
The following DBI attributes are handled by DBD::File:
AutoCommit
Always on
ChopBlanks
Works
NUM_OF_FIELDS
Valid after "$sth-"execute>
NUM_OF_PARAMS
Valid after "$sth-"prepare>
NAME
Valid after "$sth-"execute>; undef for Non-Select statements.
NULLABLE
Not really working, always returns an array ref of one's, as DBD::CSV doesn't verify input data. Valid after "$sth-"execute>; undef for
Non-Select statements.
These attributes and methods are not supported:
bind_param_inout
CursorName
LongReadLen
LongTruncOk
Additional to the DBI attributes, you can use the following dbh attribute:
f_dir
This attribute is used for setting the directory where CSV files are opened. Usually you set it in the dbh, it defaults to the current
directory ("."). However, it is overwritable in the statement handles.
f_ext
This attribute is used for setting the file extension where (CSV) files are opened. There are several possibilities.
DBI:CSV:f_dir=data;f_ext=.csv
In this case, DBD::File will open only "table.csv" if both "table.csv" and "table" exist in the datadir. The table will still be named
"table". If your datadir has files with extensions, and you do not pass this attribute, your table is named "table.csv", which is
probably not what you wanted. The extension is always case-insensitive. The table names are not.
DBI:CSV:f_dir=data;f_ext=.csv/r
In this case the extension is required, and all filenames that do not match are ignored.
f_schema
This will set the schema name. Default is the owner of the folder in which the table file resides. "undef" is allowed.
my $dbh = DBI->connect ("dbi:CSV:", "", "", {
f_schema => undef,
f_dir => "data",
f_ext => ".csv/r",
}) or die $DBI::errstr;
The effect is that when you get table names from DBI, you can force all tables into the same (or no) schema:
my @tables $dbh->tables ();
# no f_schema
"merijn".foo
"merijn".bar
# f_schema => "dbi"
"dbi".foo
"dbi".bar
# f_schema => undef
foo
bar
Driver private methods
data_sources
The "data_sources" method returns a list of subdirectories of the current directory in the form "DBI:CSV:f_dir=$dirname".
If you want to read the subdirectories of another directory, use
my ($drh) = DBI->install_driver ("CSV");
my (@list) = $drh->data_sources (f_dir => "/usr/local/csv_data" );
list_tables
This method returns a list of file names inside $dbh->{f_dir}. Example:
my ($dbh) = DBI->connect ("DBI:CSV:f_dir=/usr/local/csv_data");
my (@list) = $dbh->func ("list_tables");
Note that the list includes all files contained in the directory, even those that have non-valid table names, from the view of SQL.
KNOWN BUGS
o The module is using flock () internally. However, this function is not available on all platforms. Using flock () is disabled on
MacOS and Windows 95: There's no locking at all (perhaps not so important on MacOS and Windows 95, as there's a single user
anyways).
AUTHOR
This module is currently maintained by
H.Merijn Brand < h.m.brand at xs4all.nl > and Jens Rehsack < rehsack at googlemail.com >
The original author is Jochen Wiedmann.
COPYRIGHT AND LICENSE
Copyright (C) 2009 by H.Merijn Brand & Jens Rehsack Copyright (C) 2004 by Jeff Zucker Copyright (C) 1998 by Jochen Wiedmann
All rights reserved.
You may freely distribute and/or modify this module under the terms of either the GNU General Public License (GPL) or the Artistic License,
as specified in the Perl README file.
SEE ALSO
DBI, Text::CSV, Text::CSV_XS, SQL::Statement
perl v5.12.1 2009-06-05 DBD::File(3)