02-13-2017
Thanks. The command works fine. Just one thing.It takes really huge time to split a file say for size 3 GB. Is there a workaround for this? And just a small correction - Changed n=20000000 to n=2000000 as need files in chunks of 2 million rows and not 20 million.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have a directory of files that I need to rename by splitting the first and second halves of the filenames using the delimiter "-O" and then renaming with the second half first, followed by two underscores and then the first half. For example, natfinal1995annvol1_14.pdf -O filenum-20639 will be... (2 Replies)
Discussion started by: swimulator
2 Replies
2. Shell Programming and Scripting
Hi,
can some one please help me to split the file based on groups.
like in the below scenario x indicates the begining of the group and the file should be split each with 2 groups below there are 10 groups it should create 5 files. could you please help? (4 Replies)
Discussion started by: hitmansilentass
4 Replies
3. Shell Programming and Scripting
Here is my source, i have million lines like this on a file.
disp0201.php?poc=4060&roc=1&ps=R&ooc=13&mjv=6&mov=5&rel=5&bod=155&oxi=2&omj=5&ozn=1&dav=20&cd=&daz=& drc=&mo=&sid=&lang=EN&loc=JPN
I want to split this into columns in order to load in database, anything starts with"&mjv=6" as first... (13 Replies)
Discussion started by: elamurugu
13 Replies
4. Shell Programming and Scripting
Hi,
My inputfile contains field separaer is ^.
12^inms^
13^fakdks^ssk^s3
23^avsd^
13^fakdks^ssk^a4
I wanted to print only 2 delimiter occurence i.e
12^inms^
23^avsd^ (4 Replies)
Discussion started by: Jairaj
4 Replies
5. Shell Programming and Scripting
KSH
HP-SOL-Lin
Cannot use xAWK
I have several strings that are quite long and i want to break them down into smaller substrings.
What I have
String = "word1 word2 word3 word4 .....wordx"
What I want
String1="word1 word2"
String2="word 3 word4"
String3="word4 word5"
Stringx="wordx... (5 Replies)
Discussion started by: nitrobass24
5 Replies
6. Shell Programming and Scripting
Hi all.
I've tried searching the web but could not find similar problem to mine.
I have one large file to be splitted into several files based on the matching pattern found in each row.
For example, let's say the file content:
... (13 Replies)
Discussion started by: aimy
13 Replies
7. Shell Programming and Scripting
i have below code to count number of rows in file1.txt, if the row count is more than one then i have sending an email along with file1.txt attached and fail the process(do nothing if count is <=1),
if I test individually count part works good but when i include the email part its not working,... (1 Reply)
Discussion started by: srini_106
1 Replies
8. Shell Programming and Scripting
Hi
I want to check delimiter in file. Delimiter in my file is ~|*
sample of file :
ABC~|*edgf~|*T1J333~|*20121130
ABC~|*sdaf~|*T1J333~|*20121130
ABC~|*fsdg~|*T1J333~|*20121130
ABC~|*dfsg~|*T1J333~|*20121130
in this i want to count number delimiter occur is 4 in each row if count is... (21 Replies)
Discussion started by: MOHANP12
21 Replies
9. Shell Programming and Scripting
Greetings Experts,
I am generating a validation query through awk and facing an issue, which I need to overcome by splitting the data based on the pattern matching count in the value of an array.
File1 --
Table11@column1@date@Table21@column1@varchar(10)@d;... (4 Replies)
Discussion started by: chill3chee
4 Replies
10. Shell Programming and Scripting
The input file is a .dat file which is delimited by null (^@ in Linux). On a windows PC it looks something like this (numbers are masked with 1).
https://i.imgur.com/nta2Gqp.jpg
The entire file is in one row but it has multiple records - each record contains 80 fields i.e. there are 81 counts... (9 Replies)
Discussion started by: digitalnirvana
9 Replies
LEARN ABOUT DEBIAN
dbix::class::helper::resultset::random
DBIx::Class::Helper::ResultSet::Random(3pm) User Contributed Perl Documentation DBIx::Class::Helper::ResultSet::Random(3pm)
NAME
DBIx::Class::Helper::ResultSet::Random - Get random rows from a ResultSet
VERSION
version 2.013002
SYNOPSIS
# note that this is normally a component for a ResultSet
package MySchema::ResultSet::Bar;
use strict;
use warnings;
use parent 'DBIx::Class::ResultSet';
__PACKAGE__->load_components('Helper::ResultSet::Random');
# in code using resultset:
my $random_row = $schema->resultset('Bar')->rand->single;
DESCRIPTION
This component allows convenient selection of random rows. See "NOTE" in DBIx::Class::Helper::ResultSet for a nice way to apply it to your
entire schema.
Currently this works by doing something akin to
SELECT TOP($x) from $table ORDER BY RANDOM()
Lots of people think this is slow. My own benchmarks show that doing the above, for 10 rows in a table with just over 8 million rows, is
nearly instant. Although that was with SQL Server, and different databases will handle that differently.
So please, if you have performance issues and want this to work with your database, get in touch and I will do what I can to get it to work
quickly enough to suite your needs.
METHODS
rand
This method takes a single argument, being the size of the random ResultSet to return. It defaults to 1. This Component will throw
exceptions if the argument is not an integer or not greater than zero.
_rand_order_by
This module currently does an "ORDER BY" on some db specific function. If for some reason it guesses incorrectly for your database the
easiest way to fix that in the short-term (ie without patching upstream) is to override this method. So for example, if your db uses
"RAND()" instead of "RANDOM()" and it's not in the predefined list of dbs you could just do the following in your ResultSet class:
sub _rand_order_by { 'RAND()' }
AUTHOR
Arthur Axel "fREW" Schmidt <frioux+cpan@gmail.com>
COPYRIGHT AND LICENSE
This software is copyright (c) 2012 by Arthur Axel "fREW" Schmidt.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
perl v5.14.2 2012-06-18 DBIx::Class::Helper::ResultSet::Random(3pm)