07-15-2005
Regarding Multi-Threading
Hi All,
Here's my question
I have a 385 MB file containing 5,000,000 records.
I need to read from the file and load into a table.
Initially i thought of doing it in a single thread (execution of a single program) but when calculated accounted 16 hours of time on a standard benchmark.
Hence decided to do the same in multi-threaded way.
Here's the pseudo-code
total no of bytes in file: k
no of threads: n
for every 1,000,000 records
create a thread with file position pointer
k->k/1,000,000 with access upto (startpos, endpos to which it can access)
next thread would account for insertion of 1x10^6 to 2x10^6
with file postion pointers from k/1x10^6->k/2x10^6
similary continue for all other threads.
I havent tried this yet, i have following questions
regarding this.
a) do i need to have a separate lock when accesing the single file,
b)or there any chances of the file being corrupted.
c)any other memory or lock issues possible.
d)I would like to perform insertion operation quickly
any inputs and ideas greatly appreciated.
e)i definitely know that this is not such a great way any other way for achiveing the purpose would be of great help.
It would also be great if u suggest some websites for executing multiple threads concurrently.
10 More Discussions You Might Find Interesting
1. Programming
hi all,
can anyone tell me some good site for the mutithreading tutorials, its application, and some code examples.
-sushil (2 Replies)
Discussion started by: shushilmore
2 Replies
2. Programming
I've been doing some reading lately about threading (Posix threads) and I'm really curious about a couple things that I've read. I'm not sure if many people here have threading experience, but I thought it would be nice to be able to discuss some questions about it.
(For the record, I did... (1 Reply)
Discussion started by: DreamWarrior
1 Replies
3. Programming
Hi,
I have written a code which will run a set of process using
fork.
I want to know from You how can i start another job when one of my job in my loop is completed
My code is
#include<stdio.h>
#include<ctype.h>
main() {
int pid,cid;
ChildProcess();
... (1 Reply)
Discussion started by: sureshraju_ma
1 Replies
4. Programming
I am not sure if multi threading is the correct term, but here is what I am trying to do. I have a while loop that displays the number 1, pauses, displays the number 2, pauses , displays the number 3 ad infinitum. It just keeps counting.
While the screen displays the sequence of numbers counting... (4 Replies)
Discussion started by: enuenu
4 Replies
5. Programming
Sir,
Can I call same function in the start routines of different Threads.
I have created two different threads....and wanna call same function from both threads....is it possible???
Also can I fork inside a thread??? (1 Reply)
Discussion started by: arunchaudhary19
1 Replies
6. Programming
Hi, If we create 10 threads to invoke runQuery method at same time, Will queryProcessor will be overriden sometime or 10 different copies will be created?
We are not using any sunchronzation mechnism in runQuery(). so there is not gurantee on QueryProcessor class variables right OR each 10... (1 Reply)
Discussion started by: jramesh1
1 Replies
7. Programming
In this piece i implemented the gossip method. The first thread is invoked from inside the (msg is first sent from node -1 to 0 from main()) and the other threads are invoked from inside of the thread function itself. I used two mutexes and a condition variable to control the synchronization. ... (4 Replies)
Discussion started by: saman_glorious
4 Replies
8. UNIX for Dummies Questions & Answers
Hi, I am trying to get my head round Multi Threading and I have a few queries to try and clear up my confusion
Q1. Is multi threading a hardware / chip level concept, an OS level or an application level concept ? I am trying to work out where SMT architecture fits in.
Q2. What's the multi... (3 Replies)
Discussion started by: jimthompson
3 Replies
9. UNIX for Beginners Questions & Answers
Not just background process running ... but im looking if unix has any multi-threading concept like in Java, C# ... if not present, can you pls share the nearest feature in unix that is close to multi-threaded concept (3 Replies)
Discussion started by: i4ismail
3 Replies
10. UNIX for Beginners Questions & Answers
Hi,
Can we apply multi threading in Unix. I am using bash shell.
We have a generic script to load the data to table based on file input. For each file there is an individual table to load.
For each file found in directory I want to load the data in parallel to target table using
... (3 Replies)
Discussion started by: vedanta
3 Replies
LEARN ABOUT DEBIAN
cfdisk
CFDISK(8) GNU cfdisk Manual CFDISK(8)
NAME
GNU cfdisk - a curses-based partition table manipulation program
SYNOPSIS
cfdisk [options] [device]
DESCRIPTION
cfdisk is a disk partition manipulation program, which allows you to create, destroy, resize, move and copy partitions on a hard drive
using a simple menu-driven interface. It is useful for organising the disk space on a new drive, reorganising an old drive, creating space
for new operating systems, and copying data to new hard disks. For a list of the supported partition types, see the --list-partition-types
option below.
OPTIONS
-h, --help
displays a help message.
-v, --version
displays the program's version.
-a, --arrow-cursor
use an arrow cursor, instead of reverse video highlighting, in case your terminal doesn't support it.
-z, --new-table
create a new partition table on the disk. This is useful if you want to change the partition table type or want to repartition you
entire drive. Note that this does not delete the old table on the disk until you commit the changes.
-u, --units=UNIT
sets the default display units to UNIT. A list of possible units is given below.
-t, --list-partition-types
displays a list of supported partition types and features.
UNITS
You can choose in what unit cfdisk should display quantities like partition sizes. You can choose from sectors, percents, bytes, kilobytes,
etc. Note that one kilobyte is equal to 1,000 bytes, as this is consistent with the SI prefixes and is used by hard disk manufacturers. If
you prefer to see the sizes in units with binary prefixes, you should instead select one kilo binary byte (kibibyte), which is equal to
1,024 bytes. Whatever display unit you have chosen, you can always enter the quantities in the unit of your choice, for example 1000000B or
1000kB.
compact
display each size in the most suitable unit from B, kB, MB, GB and TB.
B one byte
kB one kilobyte (1,000 bytes)
MB one megabyte (1,000,000 bytes)
GB one gigabyte (1,000,000,000 bytes)
TB one terabyte (1,000,000,000,000 bytes)
KiB one kilo binary byte (1,024 bytes)
MiB one mega binary byte (1,048,576 bytes)
GiB one giga binary byte (1,073,741,824 bytes)
TiB one tera binary byte (1,099,511,627,776 bytes)
s one sector. It depends on the sector size of the disk. You can use it if you want to see or choose the exact size in sectors.
% one percent from the size of the disk
cyl one cylinder. It depends on the cylinder size.
chs use CHS display units.
BUGS
There are no known bugs. We are in early stages for development, so be careful.
SEE ALSO
fdisk(8), mkfs(8), parted(8) The cfdisk program is fully documented in the info(1) format GNU cfdisk User Manual manual.
fdisk 16 June, 2006 CFDISK(8)