Sponsored Content
Full Discussion: Regarding Multi-Threading
Top Forums Programming Regarding Multi-Threading Post 78060 by matrixmadhan on Friday 15th of July 2005 04:16:56 AM
Old 07-15-2005
Regarding Multi-Threading

Hi All,

Here's my question
I have a 385 MB file containing 5,000,000 records.
I need to read from the file and load into a table.

Initially i thought of doing it in a single thread (execution of a single program) but when calculated accounted 16 hours of time on a standard benchmark.
Hence decided to do the same in multi-threaded way.

Here's the pseudo-code
total no of bytes in file: k
no of threads: n

for every 1,000,000 records
create a thread with file position pointer
k->k/1,000,000 with access upto (startpos, endpos to which it can access)
next thread would account for insertion of 1x10^6 to 2x10^6
with file postion pointers from k/1x10^6->k/2x10^6
similary continue for all other threads.

I havent tried this yet, i have following questions
regarding this.
a) do i need to have a separate lock when accesing the single file,
b)or there any chances of the file being corrupted.
c)any other memory or lock issues possible.
d)I would like to perform insertion operation quickly
any inputs and ideas greatly appreciated.
e)i definitely know that this is not such a great way any other way for achiveing the purpose would be of great help.

It would also be great if u suggest some websites for executing multiple threads concurrently.
 

10 More Discussions You Might Find Interesting

1. Programming

Multi threading using posix thread library

hi all, can anyone tell me some good site for the mutithreading tutorials, its application, and some code examples. -sushil (2 Replies)
Discussion started by: shushilmore
2 Replies

2. Programming

Multi-threading questions

I've been doing some reading lately about threading (Posix threads) and I'm really curious about a couple things that I've read. I'm not sure if many people here have threading experience, but I thought it would be nice to be able to discuss some questions about it. (For the record, I did... (1 Reply)
Discussion started by: DreamWarrior
1 Replies

3. Programming

Multi threading using fork

Hi, I have written a code which will run a set of process using fork. I want to know from You how can i start another job when one of my job in my loop is completed My code is #include<stdio.h> #include<ctype.h> main() { int pid,cid; ChildProcess(); ... (1 Reply)
Discussion started by: sureshraju_ma
1 Replies

4. Programming

Multi threading?

I am not sure if multi threading is the correct term, but here is what I am trying to do. I have a while loop that displays the number 1, pauses, displays the number 2, pauses , displays the number 3 ad infinitum. It just keeps counting. While the screen displays the sequence of numbers counting... (4 Replies)
Discussion started by: enuenu
4 Replies

5. Programming

Multi-threading-- calling same function through different threads

Sir, Can I call same function in the start routines of different Threads. I have created two different threads....and wanna call same function from both threads....is it possible??? Also can I fork inside a thread??? (1 Reply)
Discussion started by: arunchaudhary19
1 Replies

6. Programming

Multi-threading

Hi, If we create 10 threads to invoke runQuery method at same time, Will queryProcessor will be overriden sometime or 10 different copies will be created? We are not using any sunchronzation mechnism in runQuery(). so there is not gurantee on QueryProcessor class variables right OR each 10... (1 Reply)
Discussion started by: jramesh1
1 Replies

7. Programming

Multi-threading

In this piece i implemented the gossip method. The first thread is invoked from inside the (msg is first sent from node -1 to 0 from main()) and the other threads are invoked from inside of the thread function itself. I used two mutexes and a condition variable to control the synchronization. ... (4 Replies)
Discussion started by: saman_glorious
4 Replies

8. UNIX for Dummies Questions & Answers

Confusion over Multi Threading

Hi, I am trying to get my head round Multi Threading and I have a few queries to try and clear up my confusion Q1. Is multi threading a hardware / chip level concept, an OS level or an application level concept ? I am trying to work out where SMT architecture fits in. Q2. What's the multi... (3 Replies)
Discussion started by: jimthompson
3 Replies

9. UNIX for Beginners Questions & Answers

Does UNIX support multi-Threading ?

Not just background process running ... but im looking if unix has any multi-threading concept like in Java, C# ... if not present, can you pls share the nearest feature in unix that is close to multi-threaded concept (3 Replies)
Discussion started by: i4ismail
3 Replies

10. UNIX for Beginners Questions & Answers

Multi threading in UNIX

Hi, Can we apply multi threading in Unix. I am using bash shell. We have a generic script to load the data to table based on file input. For each file there is an individual table to load. For each file found in directory I want to load the data in parallel to target table using ... (3 Replies)
Discussion started by: vedanta
3 Replies
BYTEPREFIX(5)							File Formats Manual						     BYTEPREFIX(5)

NAME
byteprefix - Configuration for display of sizes DESCRIPTION
There are two standard ways to use units in computing: base 10 (1 k = 10^3 = 1 000) and base 2 (1 K = 2^10 = 1 024). Historically, most computer programs have used units in base 2, where 1 KB = 1 024 bytes, 1 MB = 1 048 576 bytes, etc. However, users are more likely to expect and understand sizes in base 10, as this is the norm outside of computing. This configuration file is a method for configuring programs (that use libkibi) to display sizes in the user's preferred style. It can be configured through a configuration file or environment variable (which takes precedence). When not using the "historic" style, IEC-style prefixes (KiB, MiB, etc.) are used for base 2 units, to disambiguate them from base 10 units (kB, MB, etc.). OPTIONS
There are three possible styles (Default: base10): base2 Display all sizes in Base 2 with IEC prefixes. 1 KiB = 1 024 bytes. 1 MiB = 1 024 KiB = 1 048 576 bytes. 1 GiB = 1 024 MiB = 1 048 576 KiB = 1 073 741 824 bytes. base10 Display all sizes in Base 10, except for sizes of RAM, which use base 2 with IEC prefixes. Everything except RAM: 1 kB = 1 000 bytes. 1 MB = 1 000 kB = 1 000 000 bytes. 1 GB = 1 000 MB = 1 000 000 kB = 1 000 000 000 bytes. RAM: 1 KiB = 1 024 bytes. 1 MiB = 1 024 KiB = 1 048 576 bytes. 1 GiB = 1 024 MiB = 1 048 576 KiB = 1 073 741 824 bytes. historic Display all sizes in Base 2, without IEC prefixes. 1 KB = 1 024 bytes. 1 MB = 1 024 KB = 1 048 576 bytes. 1 GB = 1 024 MB = 1 048 576 KB = 1 073 741 824 bytes. Not recommended. This style uses base units 2 with prefixes usually associated with base 10 units. While it uses KB rather than the SI (base 10) kB, there is no such distinction beyond the kilobyte range, and the units are ambiguous. ENVIRONMENT
BYTEPREFIX This environment variable will override the configured or default style. It should just contain one of the style names, listed in OPTIONS above. XDG_CONFIG_HOME The location of the user's configuration files. If not set, it will be assumed to be ~/.config. FILES
The preferred style can be set in a system-wide configuration file and/or in user's own configuration file (which will take precedence). If no configuration file exists, the default style is base10. /etc/byteprefix or XDG_CONFIG_HOME/byteprefix This file should contain a single line: format=style. Lines beginning with # are treated as comments. EXAMPLE
A user wanting base 2 display can set the following in ~/.config/byteprefix: format=base2 SEE ALSO
units(7) libkibi January 2011 BYTEPREFIX(5)
All times are GMT -4. The time now is 01:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy