Sponsored Content
Top Forums Shell Programming and Scripting Parallel Runs in UNIX/Linix Shell Scripting Post 302975443 by prawinmca on Monday 13th of June 2016 10:16:13 AM
Old 06-13-2016
Parallel Runs in UNIX/Linix Shell Scripting

HI,

I have a file(suppose 1 million Records), i wanted to divide file into chunks(100 small files), these all files should run parllell(e.x like threads) to utilize the process for fast processing.

Can you please provide a solution for the above issue.

Ex:
Code:
 Split -b 2m file.txt file, filea, fileb, filec...

Required to read all the above created chunk files(filea fileb, filec ...) parllell.

Thanks,
Praveen

Last edited by jim mcnamara; 06-13-2016 at 03:13 PM..
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

UNIX problem? Unix programm runs windows 2000 CPU over 100%

Okee problems...!! What is happening: Unix server with some programms, workstations are windows 2000, the workstations work good but when you start a programm on the Unix server the CPU of the workstations go to 100% usage resulting that the system gets very slow. The programm well its running so... (2 Replies)
Discussion started by: zerocool
2 Replies

2. Shell Programming and Scripting

difference between AIX shell scripting and Unix shell scripting.

please give the difference between AIX shell scripting and Unix shell scripting. (2 Replies)
Discussion started by: haroonec
2 Replies

3. Shell Programming and Scripting

unix shell scripting

i have a file blah_blah_blah_yyyymmdd.dat . I need to write a shell script to extract this date and compare it with the current date. How do i do this?? Can someone provide me with the code please? (1 Reply)
Discussion started by: swarna bhat
1 Replies

4. Shell Programming and Scripting

Unix Shell Scripting

I 'm new to unix shell scripting can some one guide me to any e-book or link from where i can learn unix shell scripting .. i want to learn create interactive scripts for my day to day solaris work. Any help would be appreciated (1 Reply)
Discussion started by: fugitive
1 Replies

5. Shell Programming and Scripting

$# in unix shell scripting what does it mean

Hi can some one let me know what is the meaning of $# $@ $_ @_ in unix shell scripting. I really appreciate your time for replying to my post. Thanks In Advance (4 Replies)
Discussion started by: einsteinBrain
4 Replies

6. Shell Programming and Scripting

Unix shell scripting

Hi All, I have one file called date1.txt and it contains dates like 130112 140112 150112 160112 170112 180112 190112 201012 so i need a script to read this file line by line and find out the day of each date and assign this value in one variable. And validate Weekday="Mon" then... (4 Replies)
Discussion started by: vichuelaa
4 Replies

7. UNIX for Dummies Questions & Answers

Unix Shell Scripting( Calling from Unix to PLSQL)

Hello Experts, I have the following questions to be discussed here at this esteemed discussion forum. I have two Excel sheets which contain Unix Commands llike creating directory the structure/ftp/Copy/Zip etc to basically create an environment. I need help in understanding some of... (1 Reply)
Discussion started by: faizsaadq
1 Replies

8. UNIX for Dummies Questions & Answers

Unix Shell Scripting

I'm sorry if this doesn't go here, but I'm in depserate need of help with my last unix homework. Anyways, I'm taking summer classes, and one of them is UNIX. I've understood everything thus far, but I'm having a killer time with how my instructor has worded the problems for shell scripting. I... (3 Replies)
Discussion started by: dw15
3 Replies

9. Shell Programming and Scripting

Parallel processing in AIX (bash shell scripting)

Hi all, I am struggling to build a utility which can do the parallel processing. I achieved same in Linux using xargs -P but same is not working on AIX. I am building file copy utility where I will have all required info in a file (like souce file info and target location details), now i need... (2 Replies)
Discussion started by: ankur singh
2 Replies
nfsiod(8)						      System Manager's Manual							 nfsiod(8)

NAME
nfsiod, biod - The local NFS compatible asynchronous I/O daemon SYNOPSIS
nfsiod [ numthreads ] DESCRIPTION
The nfsiod daemon runs on an NFS compatible client machine and spawns several IO threads to service asynchronous I/O requests to its server. The I/O threads improve performance of both NFS reads and writes. Both try to enlist the aid of an idle I/O thread. If none is available, the process itself issues the request to the server and waits for the reply. The optimum number of I/O threads to run depends on many variables, such as how quickly the client will be writing, how many files will be accessed simultaneously, and the behaviour of the NFS server. For use with a Tru64 UNIX server, 7 is a good number of I/O threads for most systems. When reading, if the client believes the process is reading a file sequentially, it requests an I/O thread to read a block ahead of what the process is currently requesting. If the readahead completes before the process asks for that block, then the subsequent read system call for that data completes immediately and does not have to wait for the NFS request to complete. Read ahead will be triggered again so the read may find that next block available as well. When writing a file, the client takes the process's data, passes the request to an I/O thread and immediately returns to the process. If the process is writing data faster than the network or server can process, then eventually all the I/O threads become busy and the process has to handle a NFS write itself. This means the process has to wait until the server finishes the write. For Tru64 UNIX servers, the NFS block size is 8Kb and UFS tries to cluster I/O 64Kbs at a time. If the client is running with 7 I/O threads, 8 write requests can be in progress at once. This allows the client and server to write data 64Kbs at a time and is the reason for recommending 7 I/O threads. Unlike nfsd, each client thread can use either UDP or TCP. However, if TCP mounts are active, the nfsiod process will time out, close idle TCP connections, and acknowledge any connections closed by the server. The nfsiod process is also responsible for syncing the access time and modify times for special files and named pipes (fifos). Because I/O to these files does not go through the NFS server, NFS clients have to directly update the access time and modify time attributes. The client threads are implemented as kernel threads; they are part of Process ID 0, not the nfsiod process. The ps axml command displays idle I/O threads under PID 0. Idle threads will be waiting on nfsiod_wait. Therefore, if 7 I/O threads are configured, only 1 nfsiod process is displayed in the output from the ps command, although 7 client threads are available to handle NFS requests. FILES
Specifies the command path Specifies the file for logging NFS activity. RELATED INFORMATION
Commands: nfsd(8), nfsstat(8) Daemons: async_daemon(2) delim off nfsiod(8)
All times are GMT -4. The time now is 09:25 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy