02-06-2008
Dsarvan,
Split & process,we process few GB of file through AWK.
1.split them based lines (approximate nr)
2.process the the files in parallel ( if you server is having decent RAM & CPU's)
remember not to have same name if you use any temporary file, one way could be adding a random number or adding process id to it.
10 More Discussions You Might Find Interesting
1. Filesystems, Disks and Memory
Hi,
I needed to know how I can find out the time needed for an Unix machine(HP) to create a corefile as huge as 500MB(core created either by a SEGV or a kill -6 command). An approximate figure of the time taken would be really helpful.:confused: (4 Replies)
Discussion started by: nayeem
4 Replies
2. Programming
I have a number of files of the format filename.xfr_mmddyy_%H%M%S which i get in a specified directory daily. Now i want to search in the specified directory & delete the files which are more than 2 days old .So I use a command
find $DIR/backup/* -ctime +2 -exec rm -f {} \;
But after executing... (1 Reply)
Discussion started by: dharmesht
1 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I have some log files created in the following fashion
Ex:
file name modified date
1) s.log1 01-jan-08
2) s.log2 02-jan-08
3) s.log3 03-jan-08
4) s.log4 04-jan-08
Now I want to have the latest 2 logs and delete the others.
Can you tell me the one liner /... (1 Reply)
Discussion started by: ammu
1 Replies
4. UNIX for Dummies Questions & Answers
Hi,
I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record.
Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies
5. Shell Programming and Scripting
111111111100000000001111111111
123232323200000010001114545454
232435424200000000001232131212
342354234301000000002323423443
232435424200000000001232131212
2390898994200000000001238908092
This is the record format.
From 11th position to 20th position in a record there are 0's occuring,and... (6 Replies)
Discussion started by: mjkreddy
6 Replies
6. Shell Programming and Scripting
can you please give shell script for daleting the last 7 records of file... (7 Replies)
Discussion started by: vsairam
7 Replies
7. Shell Programming and Scripting
Hi,
I have a huge file with a single line.
But I want to break that line into lines of with each line having five columns.
My file is like this:
code:
"hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you."
I want it like this:
code:... (1 Reply)
Discussion started by: rajsharma
1 Replies
8. Shell Programming and Scripting
I have 2 files
"File 1" is delimited by ";" and "File 2" is delimited by "|".
File 1 below (3 record shown):
Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones
Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull
Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies
9. Shell Programming and Scripting
Hi,
I have a file that is 430K lines long. It has records like below
|site1|MAP
|site2|MAP
|site1|MODAL
|site2|MAP
|site2|MODAL
|site2|LINK
|site1|LINK
My task is to count the number of time MAP, MODAL, LINK occurs for a single site and write new records like below to a new file
... (5 Replies)
Discussion started by: reach.sree@gmai
5 Replies
10. UNIX for Dummies Questions & Answers
Hi,
I need to delete one row from a .dat file.
error processing column PCT_USED in row 1053295 for datafile /exp/stats/ts_stats.dat
ORA-01722: invalid number
This is used to load records using sql loader.
Please let me know the procedure to do it.
Regards,
VN (3 Replies)
Discussion started by: narayanv
3 Replies
FORK(2) System Calls Manual FORK(2)
NAME
fork - spawn new process
SYNOPSIS
fork( )
DESCRIPTION
Fork is the only way new processes are created. The new process's core image is a copy of that of the caller of fork. The only distinc-
tion is the fact that the value returned in the old (parent) process contains the process ID of the new (child) process, while the value
returned in the child is 0. Process ID's range from 1 to 30,000. This process ID is used by wait(2).
Files open before the fork are shared, and have a common read-write pointer. In particular, this is the way that standard input and output
files are passed and also how pipes are set up.
SEE ALSO
wait(2), exec(2)
DIAGNOSTICS
Returns -1 and fails to create a process if: there is inadequate swap space, the user is not super-user and has too many processes, or the
system's process table is full. Only the super-user can take the last process-table slot.
ASSEMBLER
(fork = 2.)
sys fork
(new process return)
(old process return, new process ID in r0)
The return locations in the old and new process differ by one word. The C-bit is set in the old process if a new process could not be cre-
ated.
FORK(2)