Sponsored Content
Top Forums Shell Programming and Scripting deleting multiple records from a huge file at one time Post 302164954 by brainyoung on Wednesday 6th of February 2008 09:21:11 AM
Old 02-06-2008
Dsarvan,
Split & process,we process few GB of file through AWK.
1.split them based lines (approximate nr)
2.process the the files in parallel ( if you server is having decent RAM & CPU's)
remember not to have same name if you use any temporary file, one way could be adding a random number or adding process id to it.
 

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

Time taken for creation of a huge core file

Hi, I needed to know how I can find out the time needed for an Unix machine(HP) to create a corefile as huge as 500MB(core created either by a SEGV or a kill -6 command). An approximate figure of the time taken would be really helpful.:confused: (4 Replies)
Discussion started by: nayeem
4 Replies

2. Programming

problem deleting date-time stamped file in a directory

I have a number of files of the format filename.xfr_mmddyy_%H%M%S which i get in a specified directory daily. Now i want to search in the specified directory & delete the files which are more than 2 days old .So I use a command find $DIR/backup/* -ctime +2 -exec rm -f {} \; But after executing... (1 Reply)
Discussion started by: dharmesht
1 Replies

3. UNIX for Dummies Questions & Answers

deleting files based on file name and modified time

Hi, I have some log files created in the following fashion Ex: file name modified date 1) s.log1 01-jan-08 2) s.log2 02-jan-08 3) s.log3 03-jan-08 4) s.log4 04-jan-08 Now I want to have the latest 2 logs and delete the others. Can you tell me the one liner /... (1 Reply)
Discussion started by: ammu
1 Replies

4. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

5. Shell Programming and Scripting

Grep matched records from huge file

111111111100000000001111111111 123232323200000010001114545454 232435424200000000001232131212 342354234301000000002323423443 232435424200000000001232131212 2390898994200000000001238908092 This is the record format. From 11th position to 20th position in a record there are 0's occuring,and... (6 Replies)
Discussion started by: mjkreddy
6 Replies

6. Shell Programming and Scripting

Deleting last records of a file

can you please give shell script for daleting the last 7 records of file... (7 Replies)
Discussion started by: vsairam
7 Replies

7. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

8. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

9. Shell Programming and Scripting

Awk to Count Multiple patterns in a huge file

Hi, I have a file that is 430K lines long. It has records like below |site1|MAP |site2|MAP |site1|MODAL |site2|MAP |site2|MODAL |site2|LINK |site1|LINK My task is to count the number of time MAP, MODAL, LINK occurs for a single site and write new records like below to a new file ... (5 Replies)
Discussion started by: reach.sree@gmai
5 Replies

10. UNIX for Dummies Questions & Answers

Deleting records from .dat file

Hi, I need to delete one row from a .dat file. error processing column PCT_USED in row 1053295 for datafile /exp/stats/ts_stats.dat ORA-01722: invalid number This is used to load records using sql loader. Please let me know the procedure to do it. Regards, VN (3 Replies)
Discussion started by: narayanv
3 Replies
FORK(2) 							System Calls Manual							   FORK(2)

NAME
fork - spawn new process SYNOPSIS
fork( ) DESCRIPTION
Fork is the only way new processes are created. The new process's core image is a copy of that of the caller of fork. The only distinc- tion is the fact that the value returned in the old (parent) process contains the process ID of the new (child) process, while the value returned in the child is 0. Process ID's range from 1 to 30,000. This process ID is used by wait(2). Files open before the fork are shared, and have a common read-write pointer. In particular, this is the way that standard input and output files are passed and also how pipes are set up. SEE ALSO
wait(2), exec(2) DIAGNOSTICS
Returns -1 and fails to create a process if: there is inadequate swap space, the user is not super-user and has too many processes, or the system's process table is full. Only the super-user can take the last process-table slot. ASSEMBLER
(fork = 2.) sys fork (new process return) (old process return, new process ID in r0) The return locations in the old and new process differ by one word. The C-bit is set in the old process if a new process could not be cre- ated. FORK(2)
All times are GMT -4. The time now is 08:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy