Sponsored Content
Top Forums Shell Programming and Scripting deleting multiple records from a huge file at one time Post 302164641 by dsravan on Tuesday 5th of February 2008 11:39:44 AM
Old 02-05-2008
deleting multiple records from a huge file at one time

I have a very big file of 5gb size and there are about 50 million records in there. I have to delete the records based on recrord number that I know fromoutside with out opening the file. The record numbers are very random like 5000678, 7890005 etc.


Can somebody let me know how i can remove records based on the record number all at one time and not one time
a piece please?
 

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

Time taken for creation of a huge core file

Hi, I needed to know how I can find out the time needed for an Unix machine(HP) to create a corefile as huge as 500MB(core created either by a SEGV or a kill -6 command). An approximate figure of the time taken would be really helpful.:confused: (4 Replies)
Discussion started by: nayeem
4 Replies

2. Programming

problem deleting date-time stamped file in a directory

I have a number of files of the format filename.xfr_mmddyy_%H%M%S which i get in a specified directory daily. Now i want to search in the specified directory & delete the files which are more than 2 days old .So I use a command find $DIR/backup/* -ctime +2 -exec rm -f {} \; But after executing... (1 Reply)
Discussion started by: dharmesht
1 Replies

3. UNIX for Dummies Questions & Answers

deleting files based on file name and modified time

Hi, I have some log files created in the following fashion Ex: file name modified date 1) s.log1 01-jan-08 2) s.log2 02-jan-08 3) s.log3 03-jan-08 4) s.log4 04-jan-08 Now I want to have the latest 2 logs and delete the others. Can you tell me the one liner /... (1 Reply)
Discussion started by: ammu
1 Replies

4. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

5. Shell Programming and Scripting

Grep matched records from huge file

111111111100000000001111111111 123232323200000010001114545454 232435424200000000001232131212 342354234301000000002323423443 232435424200000000001232131212 2390898994200000000001238908092 This is the record format. From 11th position to 20th position in a record there are 0's occuring,and... (6 Replies)
Discussion started by: mjkreddy
6 Replies

6. Shell Programming and Scripting

Deleting last records of a file

can you please give shell script for daleting the last 7 records of file... (7 Replies)
Discussion started by: vsairam
7 Replies

7. Shell Programming and Scripting

splitting a huge line of file into multiple lines with fixed number of columns

Hi, I have a huge file with a single line. But I want to break that line into lines of with each line having five columns. My file is like this: code: "hi","there","how","are","you?","It","was","great","working","with","you.","hope","to","work","you." I want it like this: code:... (1 Reply)
Discussion started by: rajsharma
1 Replies

8. Shell Programming and Scripting

Deleting duplicate records from file 1 if records from file 2 match

I have 2 files "File 1" is delimited by ";" and "File 2" is delimited by "|". File 1 below (3 record shown): Doc1;03/01/2012;New York;6 Main Street;Mr. Smith 1;Mr. Jones Doc2;03/01/2012;Syracuse;876 Broadway;John Davis;Barbara Lull Doc3;03/01/2012;Buffalo;779 Old Windy Road;Charles... (2 Replies)
Discussion started by: vestport
2 Replies

9. Shell Programming and Scripting

Awk to Count Multiple patterns in a huge file

Hi, I have a file that is 430K lines long. It has records like below |site1|MAP |site2|MAP |site1|MODAL |site2|MAP |site2|MODAL |site2|LINK |site1|LINK My task is to count the number of time MAP, MODAL, LINK occurs for a single site and write new records like below to a new file ... (5 Replies)
Discussion started by: reach.sree@gmai
5 Replies

10. UNIX for Dummies Questions & Answers

Deleting records from .dat file

Hi, I need to delete one row from a .dat file. error processing column PCT_USED in row 1053295 for datafile /exp/stats/ts_stats.dat ORA-01722: invalid number This is used to load records using sql loader. Please let me know the procedure to do it. Regards, VN (3 Replies)
Discussion started by: narayanv
3 Replies
INTERFACE-ORDER(5)						    resolvconf							INTERFACE-ORDER(5)

NAME
interface-order - resolvconf configuration file DESCRIPTION
The file /etc/resolvconf/interface-order is used to control the order in which resolvconf nameserver information records are processed by those resolvconf update scripts that consult this file. (The name of the file is apt because a resolvconf nameserver information record is named after the interface with which it is associated.) The file contains a sequence of shell glob patterns, one per line. The position of a record in the order is the point at which its name first matches a pattern. Patterns may not contain whitespace, slashes or initial dots or tildes. Blank lines and lines beginning with a '#' are ignored. Resolvconf update scripts in /etc/resolvconf/update.d/ that consult this file include the current default versions of dnsmasq, pdnsd and libc. (Actually they don't read the file directly; they call the utility program /lib/resolvconf/list-records which lists records in the specified order and omits the names of empty records.) EXAMPLE
# /etc/resolvconf/interface-order # Use nameservers on the loopback interface first. lo* # Next use records for Ethernet interfaces eth* # Next use records for Wi-Fi interfaces wlan* # Next use records for PPP interfaces ppp* # Last use other interfaces * AUTHOR
Resolvconf was written by Thomas Hood <jdthood@gmail.com>. COPYRIGHT
Copyright (C) 2004, 2011 Thomas Hood This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICU- LAR PURPOSE. SEE ALSO
resolvconf(8) resolvconf 18 May 2011 INTERFACE-ORDER(5)
All times are GMT -4. The time now is 04:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy