Sponsored Content
Full Discussion: Edit a large file in place
Top Forums Shell Programming and Scripting Edit a large file in place Post 302134212 by mvijayv on Wednesday 29th of August 2007 08:08:10 PM
Old 08-29-2007
Edit a large file in place

SmilieFolks,
I have a file with 50 million records having 2 columns. I have to do the below:
1. Generate some random numbers of a fixed length.
2. Replace the second column of randomly chosen rows with the random numbers.

I tried using a little bit of perl to generate random numbers and sed to replace it manually. The problem I see is that it generates an output with the replaced record with all 50 million records. I'd rather not have the output generated for each row update. I'd like to get the output once all the updates are done ....
I was wondering if I could edit the file in place using sed ... I did try to look for this in-place option .. but I dont have the GNU version of SED ...

Any thoughts ...?

Thanks
V
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

how to edit large files using vi

How to edit large file using vi where you can't increase /usr/var/tmp anymore? (3 Replies)
Discussion started by: nazri
3 Replies

2. Shell Programming and Scripting

Help to edit a large file

I am trying to edit a file that has 33k+ records. In this file I need to edit each record that has a 'Y' in the 107th position and change the 10 fields before the 'Y' to blanks. Not all records have a 'Y' in the 107th field. ex: ... (8 Replies)
Discussion started by: jxh461
8 Replies

3. Shell Programming and Scripting

how to edit large file in unix

hi All, Plz let me know how to edit a file with 2000000 records. each record contains with 40 field seperated by |. i want modify 455487 record, but i am uable to edit this large file using vi editor in unix. plz let me know how to modify this file. Thanks in advance. -Bali Reddy (3 Replies)
Discussion started by: balireddy_77
3 Replies

4. Shell Programming and Scripting

Scripting the process to edit a large file

Hi, I need to make a script to edit a file. File is a large file in below format Version: 2008120101 ;$INCLUDE ./abc/xyz/Delhi ;$INCLUDE ./abc/xyz/London $INCLUDE ./abc/xyz/New York First line in the file is version number which is in year,month,date and serial number format. Each... (5 Replies)
Discussion started by: makkar4u
5 Replies

5. Shell Programming and Scripting

sed edit in place -i issues

Hello, I am attempting to create a command that I can eventually put into a loop so I can edit 1file on many servers. I would like to edit the file in place with sed -i. If not I will take any suggestions on how to use a temp file. I need to remove a email address from the configuration file... (4 Replies)
Discussion started by: abacus
4 Replies

6. Solaris

What is the best way to copy data from place to another place?

Dear Gurus, I need you to advice or suggestion about the best solution to copy data around 200-300G from serverA(location A) to serverB(location B). Normally, I will share folder and then copy but it takes too long time(about 2 days). Do you have any suggestion or which way should be... (9 Replies)
Discussion started by: unitipon
9 Replies

7. Shell Programming and Scripting

Read from file specific place in file using inode

Hello, I am using tcsh on AIX. I would like to write a script that does the following: 1. given an inode, how do I find exactly the name of the file? I know I could do this using ls -i | grep <inode> but it returns: <inode> <filename>. I need some string manipulation or something to... (1 Reply)
Discussion started by: lastZenMaster
1 Replies

8. Shell Programming and Scripting

How to get awk to edit in place and join all lines in text file

Hi, I lack the utter fundamentals on how to craft an awk script. I have hundreds of text files that were mangled by .doc format so all the lines are broken up so I need to join all of the lines of text into a single line. Normally I use vim command "ggVGJ" to join all lines but with so many... (3 Replies)
Discussion started by: n00ti
3 Replies

9. Shell Programming and Scripting

How to edit a large file

Whenever I am trying to edit a file in unix with vi editor, I am getting the following error: <data> :Tmp file too large Is there any way that I can edit the file other than vi. Any help is really appreciated. Thanks (10 Replies)
Discussion started by: bobby1015
10 Replies

10. Shell Programming and Scripting

Search for a file in all directories and place the file in that directory

Hi All, Daily I am getting the updated file. I have to search for this file in all directories and sub directories. If the file existed in a particular directory then move this updated file to that particular directory. If the file is not existed in any of the directories then place this... (4 Replies)
Discussion started by: ROCK_PLSQL
4 Replies
RANDOM(9)						   BSD Kernel Developer's Manual						 RANDOM(9)

NAME
arc4rand, arc4random, random, read_random, srandom -- supply pseudo-random numbers SYNOPSIS
#include <sys/libkern.h> void srandom(u_long seed); u_long random(void); void arc4rand(void *ptr, u_int length, int reseed); u_int32_t arc4random(void); #include <sys/random.h> int read_random(void *buffer, int count); DESCRIPTION
The random() function will by default produce a sequence of numbers that can be duplicated by calling srandom() with '1' as the seed. The srandom() function may be called with any arbitrary seed value to get slightly more unpredictable numbers. It is important to remember that the random() function is entirely predictable, and is therefore not of use where knowledge of the sequence of numbers may be of benefit to an attacker. The arc4rand() function will return very good quality random numbers, slightly better suited for security-related purposes. The random num- bers from arc4rand() are seeded from the entropy device if it is available. Automatic reseeds happen after a certain timeinterval and after a certain number of bytes have been delivered. A forced reseed can be forced by passing a non-zero value in the reseed argument. The read_random() function is used to return entropy directly from the entropy device if it has been loaded. If the entropy device is not loaded, then the buffer is filled with output generated by random(). The buffer is filled with no more than count bytes. It is advised that read_random() is not used; instead use arc4rand() All the bits generated by random(), arc4rand() and read_random() are usable. For example, 'random()&01' will produce a random binary value. The arc4random() is a convenience function which calls arc4rand() to return a 32 bit pseudo-random integer. RETURN VALUES
The random() function uses a non-linear additive feedback random number generator employing a default table of size 31 long integers to return successive pseudo-random numbers in the range from 0 to (2**31)-1. The period of this random number generator is very large, approxi- mately 16*((2**31)-1). The arc4rand() function uses the RC4 algorithm to generate successive pseudo-random bytes. The arc4random() function uses arc4rand() to gen- erate pseudo-random numbers in the range from 0 to (2**32)-1. The read_random() function returns the number of bytes placed in buffer. AUTHORS
Dan Moschuk wrote arc4random(). Mark R V Murray wrote read_random(). BSD
September 25, 2000 BSD
All times are GMT -4. The time now is 08:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy