Sponsored Content
Top Forums Shell Programming and Scripting Optimised way for search & replace a value on one line in a very huge file (File Size is 24 GB). Post 302552323 by yazu on Friday 2nd of September 2011 11:37:59 AM
Old 09-02-2011
Afaik, in the general case no. But if your new first line has the same number of bytes or shorter and you can pad it with spaces then I believe you can do it quick with low level programming - open for read/write, read some bytes (512 for example) in a buffer, change them, rewind, and write the buffer back.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

In Line File Modifications: Search and Replace

grep -il "TEST" ${ENVIRON}/*.pde| while read pde   do     cat $pde | sed s/"TEST 3,1"/"TEST 3,0"/g | sed s/"TEST  3,1"/"TEST  3,0"/g > ${pde}.tmp     if ; then       mv ${pde}.tmp $pde ... (2 Replies)
Discussion started by: Shakey21
2 Replies

2. UNIX for Dummies Questions & Answers

how can search a String in one text file and replace the whole line in another file

i am very new to UNIX plz help me in this scenario i have two text files as below file1.txt name=Rajakumar. Discipline=Electronics and communication. Designation=software Engineer. file2.txt name=Kannan. Discipline=Mechanical. Designation=CADD Design Engineer. ... (6 Replies)
Discussion started by: kkraja
6 Replies

3. UNIX for Dummies Questions & Answers

How to search and replace a particular line in file with sed command

Hello, I have a file and in that, I want to search for a aprticular word and then replace another word in the same line with something else. Example: In file abc.txt, there is a line <host oa_var="s_hostname">test</host> I want to search with s_hostname text and then replace test with... (2 Replies)
Discussion started by: sshah1001
2 Replies

4. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

Hi, I have a folder which contains multiple config.xml files and one input file, Please see the below format. Config Files format looks like :- Code: <application name="SAMPLE-ARCHIVE"> <NVPairs name="Global Variables"> <NameValuePair> ... (0 Replies)
Discussion started by: haiksuresh
0 Replies

5. Shell Programming and Scripting

Implement in one line sed or awk having no delimiter and file size is huge

I have file which contains around 5000 lines. The lines are fixed legth but having no delimiter.Each line line contains nearly 3000 characters. I want to delete the lines a> if it starts with 1 and if 576th postion is a digit i,e 0-9 or b> if it starts with 0 or 9(i,e header and footer) ... (4 Replies)
Discussion started by: millan
4 Replies

6. Shell Programming and Scripting

Global search and replace multi line file

Hello I need to search for a mult-line strngs(with spaces in between and qoted) in a file1 and replace that text with Fixed string globally in file1. The strng to search for is in file2. The file is big with some 20K records. so speed and effciency is required file1: (where srch & rplc... (0 Replies)
Discussion started by: Hiano
0 Replies

7. Shell Programming and Scripting

Mutli line pattern search & replace in a xml file

Hello guys, I need your help for a specific sed command that would search for a multi line pattern and if found, would replace it by another multi line pattern. For instance, here is the input: <RefNickName>abcd</RefNickName> <NickName>efgh</NickName> <Customize> ... (0 Replies)
Discussion started by: xciteddd
0 Replies

8. Shell Programming and Scripting

awk search/replace specific field, using variables for regexp & subsitution then overwrite file

Hello, I'm trying the solve the following problem. I have a file which I intend to use as a csv called master.csv The columns are separated by commas. I want to change the text on a specific row in either column 3,4,5 or 6 from xxx to yyy depending upon if column 1 matches a specified pattern.... (3 Replies)
Discussion started by: cyphex
3 Replies

9. Shell Programming and Scripting

Search & Replace in Multiple Files by reading a input file

I have a environment property file which contains: Input file: value1 = url1 value2 = url2 value3 = url3 and so on. I need to search all *.xml files under directory for value1 and replace it with url1. Same thing I have to do for all values mentioned in input file. I need script in unix bash... (7 Replies)
Discussion started by: Shamkamde
7 Replies

10. UNIX for Dummies Questions & Answers

Need to replace new line characters in a huge file

Hi , I would like to replace new line characters(\n) in a huge file of about 2 million records . I tried this one (:%s/\n//g) but it's hanging there and no result. Does this command do not work if the file is big. Please let me know if you have any other options Regards Raj (1 Reply)
Discussion started by: rajeevm
1 Replies
SG_TEST_RWBUF(8)						     SG3_UTILS							  SG_TEST_RWBUF(8)

NAME
sg_test_rwbuf - Tests the SCSI host adapter by issuing write and read operations on a device's buffer and calculating checksums. SYNOPSIS
sg_test_rwbuf [--addrd=AR] [--addwr=AW] [--help] [--quick] --size=SZ [--times=NUM] [--verbose] [--version] DEVICE or an older deprecated format sg_test_rwbuf DEVICE SZ [AW] [AR] DESCRIPTION
sg_test_rwbuf writes and reads back SZ bytes to the internal buffer of DEVICE (e.g. /dev/sda or /dev/sg0). A pseudo random pattern is writ- ten to the data buffer on the device then read back. If the same pattern is found 'Success' is reported. If they do not match (checksums unequal) then this is reported and up to 24 bytes from the first point of mismatch are reported; the first line shows what was written and the second line shows what was received. For testing purposes, you can ask it to write AW or read AR additional bytes. OPTIONS
Arguments to long options are mandatory for short options as well. -r, --addrd=AR Read an additional AR bytes (than indicated by SZ) from the data buffer. Checksum is performed over the first SZ bytes. -w, --addwr=AW Write an additional AW bytes (than indicated by SZ) of zeros into the data buffer. Checksum is generated over the first SZ bytes. -h, --help Print out a usage message the exit. -q, --quick Perform a READ BUFFER descriptor command to find out the available data buffer length and offset, print them out then exit (without testing with write/read sequences). -s, --size=SZ where SZ is the size of buffer in bytes to be written then read and checked. This number needs to be less than or equal to the size of the device's data buffer which can be seen from the --quick option. Either this option or the --quick option should be given. -t, --times=NUM where NUM is the number of times to repeat the write/read to buffer test. Default value is 1 . -v, --verbose increase verbosity of output. -V, --version print version number (and data of last change) then exit. NOTES
The microcode in a SCSI device is _not_ modified by doing a WRITE BUFFER command with its mode set to "data" (0x2) as done by this utility. Therefore this utility is safe in that respect. [Mode values 0x4, 0x5, 0x6 and 0x7 are the dangerous ones :-)] WARNING: If you access the device at the same time (e.g. because it's a hard disk with a mounted file system on it) the device's buffer may be used by the device itself for other data at the same time, and overwriting it may or may not cause data corruption! HOWEVER the SPC-3 draft standard does state in its WRITE BUFFER command: "This command shall not alter any medium of the logical unit when data mode ... is specified". This implies that it _is_ safe to use this utility with devices that have mounted file systems on them. Following this theme further, a disk with active mounted file systems may cause the data read back to be different (due to caching activity) to what was written and hence a checksum error. EXIT STATUS
The exit status of sg_test_rwbuf is 0 when it is successful. Otherwise see the sg3_utils(8) man page. AUTHORS
Written by D. Gilbert and K. Garloff COPYRIGHT
Copyright (C) 2000-2011 Douglas Gilbert, Kurt Garloff This software is distributed under the GPL version 2. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PUR- POSE. sg3_utils-1.31 February 2011 SG_TEST_RWBUF(8)
All times are GMT -4. The time now is 09:53 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy