Sponsored Content
Top Forums Shell Programming and Scripting How to quickly substitute pattern within certain range of a huge file? Post 303017421 by RudiC on Tuesday 15th of May 2018 04:55:55 PM
Old 05-15-2018
I'm afraid ANY approach will have to copy >300GB, even if only 18 lines are to be modified, which will take its time. Possible solutions might be using editors like ed (search these fora), or applying some dirty tricks like
Code:
sed '1,18s/,\{1,\}/\t/g; 19q' infile_big.sam > outfile_big.sam
dd if=infile_big.sam of=outfile_big.sam skip=$(stat -c%s outfile_big.sam) iflag=skip_bytes oflag=append conv=notrunc

Still, don't expect too much ...

Last edited by RudiC; 05-15-2018 at 06:13 PM.. Reason: Eliminated TMP file.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

need solution for this quickly. please quickly.

Write a nawk script that will produce the following report: ***FIRST QUARTERLY REPORT*** ***CAMPAIGN 2004 CONTRIBUTIONS*** ------------------------------------------------------------------------- NAME PHONE Jan | ... (5 Replies)
Discussion started by: p.palakj.shah
5 Replies

2. Shell Programming and Scripting

print range between two patterns if it contains a pattern within the range

I want to print between the range two patterns if a particular pattern is present in between the two patterns. I am new to Unix. Any help would be greatly appreciated. e.g. Pattern1 Bombay Calcutta Delhi Pattern2 Pattern1 Patna Madras Gwalior Delhi Pattern2 Pattern1... (2 Replies)
Discussion started by: joyan321
2 Replies

3. Shell Programming and Scripting

sed pattern range

Hi guys, trying to replace a '#' with a ' ' (space) but only between the brackets '(' and ')' N="text1#text2#text3(var1#var2#var3)" N=`echo $N |sed '/(/,/) s/#. //'` echo $N Looking for an output of "text1#text2#text3(var1 var2 var3)" Any ideas? (15 Replies)
Discussion started by: mikepegg
15 Replies

4. Shell Programming and Scripting

How to combine lines within range of pattern

I've a file say having line 1 line 2 (NP line 3 line 4 line 5) line 6 I want to combine lines starting from (NP and ending with ) then it will look like line 1 line 2 (NP line3 line4 line5) line 6 I tried using sed '/(NP/,/)$/ s/\n/ /' but it's not working. Any help please? ... (8 Replies)
Discussion started by: neg
8 Replies

5. Shell Programming and Scripting

Removing tmp file too quickly?

Still trying to get the basics down and I would like a different solution to what I'm currently doing and a better understanding of why it's happening. I've written a simple backup script that tars individual directories and then dumps them to a NFS drive. STDERR is being dumped into a process... (2 Replies)
Discussion started by: mandelbrot333
2 Replies

6. Shell Programming and Scripting

Pattern Matchin Huge File

Hi Experts, I've issue with the huge file. My requirement is I need to search a pattern between the 155-156 position and if its match's to 31 or 36 then need to route that to a new separate files. The main file has around 1459328 line and 2 GB in size. I tired with the below code which take... (9 Replies)
Discussion started by: senthil.ak
9 Replies

7. Shell Programming and Scripting

Print pattern range to a new file

Hi Everyone! I really appreciate all of your help, I'm learning so much, can't wait until I get good enough to start answering questions! I have a problem ... from one large file, I'd like to create multiple new files for each pattern block beginning with /^ISA/ ending with /^IEA/ ... (2 Replies)
Discussion started by: verge
2 Replies

8. UNIX for Dummies Questions & Answers

Split a huge 7 GB File Based on Pattern into 4 files

Hi, I have a Huge 7 GB file which has around 1 million records, i want to split this file into 4 files to contain around 250k messages each. Please help me as Split command cannot work here as it might miss tags.. Format of the file is as below <!--###### ###### START-->... (6 Replies)
Discussion started by: KishM
6 Replies

9. Shell Programming and Scripting

sed Range Pattern and 2 lines before Start Pattern

Hi all, I have been searching all over Google but I am unable to find a solution for a particular result that I am trying to achieve. Consider the following input: 1 2 3 4 5 B4Srt1--Variable-0000 B4Srt2--Variable-1111 Srt 6 7 8 9 10 End (3 Replies)
Discussion started by: y2jacky
3 Replies

10. Shell Programming and Scripting

Extract range from config file matching pattern

I have config file like this: server_name xx opt1 opt2 opt3 suboptions1 #suboptions - disabled suboptions2 pattern suboptions3 server_name yy opt1 opt2 opt3 suboptions1 pattern #suboptions - disabled suboptions2 So basically I want to extract the server... (1 Reply)
Discussion started by: nemesis911
1 Replies
cachefslog(1M)						  System Administration Commands					    cachefslog(1M)

NAME
cachefslog - Cache File System logging SYNOPSIS
cachefslog [-f logfile | -h] cachefs_mount_point DESCRIPTION
The cachefslog command displays where CacheFS statistics are being logged. Optionally, it sets where CacheFS statistics are being logged, or it halts logging for a cache specified by cachefs_mount_point. The cachefs_mount_point argument is a mount point of a cache file system. All file systems cached under the same cache as cachefs_mount_point will be logged. OPTIONS
The following options are supported. You must be super-user to use the -f and -h options. -f logfile Specify the log file to be used. -h Halt logging. OPERANDS
cachefs_mount_point A mount point of a cache file system. USAGE
See largefile(5) for the description of the behavior of cachefslog when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). EXAMPLES
Example 1: Checking the Logging of a directory. The example below checks if the directory /home/sam is being logged: example% cachefslog /home/sam not logged: /home/sam Example 2: Changing the logfile. The example below changes the logfile of /home/sam to /var/tmp/samlog: example# cachefslog -f /var/tmp/samlog /home/sam /var/tmp/samlog: /home/sam Example 3: Verifying the change of a logfile. The example below verifies the change of the previous example: example% cachefslog /home/sam /var/tmp/samlog: /home/sam Example 4: Halting the logging of a directory. The example below halts logging for the /home/sam directory: example# cachefslog -h /home/sam not logged: /home/sam EXIT STATUS
The following exit values are returned: 0 success non-zero an error has occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
cachefsstat(1M), cachefswssize(1M), cfsadmin(1M), attributes(5), largefile(5) DIAGNOSTICS
Invalid path It is illegal to specify a path within a cache file system. SunOS 5.10 7 Feb 1997 cachefslog(1M)
All times are GMT -4. The time now is 05:48 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy