Sponsored Content
Top Forums Shell Programming and Scripting How to cut some data from big file Post 302344221 by jp2542a on Saturday 15th of August 2009 07:01:07 AM
Old 08-15-2009
Any method that works on a variable length record size requires scanning the data to find a record. This will always incur a cpu penalty. But if your records are of a fixed sized, then we will be able to calculate the offset to the beginning and end of the section of interest and use more efficient ways of copying the data that remove the cpu intensive bits of the operation..
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to read data from a file (cut/awk)

Hi list, i have an orcale spool file with SQL> select COMPMAP as SVC, ITEM, UNIT, sum(JAN), sum(FEB) SVC ITE U SUM(JAN) SUM(FEB) ------ --- - ---------- ---------- 401500 IOC Q 14 14 406200 LC Q 1 1 410124 IOC Q 5 4 410124 LC... (1 Reply)
Discussion started by: rejirajraghav
1 Replies

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held... (8 Replies)
Discussion started by: dlam
8 Replies

3. UNIX for Dummies Questions & Answers

How to cut data block from .txt file in shell scripting

Hi All, Currently i have to write a script. For which i need to cut a block from .txt file. I know the specific word that starts the block and ends the block. Can we do it in shell scripting..? Please suggest.... (6 Replies)
Discussion started by: pank29
6 Replies

4. Shell Programming and Scripting

Cut big text file into 2

I have a big text file. I want to cut it into 2 pieces at known point or I know the pattern of the contents from where it can separate the files. Is there any quick command/solution? (4 Replies)
Discussion started by: sandy221
4 Replies

5. Shell Programming and Scripting

Sort a big data file

Hello, I have a big data file (160 MB) full of records with pipe(|) delimited those fields. I`m sorting the file on the first field. I'm trying to sort with "sort" command and it brings me 6 minutes. I have tried with some transformation methods in perl but it results "Out of memory". I was... (2 Replies)
Discussion started by: rubber08
2 Replies

6. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

7. Shell Programming and Scripting

Cut and paste data in new file

HI Guys, I have file A: Abc XyZ Abc Xyz Kal Kaloo Abc XyZ Abc Xyz Kalpooo Abc XyZ Abc Xyz Kloo Abc Abc Klooo I want file B Abc XyZ Abc Xyz Kal Kaloo Abc XyZ Abc Xyz Kalpooo Abc XyZ Abc Xyz Kloo File A is now 1 lines Abc Abc Klooo Cut all lines which have xyz... (2 Replies)
Discussion started by: asavaliya
2 Replies

8. Shell Programming and Scripting

Need to cut a some required data from file

Data_Consolidation_Engine_Part_2_Job2..TgtArBkt: ORA-00942: table or view does not exist I have some thing like above in the file.. Upto this portion Data_Consolidation_Engine_Part_2_Job2..TgtArBkt: the length can be vary .. Can some one help me in taking this portion alone ORA-00942:... (7 Replies)
Discussion started by: saj
7 Replies

9. UNIX for Dummies Questions & Answers

How to cut a big file into small ones?

Hello all, Currently I have a txt file named as a.txt with the content as: f e100 aa bb cc dd ee ff f e222 aa dd ff gg f e987 dd aa f e2222 gg ff gg aa dd ff ee ee While, for some reason I want to cut a.txt into small ones, e.g. f1.txt, f2.txt, f3.txt and f4.txt. The routine is to... (6 Replies)
Discussion started by: locohd
6 Replies
diffmk(1)							   User Commands							 diffmk(1)

NAME
diffmk - mark differences between versions of a troff input file SYNOPSIS
diffmk oldfile newfile markedfile DESCRIPTION
diffmk compares two versions of a file and creates a third version that includes "change mark" (.mc) commands for nroff(1) and troff(1). oldfile and newfile are the old and new versions of the file. diffmk generates markedfile, which, contains the text from newfile with troff(1) "change mark" requests (.mc) inserted where newfile differs from oldfile. When markedfile is formatted, changed or inserted text is shown by | at the right margin of each line. The position of deleted text is shown by a single *. USAGE
See largefile(5) for the description of the behavior of diffmk when encountering files greater than or equal to 2 Gbyte ( 2^31 bytes). EXAMPLES
Example 1 An example of the diffmk command. diffmk can also be used in conjunction with the proper troff requests to produce program listings with marked changes. In the following command line: example% diffmk old.c new.c marked.c ; nroff reqs marked.c | pr the file reqs contains the following troff requests: .pl 1 .ll 77 .nf .eo .nh which eliminate page breaks, adjust the line length, set no-fill mode, ignore escape characters, and turn off hyphenation, respectively. If the characters | and * are inappropriate, you might run markedfile through sed(1) to globally change them. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWdoc | +-----------------------------+-----------------------------+ SEE ALSO
diff(1), nroff(1), sed(1), troff(1), attributes(5), largefile(5) BUGS
Aesthetic considerations may dictate manual adjustment of some output. File differences involving only formatting requests may produce undesirable output, that is, replacing .sp by .sp 2 will produce a "change mark" on the preceding or following line of output. SunOS 5.11 14 Sep 1992 diffmk(1)
All times are GMT -4. The time now is 01:17 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy