Sponsored Content
Top Forums Shell Programming and Scripting How to cut some data from big file Post 302344187 by durden_tyler on Friday 14th of August 2009 11:07:25 PM
Old 08-15-2009
Well, a 30 GB file is a *HUGE* file and any shell command you run on it will take time for processing.

Maybe you want to split up the file and then work on the smaller components ? Run the command:

Code:
man split

to see what your options are.

tyler_durden
 

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to read data from a file (cut/awk)

Hi list, i have an orcale spool file with SQL> select COMPMAP as SVC, ITEM, UNIT, sum(JAN), sum(FEB) SVC ITE U SUM(JAN) SUM(FEB) ------ --- - ---------- ---------- 401500 IOC Q 14 14 406200 LC Q 1 1 410124 IOC Q 5 4 410124 LC... (1 Reply)
Discussion started by: rejirajraghav
1 Replies

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held... (8 Replies)
Discussion started by: dlam
8 Replies

3. UNIX for Dummies Questions & Answers

How to cut data block from .txt file in shell scripting

Hi All, Currently i have to write a script. For which i need to cut a block from .txt file. I know the specific word that starts the block and ends the block. Can we do it in shell scripting..? Please suggest.... (6 Replies)
Discussion started by: pank29
6 Replies

4. Shell Programming and Scripting

Cut big text file into 2

I have a big text file. I want to cut it into 2 pieces at known point or I know the pattern of the contents from where it can separate the files. Is there any quick command/solution? (4 Replies)
Discussion started by: sandy221
4 Replies

5. Shell Programming and Scripting

Sort a big data file

Hello, I have a big data file (160 MB) full of records with pipe(|) delimited those fields. I`m sorting the file on the first field. I'm trying to sort with "sort" command and it brings me 6 minutes. I have tried with some transformation methods in perl but it results "Out of memory". I was... (2 Replies)
Discussion started by: rubber08
2 Replies

6. Shell Programming and Scripting

parsing data from a big file using keys from another smaller file

Hi, I have 2 files format of file 1 is: a1 b2 a2 c2 d1 f3 format of file 2 is (tab delimited): a1 1.2 0.5 0.06 0.7 0.9 1 0.023 a3 0.91 0.007 0.12 0.34 0.45 1 0.7 a2 1.05 2.3 0.25 1 0.9 0.3 0.091 b1 1 5.4 0.3 9.2 0.3 0.2 0.1 b2 3 5 7 0.9 1 9 0 1 b3 0.001 1 2.3 4.6 8.9 10 0 1 0... (10 Replies)
Discussion started by: Lucky Ali
10 Replies

7. Shell Programming and Scripting

Cut and paste data in new file

HI Guys, I have file A: Abc XyZ Abc Xyz Kal Kaloo Abc XyZ Abc Xyz Kalpooo Abc XyZ Abc Xyz Kloo Abc Abc Klooo I want file B Abc XyZ Abc Xyz Kal Kaloo Abc XyZ Abc Xyz Kalpooo Abc XyZ Abc Xyz Kloo File A is now 1 lines Abc Abc Klooo Cut all lines which have xyz... (2 Replies)
Discussion started by: asavaliya
2 Replies

8. Shell Programming and Scripting

Need to cut a some required data from file

Data_Consolidation_Engine_Part_2_Job2..TgtArBkt: ORA-00942: table or view does not exist I have some thing like above in the file.. Upto this portion Data_Consolidation_Engine_Part_2_Job2..TgtArBkt: the length can be vary .. Can some one help me in taking this portion alone ORA-00942:... (7 Replies)
Discussion started by: saj
7 Replies

9. UNIX for Dummies Questions & Answers

How to cut a big file into small ones?

Hello all, Currently I have a txt file named as a.txt with the content as: f e100 aa bb cc dd ee ff f e222 aa dd ff gg f e987 dd aa f e2222 gg ff gg aa dd ff ee ee While, for some reason I want to cut a.txt into small ones, e.g. f1.txt, f2.txt, f3.txt and f4.txt. The routine is to... (6 Replies)
Discussion started by: locohd
6 Replies
VORBISCOMMENT(1)						   Vorbis Tools 						  VORBISCOMMENT(1)

NAME
vorbiscomment - edits Ogg Vorbis comments SYNOPSIS
vorbiscomment [-l] file.ogg vorbiscomment -a [ -t "tag=value" ] in.ogg out.ogg vorbiscomment -w in.ogg out.ogg DESCRIPTION
vorbiscomment reads, modifies, and appends Ogg Vorbis audio file metadata tags. OPTIONS
-a Append comments. -c commentfile Take comments from a file. The file is the same format as is output by the the -l option: one element per line in 'tag=value' for- mat. -h Show command help. -l List the comments in the ogg vorbis file. -q Quiet mode. No messages are displayed. -t 'tag=value' Specify a new tag on the command line. Each tag is given as a single string. The part before the '=' is treated as the tag name and the part after as the value. -w Replace comments with the new set given either on the command line with -t or from a file with -c. EXAMPLES
To just see what comment tags are in a file: vorbiscomment -l file.ogg To edit those comments: vorbiscomment -l file.ogg > file.txt [edit the comments in file.txt to your statisfaction] vorbiscomment -w -c file.txt file.ogg newfile.ogg To simply add a comment: vorbiscomment -a -t 'ARTIST=No One You Know' file.ogg newfile.ogg SEE ALSO
See http://xiph.org/ogg/vorbis/doc/v-comment.html for documentation on the Ogg Vorbis tag format, including a suggested list of canonical tag names. AUTHORS
Program Authors: Michael Smith <msmith@labyrinth.net.au> Ralph Giles <giles@xiph.org> Manpage Author: Christopher L Cheney <ccheney@debian.org> Xiph.org Foundation December 24, 2001 VORBISCOMMENT(1)
All times are GMT -4. The time now is 01:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy