Need help for faster file read and grep in big files Post: 303018517

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to grep faster ?

Hi I have to grep for 2000 strings in a file one after the other.Say the file name is Snxx.out which has these strings. I have to search for all the strings in the file Snxx.out one after the other. What is the fastest way to do it ?? Note:The current grep process is taking lot of time per...

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Morning guys. Another day another question. :rolleyes: I am knocking up a script to pull some data from a file. The problem is the file is very big (up to 1 gig in size), so this solution: for results in `grep "^\ ... works, but takes ages (we're talking minutes) to run. The data is held...

3. UNIX for Advanced & Expert Users

Split a big file into two others files

Hello, i have a very big file that has more then 80 MBytes (100MBytes). So with my CVS Application I cannot commit this file (too Big) because it must have < 80 MBytes. How can I split this file into two others files, i think the AIX Unix command : split -b can do that, buit how is the right...

4. UNIX for Dummies Questions & Answers

Faster than nested while read loops?

Hi experts, I just want to know if there is a better solution to my nested while read loops below: while read line; do while read line2; do while read line3; do echo "$line $line2 $line3" done < file3.txt done < file2.txt done < file1.txt >...

5. Shell Programming and Scripting

Read a file with n records as one big string using linux

Hello! Is there a way i can read a file with n records as one big string using linux shell script? I have a file in the below format - REC1 REC2 REC3 . . . REC4 Record length is 3000 bytes per record and with a newline char at the end. What i need to do is - read this file as one...

6. Shell Programming and Scripting

Grep -v -f and sort|diff which way is faster

Hi Gurus, I have two big files. I need to compare the different. currently, I am using sort file1 > file1_temp; sort file2 > file2_tmp diff file1_tmp file2_tmp I can use command grep -v -f file1 file2 just wondering which way is fast to compare two big files. Thanks...

7. UNIX for Dummies Questions & Answers

What is the faster way to grep from huge file?

Hi All, I am new to this forum and this is my first post. My requirement is like to optimize the time taken to grep the file with 40000 lines. There are two files FILEA(40000 lines) FILEB(40000 lines). The requirement is like this, both the file will be in the format below...

8. Shell Programming and Scripting

A faster way to read and search

I have a simple script that reads in data from fileA.txt and searches line by line for that data in multiple files (*multfiles.txt). It only prints the data when there is more than 1 instance of it. The problem is that its really slow (3+ hours) to complete the entire process. There are nearly 1500...

9. UNIX for Beginners Questions & Answers

Grep -f for big files

ok guys. this isnt homework or anything. i have been using grep -f all my life but i am trying this for a huge file and it doesnt work. can someone give me a replacement for grep -f pattern file for big files? thanks

10. Solaris

Split a big file system to several files

Gents Actually I have question and i need your support. I have this NAS file system mounted as /coresys has size of 7 TB I need to Split this file system into several file systems as mount points I mean how to can I Split it professionally to different NAS mount points how to can I decide...

LEARN ABOUT DEBIAN

avimerge

avimerge(1)						      General Commands Manual						       avimerge(1)

NAME

       avimerge - merge several AVI-files into one

SYNOPSIS

       avimerge -o name -i file1 [ file2 [ ... ] ] [ -p file ] [ -a num ] [ -A num ] [ -b num ] [ -c ] [ -f commentfile ] [ -x indexfile ]

COPYRIGHT

       avimerge is Copyright (C) by Thomas Oestreich.

DESCRIPTION

       avimerge is a  versatile tool.  It can contatenate several AVI files into one. It can also be used to fix an index of a broken file and can
       also replace audio tracks or muxes new ones. It can read raw AC3 and MP3 files for multplexing.

OPTIONS

       -o name
	      Specify the name of the output file.

       -i file
	      Specify the name(s) of the input file(s) to merge into the output file.

       -p file
	      Specify the name of the audio file to multiplex into the output file. The type of file can be either another AVI file or an MP3 or
	      AC3 file.

       -b num Specify if avimerge should write an VBR mp3 header into the AVI file. Default is dependent on the input file (and usually correct).
	      num is either 1 or 0.

       -c     Drop video frames in case audio is missing [off]
	      Only when merging multiple AVI files. Some AVI files run a little bit (usually for one or two video frames) short on audio. This
	      means avimerge cannot keep up sync when concatinating them. The files play fine when played individually but not when merged because
	      audio from the new file gets played back with video from the old file.  avimerge will print a message like

		No audiodata left for track 0->0 (59950.25=59950.25) continuing ..

	      When you turn on the -c option, the video which is too much will be dropped.

       -f commentfile
	      Read AVI tombstone data for header comments from commentfile. See /docs/avi_comments.txt for a sample.

       -x indexfile
	      Read the AVI index from indexfile.  See aviindex(1) for information on how to create such a file.

       -a num Specify the number of the audio track you want to use from the input file.

       -A num Specify the number of the audio track you want to use in the output file. If you specify an existing track number, the track will be
	      replaced. If omitted, the next free slot will be used.

EXAMPLES

       The command

       avimerge -o big.avi -i my_file1.avi my_file2.avi my_file3.avi

       merges the three input files my_file[123].avi into one big AVI-file big.avi.

       avimerge -o out.avi -i in.avi -p audio2.avi -a 1

       merges track number 1 form in.avi to the next free track number in out.avi. You can create audio-only AVI-files using

       transcode -i song.mp3 -x null,mp3 -g 0x0 -y raw -a 1 -o audio2.avi -u 50

       The command

       avimerge -o out.avi -i in.avi -p sound.mp3

       merges the file sound.mp3 as an additional audio track into out.avi.

AUTHORS

       avimerge was written by Thomas Oestreich
       <ostreich@theorie.physik.uni-goettingen.de> with contributions from many others.  See AUTHORS for details.

SEE ALSO

       aviindex(1), avifix(1), avisplit(1), tccat(1), tcdecode(1), tcdemux(1), tcextract(1), tcprobe(1), tcscan(1), transcode(1)

avimerge(1)							 26th January 2004						       avimerge(1)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to grep faster ?

Discussion started by: preethgideon

2. Shell Programming and Scripting

Big data file - sed/grep/awk?

Discussion started by: dlam

3. UNIX for Advanced & Expert Users

Split a big file into two others files

Discussion started by: steiner

4. UNIX for Dummies Questions & Answers

Faster than nested while read loops?

Discussion started by: chstr_14