virtual memory and diff'ing very large files


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers virtual memory and diff'ing very large files
# 1  
Old 02-13-2012
[deleted]

[deleted]

Last edited by uiop44; 02-13-2012 at 08:17 PM.. Reason: solved w/workaround
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Diff 3 files, but diff only their 2nd column

Guys i have 3 files, but i want to compare and diff only the 2nd column path=`/home/whois/doms` for i in `cat domain.tx` do whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

2. Shell Programming and Scripting

Severe performance issue while 'grep'ing on large volume of data

Background ------------- The Unix flavor can be any amongst Solaris, AIX, HP-UX and Linux. I have below 2 flat files. File-1 ------ Contains 50,000 rows with 2 fields in each row, separated by pipe. Row structure is like Object_Id|Object_Name, as following: 111|XXX 222|YYY 333|ZZZ ... (6 Replies)
Discussion started by: Souvik
6 Replies

3. UNIX for Advanced & Expert Users

Out of Memory error when free memory size is large

I was running a program and it stopped and showed "Out of Memory!". at that time, the RAM used by this process is around 4G and the free memory size of the machine is around 30G. Does anybody know what maybe the reason? this program is written with Perl. the OS of the machine is Solaris U8. And I... (1 Reply)
Discussion started by: lilili07
1 Replies

4. Shell Programming and Scripting

grep'ing and sed'ing chunks in bash... need help on speeding up a log parser.

I have a file that is 20 - 80+ MB in size that is a certain type of log file. It logs one of our processes and this process is multi-threaded. Therefore the log file is kind of a mess. Here's an example: The logfile looks like: "DATE TIME - THREAD ID - Details", and a new file is created... (4 Replies)
Discussion started by: elinenbe
4 Replies

5. UNIX for Dummies Questions & Answers

cpu, memory and virtual memory usage

Hi All, Does anyone know what the best commands in the UNIX command line are for obtaining this info: current CPU usage memory usage virtual memory usage preferably with date and time parameters too? thanks ocelot (4 Replies)
Discussion started by: ocelot
4 Replies

6. AIX

ulimits max locked memory virtual memory

Hi, Would any one be so kind to explain me : are ulimits defined for each user seperately ? When ? Specialy what is the impact of : max locked memory and virtual memory on performance of applications for a user. Many thanks. PS : this is what I can see in MAN : ulimit ] ... (5 Replies)
Discussion started by: big123456
5 Replies

7. HP-UX

Virtual Memory

Hi! I work with HP-UX and I have to monitorize the use of virtual memory for different processes. (java processes for Tibco Adapter) And if these processes exceed a limit send a message to the syslog. I donīt know how to monitorize this... Should I do a script? or use an aplication, for example... (3 Replies)
Discussion started by: Kurohana
3 Replies

8. Programming

about virtual memory and memory leak

Hi, First of all I appreciate this group very much for its informative discussions and posts. Here is my question. I have one process whose virtual memory size increases linearly from 6MB to 12MB in 20 minutes. Does that mean my process has memory leaks? In what cases does the... (4 Replies)
Discussion started by: shriashishpatil
4 Replies

9. Linux

Memory issue while diff !!!

Hi All Any idea? why am I having this memory issue? perforce@ixca-pforce-bak:/home/p4-demo-root$ diff checkpoint_offline ../p4-demo-root2/checkpoint.1150 diff: memory exhausted Thanks a lot C Saha (0 Replies)
Discussion started by: csaha
0 Replies

10. Shell Programming and Scripting

diff 2 files; output diff's to 3rd file

Hello, I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3. For example: file 1 123 1234 123456 file 2 123 2345 23456 file 3 should have 2345 23456 I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies
Login or Register to Ask a Question
bdiff(1)							   User Commands							  bdiff(1)

NAME
bdiff - big diff SYNOPSIS
bdiff filename1 filename2 [n] [-s] DESCRIPTION
bdiff is used in a manner analogous to diff to find which lines in filename1 and filename2 must be changed to bring the files into agree- ment. Its purpose is to allow processing of files too large for diff. If filename1 (filename2) is -, the standard input is read. bdiff ignores lines common to the beginning of both files, splits the remainder of each file into n-line segments, and invokes diff on cor- responding segments. If both optional arguments are specified, they must appear in the order indicated above. The output of bdiff is exactly that of diff, with line numbers adjusted to account for the segmenting of the files (that is, to make it look as if the files had been processed whole). Note: Because of the segmenting of the files, bdiff does not necessarily find a smallest sufficient set of file differences. OPTIONS
n The number of line segments. The value of n is 3500 by default. If the optional third argument is given and it is numeric, it is used as the value for n. This is useful in those cases in which 3500-line segments are too large for diff, causing it to fail. -s Specifies that no diagnostics are to be printed by bdiff (silent option). Note: However, this does not suppress possible diagnos- tic messages from diff, which bdiff calls. USAGE
See largefile(5) for the description of the behavior of bdiff when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes). FILES
/tmp/bd????? ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |enabled | +-----------------------------+-----------------------------+ SEE ALSO
diff(1), attributes(5), largefile(5) DIAGNOSTICS
Use help for explanations. SunOS 5.10 14 Sep 1992 bdiff(1)