UNIX for Advanced & Expert Users

Hi guys,
I have to filter out certain patterns from a log file (arerror.log, for those who know BMC Remedy). The problem is that the file is big, too big to be read properly. Filters aren't working properly on the file, and neither the entire contents are visible using more, or vi.
I FTP'd it to windows, and I was able to see 32768 pages of the log file, which constituted only one year of log.

Another funny thing is that, on unix I can see the logs of 2006(only), however on windows it's only for 2010. However I am sure that it contains logs from 2011 and 2012 as well.

The problem that I face is that any script I write won't work on the file as well. Any help on how to proceed would be highly appreciated.

Using:
Solaris 8
BMC Remedy 6.3 (JFYI)

Thanks a lot.

what pattern you need to filter out ?

did u try grep and nawk or perl to process the file ?

I tried grep, but it only processes the data that can be seen using the more pager, i.e. for the year 2006. If I grep it with "2010" or "2011", it doesn't show any records.

I didn't try perl or awk though.

what is mean by "only processes the data that can be seen using the more pager" ?

post the sample data and what command you tried ?

I mean the data that I can see by,

Code:

cat arerror.log | more"

It seems that only that much is being filtered by grep.

Sample data:
This was taken from the file I FTP'd to windows. As you can see the year specified is 2010.
If I grep "2010" arerror.log in unix, no result is fetched.

Thanks for replying.

can you try this.

Code:

 
nawk '/ 2010 /' arerror.log

Lets get some base facts:

Code:

#1How big is the file?
ls -lad arerror.log

#How many records in the file?
wc -l arerror.log

#Contents of first record:
head -1 arerror.log

#Contents of last record:
tail -1 arerror.log

Is your limit of 32768 lines in Windows because you are using Microsoft Excel?
What version of Windows?


Was the file open by Remedy on the unix system when you tried to read the file with "grep"?