Sponsored Content
Top Forums Shell Programming and Scripting Advice on monitoring gziped files Post 302811853 by SkySmart on Friday 24th of May 2013 10:37:27 AM
Old 05-24-2013
Quote:
Originally Posted by Corona688
If you look inside gzcat on many systems, you'll find that it's a shell script containing this:

Code:
gunzip -cd "$@"

-c for "print to stdout", -d for "decompress".

so

Code:
( gunzip -cd /path/to/file.log.gz ; cat /path/to/logfile ) | grep ...

thank you so much!!! had no idea it could be done this way.

the only problem i have is. i dont know how big the gzip log file could be. so "cat"ing it can be disastrous if the gzip file is huge.

the only alternative is to write the uncompressed output to disk but that's a bad idea.


if there's absolutely no other solution, would the following be the most efficient way to go about what i'm trying to do?:

Code:
( gunzip -cd /path/to/file.log.gz ; cat /path/to/logfile | awk 'NR>100' ) | grep FAIL

also, i worry that the command could/would overwrite the new copy (dataccess.log) that was created when the data file was rotated.
 

10 More Discussions You Might Find Interesting

1. Programming

monitoring files copied onto hard disk

hi... i need pointers to books/website... 'm trytin to write a daemon that monitors files of particular type(eg. text or pdfs) copied onto the hard disk. the daemon should detect the above n write the file name (along with the absolute path) to a file. please DO NOT give me the code... (2 Replies)
Discussion started by: abhi_abhijith
2 Replies

2. Shell Programming and Scripting

Monitoring a directory for new files with .xx and executing command if found

Hi Guys. I am a complete shell scripting newbie with some syntax and commands understanding. I'm more of a win admin. With that said: I need to write a shell script to monitor a directory '/Mon_Dir' for new occurrences of files with .xx extension. Once a new file is detected in the directory, a... (4 Replies)
Discussion started by: krkan
4 Replies

3. UNIX for Dummies Questions & Answers

Monitoring & emailing log files

Hi ..first post ! I have a Unix v445 using solaris 10. I've trolled through various web pages but can't find exactly what I'm looking for. I have an alert log...or any messages file for that matter I need to check for certain key (error type) phrases - if I find them, they are redirected to... (11 Replies)
Discussion started by: davidra
11 Replies

4. Shell Programming and Scripting

advice on managing files between two systems

I am looking for some guidance to see if this is the most efficient solution. I have a piece of software that updates a directory every night at 2am. The directory contains xml files so at 2am any new files are written, any modified files are modified and any deleted files are removed from the... (1 Reply)
Discussion started by: yabai
1 Replies

5. UNIX for Dummies Questions & Answers

Monitoring specific files and folders

I want a mechanism to monitor a folder full of files that are sensitive. I want to log all accesses,modifications and changes to any file within the folder in a log file which should give me access/modify/change times,the user id of the process which tried and the pid. Even some idea of what to... (1 Reply)
Discussion started by: Vivek788
1 Replies

6. Solaris

Could not able to delete the files through find command need expert advice

Hi Gurus I am facing a problem, there is a folder called /a where there are lots of files which are occupying space anything between 30 GB to 100 GB as I am not able to check the space occupied by that folder through "du -sh /a" command as I don't see any output after more than 1 hour of... (4 Replies)
Discussion started by: amity
4 Replies

7. Shell Programming and Scripting

password for compressed/gziped

Dear All, I have script which generates the reports in csv format and compress/gzip these files. I need to put password for these compressed/gziped files and these can be opened on windows using same password. is it possible to do the same.... (1 Reply)
Discussion started by: arvindng
1 Replies

8. SCO

Need advice: Copying large CSV report files off SCO system

I have a SCO Unix server from 1999 running SCO 5.0.5 and some ancient accounting software called Real World A report writer program on the system is used to generate CSV files from accounting that we write with DOSCOPY commands to 3.5" floppies In the next 60 days we will be decommissioning... (11 Replies)
Discussion started by: magnetman
11 Replies

9. Red Hat

Advice regarding filesystems handling large number of files

Hi All, I have a CentOS operating system installed. I work with really huge number of files which are not only huge in number but some of them really huge in size. Minimum number of files could be 1 million to 2 million in one directory itself. Some of the files are even several Gigabytes in... (2 Replies)
Discussion started by: shoaibjameel123
2 Replies

10. UNIX for Dummies Questions & Answers

Realtime monitoring of files in a directory

Hello Experts, Here are my requirements. I want to monitor a directory and retrieve file properties like file names along with file sizes of files being generated today and that exceeds a threshold limit (say 10MB) using a shell script. Basically this shell script should monitor (kinda... (1 Reply)
Discussion started by: RamanM
1 Replies
CSPLIT(1)						    BSD General Commands Manual 						 CSPLIT(1)

NAME
csplit -- split files based on context SYNOPSIS
csplit [-ks] [-f prefix] [-n number] file args ... DESCRIPTION
The csplit utility splits file into pieces using the patterns args. If file is a dash ('-'), csplit reads from standard input. The options are as follows: -f prefix Give created files names beginning with prefix. The default is ``xx''. -k Do not remove output files if an error occurs or a HUP, INT or TERM signal is received. -n number Use number of decimal digits after the prefix to form the file name. The default is 2. -s Do not write the size of each output file to standard output as it is created. The args operands may be a combination of the following patterns: /regexp/[[+|-]offset] Create a file containing the input from the current line to (but not including) the next line matching the given basic regular expression. An optional offset from the line that matched may be specified. %regexp%[[+|-]offset] Same as above but a file is not created for the output. line_no Create containing the input from the current line to (but not including) the specified line number. {num} Repeat the previous pattern the specified number of times. If it follows a line number pattern, a new file will be created for each line_no lines, num times. The first line of the file is line number 1 for historic reasons. After all the patterns have been processed, the remaining input data (if there is any) will be written to a new file. Requesting to split at a line before the current line number or past the end of the file will result in an error. ENVIRONMENT
The LANG, LC_ALL, LC_COLLATE and LC_CTYPE environment variables affect the execution of csplit as described in environ(7). EXIT STATUS
The csplit utility exits 0 on success, and >0 if an error occurs. EXAMPLES
Split the mdoc(7) file foo.1 into one file for each section (up to 20): csplit -k foo.1 '%^.Sh%' '/^.Sh/' '{20}' Split standard input after the first 99 lines and every 100 lines thereafter: csplit -k - 100 '{19}' SEE ALSO
sed(1), split(1), re_format(7) STANDARDS
The csplit utility conforms to IEEE Std 1003.1-2001 (``POSIX.1''). HISTORY
A csplit command appeared in PWB UNIX. BUGS
Input lines are limited to LINE_MAX (2048) bytes in length. BSD
January 26, 2005 BSD
All times are GMT -4. The time now is 10:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy