Sponsored Content
Top Forums Shell Programming and Scripting Loop and grepping into a file Post 26982 by skotapal on Monday 26th of August 2002 06:04:37 PM
Old 08-26-2002
Question Loop and grepping into a file

I wrote this script for:
1. Get the Web log for today
2. Give me a list of all the IP addresses that have accessed the web server today
3. Remove a list of known IPs listed in a file (line by line)
4. Mail the final file to selected recipients.
I am unable to do part 3. In the script pasted below, I can get only the last IP address from the list removed. It is overwriting the file ipaccess after grepping for each on the known IPs. When I use the >> option it appends to the file. But I want a consolidated list.

Can u gimme some idea how to finish it up?

Srini

#! /bin/sh

IP_LIST=/home/ksrinivas/knownips
SOURCE_DIR=/disk2/MUSTANG/
currentDate=`date +%d"/"%b"/"%Y`

awk ' {print $1" "$4" "$7}' /etc/httpd/logs/access_log | sed /exe/D | grep "$currentDate" > $SOURCE_DIR/access.txt

awk ' {print $1}' $SOURCE_DIR/access.txt > $SOURCE_DIR/ip-add

sort -u $SOURCE_DIR/ip-add > $SOURCE_DIR/ips

cat $IP_LIST | while read IP_ADD
do
grep -v $IP_ADD $SOURCE_DIR/ips >> $SOURCE_DIR/ipaccess
done

uuencode $SOURCE_DIR/ipaccess | mail -s "IPs Accessing mywebserver.com today" ksrinivas
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

grepping the first 3 characters from a file

Hi I was wondering if it's possible to use a command to get the first 3 characters of a line in a text file, I tried grep but it returns the whole line but I am only interested in the first 3 characters. Is this possible with grep or I need any other command? Also is it possible deleting from... (2 Replies)
Discussion started by: g-e-n-o
2 Replies

2. UNIX for Dummies Questions & Answers

grepping the first 3 characters from a file

give this a try and let me know if it works grep '^' filename rachael (2 Replies)
Discussion started by: rachael
2 Replies

3. Shell Programming and Scripting

Grepping Errors in a file

Hey All, I have to grep for an error from a file and get the results of errror in a different file...... But there should be no duplicate entries. Can anyone help me in giving a shell script for this This is file which contains pattern error which I am supposed to grep and put this in a... (4 Replies)
Discussion started by: achararun
4 Replies

4. Shell Programming and Scripting

Grepping log file

Dear All, I have a log file that is dislpayed as: <msg time='2009-10-14T05:46:42.580+00:00' org_id='oracle' comp_id='tnslsnr' type='UNKNOWN' level='16' host_id='mtdb_a' host_addr='UNKNOWN' version='1'> <txt>14-OCT-2009 05:46:42 *... (19 Replies)
Discussion started by: x-plicit78
19 Replies

5. Shell Programming and Scripting

Grepping string from out file

Guys .. Need to pull this highlighted strings irrespective of line numbers & should be echoed . But these strings are from Outfile from different dir. In which way this can be grepped ?? Need an idea http-timeout 120 seconds persistent-timeout 180 seconds host-rewriting on ... (7 Replies)
Discussion started by: raghunsi
7 Replies

6. UNIX for Advanced & Expert Users

Insert a line grepping a file

Hi I have a file name file1.txt & file2.txt which looks like below file1.txt Name=alan Name=math Name=don file2.txt afdsfsdf asdfasdfsd sdfsd dsfas Now I have to grep for all the names and insert a line in file2.txt for each name after line3 as follows file2.txt (2 Replies)
Discussion started by: ammu
2 Replies

7. Shell Programming and Scripting

Grepping file and returning passed variable if the value does not exist in file at all.

I have a list of fields that I want to check a file for, returning that field if it not found at all in the file. Is there a way to do a grep -lc and return the passed variable too rather then just the count? I am doing some crappy work-around now but I was not sure how to regrep this for :0 so... (3 Replies)
Discussion started by: personalt
3 Replies

8. Shell Programming and Scripting

Display file date after grepping a string in the file

Hi All, I need to recursively grep several folders for a MAC address and display the results with the date of the file name at the start. Even better would be if the final results were displayed chronologically so the newest file is always at the end. Oldest at the top, regardless of what... (8 Replies)
Discussion started by: quemalr
8 Replies

9. Shell Programming and Scripting

Grepping text block by block by using for loop

Hei buddies, Need ur help once again. I have a file which has bunch of lines which starts from a fixed pattern and ends with another fixed pattern. I want to make use of these fixed starting and ending patterns to select the bunch, one at a time. The input file is as follows. Hi welcome... (12 Replies)
Discussion started by: anushree.a
12 Replies

10. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies
LAZYGAL(1)							   User commands							LAZYGAL(1)

NAME
lazygal - static web gallery generator SYNOPSIS
lazygal [-h | -v | [options] albumdir] DESCRIPTION
This manual page explains the lazygal program. This program is a static web gallery generator written in Python. lazygal works so: you should have an original store of files - possibly containg subdirectories (their names serving as headings if not using the album metadata feature). This is the source file hierarchy. It will never be modified by lazygal. Then, when launching: $ lazygal -o /var/www/MyAlbum /home/user/SourceDir lazygal will analyze the contents of the source hierarchy and will (re)create the target hierarchy, but with all the bells and whistles defined by the templates. Only missing parts or parts that are not up to date will be generated. There is a limitation to this mechanism though: although updates in the source directory, in the metadata or in the themes is detected, changes in command line options and configuration files since last generation are not and the user should manually delete files that need to be generated again. OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. For a complete description, see the -h switch. -v --version Show program's version number and exit. -h --help Show summary of options. --quiet Don't output anything except for errors. --debug Output everything that lazygal is doing. -o DEST_DIR --output-directory=DEST_DIR Directory where web pages, slides and thumbs will be written (default is current directory). -t THEME --theme=THEME Theme name (looked up in theme directory) or theme full path. --default-style=DEFAULT_STYLE Default style to apply to the theme. --template-vars=TPL_VARS Common variables to load all templates with, e.g. --template-vars='footer=foo bar,color=baz'. For longer variable contents, it is easier to use a configuration file (see lazygal.conf(5)). -f --force-gen-pages Force rebuild of web pages, regardless of the modification times of their dependencies. This is handy when changing a configuration option affecting these (theme, directory flattening, etc.). --clean-destination Clean destination directory of files that should not be there (default is to print a warning but not to delete). --check-all-dirs Exhaustively go through all directories regardless of source modification time. -s IMAGE_SIZE --image-size=IMAGE_SIZE Size of images, define as name=xxy, ..., eg. small=800x600,medium=1024x768. The special dimensions 0x0 use original size. Refer to the IMAGE RESIZE DESCRIPTION section for more information on the available syntax. -T THUMBNAIL_SIZE --thumbnail-size=THUMBNAIL_SIZE Size of thumbnails, eg. 150x113. Refer to the IMAGE RESIZE DESCRIPTION section for more information on the available syntax. -q QUALITY --quality=QUALITY Quality of generated JPEG images (default is 85). -O --original Include original photos in output. --orig-base=RELATIVE_PATH Do not copy original photos in output directory, instead link them using RELATIVE_PATH as base for those links (discarded without -O). --orig-symlink Do not copy original photos in output directory, instead create symlinks to their original locations. This is useful when you plan transferring the whole directory which lazygal generated to some other location, perhaps with rsync, and you wish to avoid creating an extra copy of each photo. Caution This option is not available on Windows; if you try to use it on that operating system, lazygal will immediately exit with an exit status of 1. --puburl=PUB_URL Publication URL (only useful for feed generation). -m --generate-metadata Generate metadata description files where they don't exist in the source tree instead of generating the web gallery. This disables all other options. -n THUMBS_PER_PAGE --thumbs-per-page=THUMBS_PER_PAGE Maximum number of thumbs per index page. This enables index pagination (0 is unlimited). --pic-sort-by=ORDER Sort order for images in a subgallery, among 'mtime', 'filename', or 'exif'. (default is 'exif' which is by EXIF date if EXIF data is available, filename otherwise, sorting EXIF-less images before). Add ':reverse' to reverse the sort order (e.g. --pic-sort-by=mtime:reverse). --subgal-sort-by=ORDER Sort order for subgalleries, among 'exif' (EXIF date of the latest picture in sub-gallery), 'mtime' or 'dirname' (default is 'dirname'). Add ':reverse' to reverse the sort order (e.g. --subgal-sort-by=dirname:reverse). --dir-flattening-depth=LEVEL Level below witch the directory tree is flattened. Default is no flattening ('No'). This option makes the program include the web gallery index of child galleries in their parent's gallery index, if their level is greater than the supplied LEVEL. The level of the album root is 0. Index pages with multiple galleries (which happens when this section is used) show the pictures links in gallery sections. The following examples show the produced indexes for a sample album (2 sub-galleries, 1 sub-sub-gallery, 1 picture in each one of those). Example 1. --dir-flattening-depth=No (default) index.html <- sub-gallery links subgal1/index.html <- index with img1 subgal1/img1.html subgal1/subsubgal1/index.html <- index with img2 subgal1/subsubgal1/img2.html subgal2/index.html <- index with img3 subgal2/img3.html Example 2. --dir-flattening-depth=0 index.html <- contains index for all pics subgal1/img1.html subgal1/subsubgal1/img2.html subgal2/img3.html Example 3. --dir-flattening-depth=1 index.html <- contains index for all pics subgal1/index.html <- index with img1 and img2 subgal1/img1.html subgal1/subsubgal1/img2.html subgal2/index.html <- index with img3 subgal2/img3.html -z --make-dir-zip Make a zip archive of original pictures for each directory. --webalbum-pic-bg=WEBALBUMPIC_BG Webalbum picture background color. Default is transparent, and implies the PNG format. Any other value, e.g. red, white, blue, uses JPEG. --webalbum-pic-type=WEBALBUMPIC_TYPE What type of web album thumbnails to generate. By default, lazygal generates the well-loved "messy" thumbnails with randomly selected pictures from the album each rotated by a random amount and pasted together. This default can also be forced by specifying 'messy' as WEBALBUMPIC_TYPE. On the other hand, specifying 'tidy' as the value of this option forces lazygal to skip the rotations, resulting in more regularly shaped thumbnails which can also be more densely packed. This can be an advantage if not all users of your albums have huge screens :-) --keep-gps-data Do not remove GPS data from EXIF tags. By default the location tags are removed for privacy reasons. However, there are situations when having the location data makes sense and is desired. This is mostly meant to be used with holiday photos. THEMES
A theme maps to a directory that contains the following items: theme/browse.thtml The XHTML template for the theme browse page (displaying one picture). theme/dirindex.thtml The XHTML template for the directory index page (pictures and sub-galleries links). Missing template files will be searched for in the default theme. This must be valid XML. See http://genshi.edgewall.org/wiki/Documentation/xml-templates.html for syntax. Dependencies for statically included templates (i.e. with filenames not computed from variables) are automatically computed: when an included template is modified, the software will automatically figure out which pages to re-generate. Each file which name starts with SHARED_ is copied to the web album destination under the directory shared and is aimed at being a shared resource accross your web album (CSS stylesheet, button, etc.). Each file which extension starts with t is processed through the Genshi text template engine before being copied. See http://genshi.edgewall.org/wiki/Documentation/text-templates.html for syntax. Please refer to the examples supplied in /usr/share/lazygal/themes ALBUM METADATA
If a directory from the source album contains a file named album_description, it is processed as a source of album metadata. The format is borrowed from another album generating tool - Matew. Each line is treated as one possible tag, unknown lines are simply ignored. Example content of this file follows: Example 4. album_description Album name "My album" Album description "Description, which can be very long." Album image identifier relative/path/to/image.jpg Otherwise, the user can provide metadata in the following files. SOURCE_DIR/album-name The title to use for this album directory. SOURCE_DIR/album-description The description for this album directory. HTML tags are used verbatim from this file. SOURCE_DIR/album-picture The image to use at the top of the album picture stack. SOURCE_DIR/PICTURE_FILENAME.comment The description to use for this particular image. Please note that HTML tags are taken as provided in this file for output in the templates. FILES
~/.lazygal User configuration directory. ~/.lazygal/themes User themes directory. CONFIGURATION FILES
Multiple configuration files are processed by lazygal. The configuration is initially set up with the defaults. The defaults can be found in the lazygal source distribution in lazygal/defaults.conf. Then, the configuration files are processed in the following order, each newly defined value overloading formerly defined values. Finally, any command-line-provided parameter takes precedence on any configuration file value. ~/.lazygal/config User configuration file. See lazygal.conf(5) for format. SOURCE_DIR/.lazygal Album root configuration file. See lazygal.conf(5) for format. SOURCE_DIR/gal/.lazygal Web gallery configuration file. Only the webgal and template-vars sections are red in these files. The configuration applies to the gallery representing the directory of the configuration file, and all of its sub-directories, unless another configuration file in a sub-directory overloads some of the defined configuration values. See lazygal.conf(5) for format. SIZE DESCRIPTION
The size string follows the same syntax as ImageMagick's. scale% Height and width both scaled by specified percentage. xscale%yscale% Height and width individually scaled by specified percentages. width Width given, height automatically selected to preserve aspect ratio. xheight Height given, width automatically selected to preserve aspect ratio. widthxheight Maximum values of height and width given, aspect ratio preserved. widthxheight^ Minimum values of width and height given, aspect ratio preserved. widthxheight! Width and height emphatically given, original aspect ratio ignored. widthxheight> Change as per the supplied dimensions but only if an image dimension exceeds a specified dimension. widthxheight< Change dimensions only if both image dimensions exceed specified dimensions. pixels@ Resize image to have specified area in pixels. Aspect ratio is preserved. SEE ALSO
lazygal.conf(5). More information is available on the program website: http://sousmonlit.dyndns.org/~niol/playa/oss/projects/lazygal. AUTHOR
This manual page was written for the Debian system (but may be used by others). Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License, Version 2 any later version published by the Free Software Foundation. On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL. COPYRIGHT
Copyright (C) 2010 lazygal August 2011 LAZYGAL(1)
All times are GMT -4. The time now is 08:29 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy