02-10-2007
Advice on Script
I would like some advice on how to logically put together a script to handle a daily task of data gathering for the following problem.
I have two files, file1 has 125,000 records that I cut and remove unwanted fields through scripts and cron. In file2, I have 25000 records that has the same data as file1 plus one extra field of data. I would like the records from file2 (25,000) to replace entries in file1 with the extra data which would then be file3.
I hope I can express this clearly and appreicate any help.
Eample:
file1
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|
file2
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|6 days
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|37 day
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|24 days
file3
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|6 days
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|37 day
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|24 days
00:0A:73:25:B9:EC|20070103_120039|1|7|0|0|
00:01:A6:96:48:60|20070103_120040| | | | |
00:01:A6:A9:3A:7E|20070103_120040|1|8|0|0|
00:0A:73:1F:13:54|20070103_120041|1|8|0|0|
00:40:7B:D6:74:E8|20070103_120041|1|8|0|0|
00:0A:73:23:BA:A8|20070103_120042| | | | |
Last edited by Perderabo; 02-10-2007 at 04:26 PM..
Reason: Disable smilies for readability
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi, I have the following script running in my cron.
--------------------------------------------------------------------
#!/bin/csh
bnstat -p GPD_VSLinux | grep pg | grep varcon | awk '{print $1, $2, $3, $4, $5, $6, $7, $8, $9, $10}' > /tmp/LX_xbatch.log
bnstat -p GPD_VSLinux_test | grep pg... (2 Replies)
Discussion started by: killerserv
2 Replies
2. Shell Programming and Scripting
Hello everyone,
This is my first post here and this is the first time I am using UNIX OS (Slackware). I find it really useful and powerful and would like to master it but as you may guess I am expreicing quite a few problems.
I've been reading a few documentations about it and bash this week... (17 Replies)
Discussion started by: sanchopansa
17 Replies
3. Shell Programming and Scripting
Ok. I want to parse a log file and search only for denied traffic for the previous hour. The log looks like this:
Jun 18 17:47:56 routername 36806: Jun 18 17:53:01.088: %SEC-6-IPACCESSLOG: list ingress-filter denied tcp 1.2.3.4(1234) -> 6.7.8.9(53), 4 packets
I only really care about the... (12 Replies)
Discussion started by: earnstaf
12 Replies
4. Shell Programming and Scripting
Alright, I feel like I have a pretty good basic knowledge of shell scripting, but this one is throwing me for a loop. I know I've seen something similar done with awk, but I couldn't find it with the search function.
I've grepped through my log file and get results like this:
--... (14 Replies)
Discussion started by: earnstaf
14 Replies
5. Shell Programming and Scripting
Hello,
I have this script running on cron every 20 minutes.
By 12pm daily, our system is expecting all input files to be uploaded by the script.
After this cutoff time, the script would still be running though, but i need some kind of alerts/logs to know which input files weren't received for... (1 Reply)
Discussion started by: gholdbhurg
1 Replies
6. Shell Programming and Scripting
I already have a solution to my problem, but I'm looking to see if it can be made more succinct and faster. The problem: given a list, as shown below, extract the pathname for any file in a directory named 'ample' and return it's index into the list. The index is also in the data itself. Note that... (1 Reply)
Discussion started by: prigo
1 Replies
7. Shell Programming and Scripting
Hi folks,
I use following script:-
#!/bin/sh
# cd Linbread
TODAY=`date +"%m%d"`
DATA=`grep $TODAY linbread.dat`
HOUR=`date +"%H"`
if
then TOD="Morning"
elif
then TOD="Afternoon"
else
TOD="Evening"
fi
echo $DATA | gawk -F"|" '{printf("%s\n\n%s",$2,$3)}' > $$tmp
fold -s -w60... (0 Replies)
Discussion started by: satimis
0 Replies
8. Shell Programming and Scripting
Greetings all. I have a repository server which receives, without exhaggeration, several million files a week. The majority of these files are in .csv format, which means they're highly compressable. They are spread throughout numerous directories where there are configured monitoring utilities... (4 Replies)
Discussion started by: msarro
4 Replies
9. Shell Programming and Scripting
All
I have 2 parent directories - input and output. Each parent has multiple sub-directories...each sub-directory has multiple files. Each parent directory structure is a mirror image of itself
I need to poll the imput directory and if a new file is found, encrypt the file, move the file to... (2 Replies)
Discussion started by: davidra
2 Replies
10. UNIX for Advanced & Expert Users
Hi,
Not sure whether this is the right place to post it. I decided to post it here 'coz Advanced and Expert users will most likely have the answer to what I am looking for.
I want to backup scripts that I have access to to a tar file file and zip it. At the moment I am creating a directory... (4 Replies)
Discussion started by: newbie_01
4 Replies
MERGE(1) General Commands Manual MERGE(1)
NAME
merge - three-way file merge
SYNOPSIS
merge [ options ] file1 file2 file3
DESCRIPTION
merge incorporates all changes that lead from file2 to file3 into file1. The result ordinarily goes into file1. merge is useful for com-
bining separate changes to an original. Suppose file2 is the original, and both file1 and file3 are modifications of file2. Then merge
combines both changes.
A conflict occurs if both file1 and file3 have changes in a common segment of lines. If a conflict is found, merge normally outputs a
warning and brackets the conflict with <<<<<<< and >>>>>>> lines. A typical conflict will look like this:
<<<<<<< file A
lines in file A
=======
lines in file B
>>>>>>> file B
If there are conflicts, the user should edit the result and delete one of the alternatives.
OPTIONS
-A Output conflicts using the -A style of diff3(1), if supported by diff3. This merges all changes leading from file2 to file3 into
file1, and generates the most verbose output.
-E, -e These options specify conflict styles that generate less information than -A. See diff3(1) for details. The default is -E. With
-e, merge does not warn about conflicts.
-L label
This option may be given up to three times, and specifies labels to be used in place of the corresponding file names in conflict
reports. That is, merge -L x -L y -L z a b c generates output that looks like it came from files x, y and z instead of from files
a, b and c.
-p Send results to standard output instead of overwriting file1.
-q Quiet; do not warn about conflicts. -V Print 's version number.
DIAGNOSTICS
Exit status is 0 for no conflicts, 1 for some conflicts, 2 for trouble.
IDENTIFICATION
Author: Walter F. Tichy.
Manual Page Revision: 5.7; Release Date: 1995/06/01.
Copyright (C) 1982, 1988, 1989 Walter F. Tichy.
Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995 Paul Eggert.
SEE ALSO
diff3(1), diff(1), rcsmerge(1), co(1).
BUGS
It normally does not make sense to merge binary files as if they were text, but merge tries to do it anyway.
GNU
1995/06/01 MERGE(1)