Using grep -f works, but with two files as large as indicated will take its (serious) time, and may eventually run out of memory. Try
, then use the resultant file in similar way (uniq -u) to extract unique values from either original file.
Comparison of both approaches on ~20k files: EDIT: Times spent for two files with roughly 4E6 entries each, and about 1E6 lines overlap (on a two processor linux host):
Hello
I do want to write a script which will check any errors say "-error" in the log file then have to send email to the concern person . And the concern person will correct the error .
Next time if the script runs eventhough the error has been corrected it will ... (1 Reply)
Hi all,
I'm having some trouble with a shell script that I have put together to search our web pages for links to PDFs.
The first thing I did was:
ls -R | grep .pdf > /tmp/dave_pdfs.outWhich generates a list of all of the PDFs on the server. For the sake of arguement, say it looks like... (8 Replies)
I have a file that is 20 - 80+ MB in size that is a certain type of log file.
It logs one of our processes and this process is multi-threaded. Therefore the log file is kind of a mess. Here's an example:
The logfile looks like: "DATE TIME - THREAD ID - Details", and a new file is created... (4 Replies)
I need help in the following script. I want to grep the sql errors insert into the error table and exit the shell script if there is any error, otherwise keep running the scripts.
Here is my script
#!/bin/csh -f
source .orapass
set user = $USER
set pass = $PASS
cd /opt/data/scripts
echo... (2 Replies)
Hi guys - below is my script that is checking for current file, size and timestamp.
However I added a "grep" feature in it (line in red), but not getting the desired result.
I am trying to acheive in output:
1. Show me the file name, timestamp, size and grep'ed words
It would be a... (2 Replies)
Hi all,
I have problem with searching hundreds of CSV files, the problem is that search is lasting too long (over 5min).
Csv files are "," delimited, and have 30 fields each line, but I always grep same 4 fields - so is there a way to grep just those 4 fields to speed-up search.
Example:... (11 Replies)
Hello everybody,
I'm still slowly treading my way into bash scripting (without any prior programming experience) and hence my code is mostly what some might call "creative" if they meant well :D
I have created a script that serves its purpose but it does so very slowly, since it needs to work... (4 Replies)
This is my first experience writing unix script. I've created the following script. It does what I want it to do, but I need it to be a lot faster. Is there any way to speed it up?
cat 'Tax_Provision_Sample.dat' | sort | while read p; do fn=`echo $p|cut -d~ -f2,4,3,8,9`; echo $p >> "$fn.txt";... (20 Replies)
Hi,
I've written a ksh script that read a file and parse/filter/format each line. The script runs as expected but it runs for 24+ hours for a file that has 2million lines. And sometimes, the input file has 10million lines which means it can be running for more than 2 days and still not finish.... (9 Replies)
Hello experts,
we have input files with 700K lines each (one generated for every hour). and we need to convert them as below and move them to another directory once.
Sample INPUT:-
# cat test1
1559205600000,8474,NormalizedPortInfo,PctDiscards,0.0,Interface,BG-CTA-AX1.test.com,Vl111... (7 Replies)
Discussion started by: prvnrk
7 Replies
LEARN ABOUT DEBIAN
vm
VM(1) mgetty VM(1)NAME
vm - VoiceModem is the program for handling the voice modem functionality from shell scripts
ACTIONS
beep options [<frequency [<length in 0.001sec>]]>
diagnostics options device name (e.g. ttyS2
dial options phone number
help
play options [<file names]+>
record options file name
shell options [<shell script [shell options]]>
wait options [<time in seconds]>
devicetest
OPTIONS -c n use compression type n
-d n set i/o device
-t, -m, -i, -e, -s, -H equals to -d <2,3,4,5,6,7>
-l s set device string (e.g. -l ttyS2:ttyC0)
-v verbose output
-w use off / on hook signal from local handset to start and stop recording
-x n set debug level
-L n set maximum recording length in sec
-P print first DTMF tone on stdout and exit
-R read and print DTMF string on stdout and exit
-S s set default shell for shell scripts (e.g. -S /bin/sh)
-T n set silence timeout in 0.1sec
-V n set silence threshold to <n> (0-100%%)
SEE ALSO vgetty(1)POD ERRORS
Hey! The above document had some coding errors, which are explained below:
Around line 30:
You forgot a '=back' before '=head1'
Around line 32:
'=item' outside of any '=over'
Around line 71:
You forgot a '=back' before '=head1'
perl v5.10.1 2010-04-04 VM(1)