06-01-2009
Thanks for the input. Request for further inputs.
Thanks for the input.
Using database is a good idea, however placing the contents of all html files here into DB, (as a BLOB/CLOB field) or placing only the <a href > lines, that contains reference to any documents, into DB is a big task.
( i.e. How to insert all these lines into DB, ex- in one html file, if there are 100 <a href> lines, how to place all those lines into DB for 300,000 html files ?)
Can it be done quickly , with an execution of any linux command ?
Actually, the linux server is a 8 core processor. Is there any other way, to quicken the search/grep operation and loop operation by assigning the tasks to multiple cores ?
Please give your inputs.
Thanks and Regards,
Jitendriya Dash.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hello,
i need a searching algorithm in unix. since my input file is very bulky, so need a real fast searching algorithm, to match words. i am already using grep. (3 Replies)
Discussion started by: rochitsharma
3 Replies
2. UNIX for Advanced & Expert Users
Dear All
I have group of files named :
CDR.1,CDR.2.,CDR.3,CDR.4,CDR.5,CDR.6,etc.......
I am performing an awk command look like this : nawk -f script CDR.*
What i want is that i want to perform this command on range of files not all of them.
Instead of writing CDR.* i want to write... (3 Replies)
Discussion started by: zanetti321
3 Replies
3. Shell Programming and Scripting
Hi,
I have a file that contains thousands of records. Each record starts with "New Record". I need to search this file based on a given value of "timestamp" and write all the records that match this timestamp into a new file.
I was able to locate the existence of a given value of timestamp using... (10 Replies)
Discussion started by: shoponek
10 Replies
4. Shell Programming and Scripting
Friends,
I have a file with contents like:
interface Serial0/4/0/0/1/1/1/1:0
encapsulation mfr
multilink
group 101
Now I need to manipulate the file in such a way that to all the numbers less than 163, 63 gets added and to all numbers greater than 163, 63 gets deducted.(The numbers... (2 Replies)
Discussion started by: shrijith1
2 Replies
5. Shell Programming and Scripting
I have a scropt that looks something like this:
#!/bin/bash
ssh user@domain1.com
sleep 10
some_command
exit
ssh different_user@domain2.com
sleep 10
some_command
exit
However, the script is not logging into those accounts and doing the actions. The accounts are configured in my... (3 Replies)
Discussion started by: dotancohen
3 Replies
6. Shell Programming and Scripting
Guys,
The below expression is valid in which shells (sh,ksh,bash,csh)?
VAR1=2
VAR2=$(($VAR1 -2))
Thanks (1 Reply)
Discussion started by: rprajendran
1 Replies
7. Shell Programming and Scripting
Hello,
I have a bunch of xml file that needs to have edits made and I was wondering if a BASH script could handle it.
I would like the script to look within my xml files and replace all integers greater than 5px with a value that is 25% smaller. For example, 100px = 75px. Since the integers... (12 Replies)
Discussion started by: jl487
12 Replies
8. Shell Programming and Scripting
Hello,
I am working on building a script that does the below actions together in my Linux server.
1) First, have to read the list of strings mentioned in CSV and store it in the shell script
2) Second, pick one by one from the string list, and search a particular folder for files that... (2 Replies)
Discussion started by: vikrams
2 Replies
9. Shell Programming and Scripting
Hi
I want to perform arithmetic operations on output of `wc -l`.
for example
user046@sshell ~ $ ls -l
total 0
where "total 0" will increase one line in wc -l
filecount=`ls -l | wc -l`
here $filecount will be 1 but is should be 0
how to get rid of it ? (1 Reply)
Discussion started by: anandgodse
1 Replies
10. Shell Programming and Scripting
As I have sometimes problems with passenger module loading correctly after restart of apache2 we wrote a short bash-script to check correct loading of application (redmine) and - if not- restarting apache2 until application is loaded by passenger. Script is invoked using cron.
To do everything... (2 Replies)
Discussion started by: awilhelmy
2 Replies
BM(PUBLIC) BM(PUBLIC)
NAME
bm - search a file for a string
SYNOPSIS
/usr/public/bm [ option ] ... [ strings ] [ file ]
DESCRIPTION
Bm searches the input files (standard input default) for lines matching a string. Normally, each line found is copied to the standard out-
put. It is blindingly fast. Bm strings are fixed sequences of characters: there are no wildcards, repetitions, or other features of regu-
lar expressions. Bm is also case sensitive. The following options are recognized.
-x (Exact) only lines matched in their entirety are printed
-l The names of files with matching lines are listed (once) separated by newlines.
-c Only a count of the number of matches is printed
-e string
The string is the next argument after the -e flag. This allows strings beginning with '-'.
-h No filenames are printed, even if multiple files are searched.
-n Each line is preceded by the number of characters from the beginning of the file to the match.
-s Silent mode. Nothing is printed (except error messages). This is useful for checking the error status.
-f file
The string list is taken from the file.
Unless the -h option is specified the file name is shown if there is more than one input file. Care should be taken when using the charac-
ters $ * [ ^ | ( ) and in the strings (listed on the command line) as they are also meaningful to the Shell. It is safest to enclose the
entire expression argument in single quotes ' '.
Bm searches for lines that contain one of the (newline-separated) strings, using the Boyer-Moore algorithm. It is far superior in terms of
speed to the grep (egrep, fgrep) family of pattern matchers for fixed-pattern searching, and its speed increases with pattern length.
SEE ALSO
grep(1)
DIAGNOSTICS
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files.
AUTHOR
Peter Bain (pdbain@wateng), with modifications suggested by John Gilmore
BUGS
Only 100 patterns are allowed.
Patterns may not contain newlines.
If a line (delimited by newlines, and the beginning and end of the file) is longer than 8000 charcters (e.g. in a core dump), it will not
be completely printed.
If multiple patterns are specified, the order of the ouput lines is not necessarily the same as the order of the input lines.
A line will be printed once for each different string on that line.
The algorithm cannot count lines.
The -n and -c work differently from fgrep.
The -v, -i, and -b are not available.
4th Berkeley Distribution 8 July 1985 BM(PUBLIC)