Sponsored Content
Top Forums Shell Programming and Scripting Performing fast searching operations with a bash script Post 302321457 by jitendriya.dash on Monday 1st of June 2009 08:51:39 AM
Old 06-01-2009
Error Thanks for the input. Request for further inputs.

Thanks for the input.

Using database is a good idea, however placing the contents of all html files here into DB, (as a BLOB/CLOB field) or placing only the <a href > lines, that contains reference to any documents, into DB is a big task.

( i.e. How to insert all these lines into DB, ex- in one html file, if there are 100 <a href> lines, how to place all those lines into DB for 300,000 html files ?)

Can it be done quickly , with an execution of any linux command ?

Actually, the linux server is a 8 core processor. Is there any other way, to quicken the search/grep operation and loop operation by assigning the tasks to multiple cores ?

Please give your inputs.

Thanks and Regards,

Jitendriya Dash.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

fast searching algorithm

hello, i need a searching algorithm in unix. since my input file is very bulky, so need a real fast searching algorithm, to match words. i am already using grep. (3 Replies)
Discussion started by: rochitsharma
3 Replies

2. UNIX for Advanced & Expert Users

Performing Script on Multiple Files

Dear All I have group of files named : CDR.1,CDR.2.,CDR.3,CDR.4,CDR.5,CDR.6,etc....... I am performing an awk command look like this : nawk -f script CDR.* What i want is that i want to perform this command on range of files not all of them. Instead of writing CDR.* i want to write... (3 Replies)
Discussion started by: zanetti321
3 Replies

3. Shell Programming and Scripting

Bash script for searching a string

Hi, I have a file that contains thousands of records. Each record starts with "New Record". I need to search this file based on a given value of "timestamp" and write all the records that match this timestamp into a new file. I was able to locate the existence of a given value of timestamp using... (10 Replies)
Discussion started by: shoponek
10 Replies

4. Shell Programming and Scripting

Need help is manipulating a file with some arithmetic operations using bash script

Friends, I have a file with contents like: interface Serial0/4/0/0/1/1/1/1:0 encapsulation mfr multilink group 101 Now I need to manipulate the file in such a way that to all the numbers less than 163, 63 gets added and to all numbers greater than 163, 63 gets deducted.(The numbers... (2 Replies)
Discussion started by: shrijith1
2 Replies

5. Shell Programming and Scripting

Performing remote operations in a script

I have a scropt that looks something like this: #!/bin/bash ssh user@domain1.com sleep 10 some_command exit ssh different_user@domain2.com sleep 10 some_command exit However, the script is not logging into those accounts and doing the actions. The accounts are configured in my... (3 Replies)
Discussion started by: dotancohen
3 Replies

6. Shell Programming and Scripting

Arithmetic operations in bash,ksh,sh

Guys, The below expression is valid in which shells (sh,ksh,bash,csh)? VAR1=2 VAR2=$(($VAR1 -2)) Thanks (1 Reply)
Discussion started by: rprajendran
1 Replies

7. Shell Programming and Scripting

Can BASH handle mathematical operations and do a Search & Replace?

Hello, I have a bunch of xml file that needs to have edits made and I was wondering if a BASH script could handle it. I would like the script to look within my xml files and replace all integers greater than 5px with a value that is 25% smaller. For example, 100px = 75px. Since the integers... (12 Replies)
Discussion started by: jl487
12 Replies

8. Shell Programming and Scripting

Reading the data from CSV and performing search through shell script

Hello, I am working on building a script that does the below actions together in my Linux server. 1) First, have to read the list of strings mentioned in CSV and store it in the shell script 2) Second, pick one by one from the string list, and search a particular folder for files that... (2 Replies)
Discussion started by: vikrams
2 Replies

9. Shell Programming and Scripting

Performing arithmetic operations on output of `wc -l`

Hi I want to perform arithmetic operations on output of `wc -l`. for example user046@sshell ~ $ ls -l total 0 where "total 0" will increase one line in wc -l filecount=`ls -l | wc -l` here $filecount will be 1 but is should be 0 how to get rid of it ? (1 Reply)
Discussion started by: anandgodse
1 Replies

10. Shell Programming and Scripting

Where to place operations bash scripts?

As I have sometimes problems with passenger module loading correctly after restart of apache2 we wrote a short bash-script to check correct loading of application (redmine) and - if not- restarting apache2 until application is loaded by passenger. Script is invoked using cron. To do everything... (2 Replies)
Discussion started by: awilhelmy
2 Replies
BM(PUBLIC)																BM(PUBLIC)

NAME
bm - search a file for a string SYNOPSIS
/usr/public/bm [ option ] ... [ strings ] [ file ] DESCRIPTION
Bm searches the input files (standard input default) for lines matching a string. Normally, each line found is copied to the standard out- put. It is blindingly fast. Bm strings are fixed sequences of characters: there are no wildcards, repetitions, or other features of regu- lar expressions. Bm is also case sensitive. The following options are recognized. -x (Exact) only lines matched in their entirety are printed -l The names of files with matching lines are listed (once) separated by newlines. -c Only a count of the number of matches is printed -e string The string is the next argument after the -e flag. This allows strings beginning with '-'. -h No filenames are printed, even if multiple files are searched. -n Each line is preceded by the number of characters from the beginning of the file to the match. -s Silent mode. Nothing is printed (except error messages). This is useful for checking the error status. -f file The string list is taken from the file. Unless the -h option is specified the file name is shown if there is more than one input file. Care should be taken when using the charac- ters $ * [ ^ | ( ) and in the strings (listed on the command line) as they are also meaningful to the Shell. It is safest to enclose the entire expression argument in single quotes ' '. Bm searches for lines that contain one of the (newline-separated) strings, using the Boyer-Moore algorithm. It is far superior in terms of speed to the grep (egrep, fgrep) family of pattern matchers for fixed-pattern searching, and its speed increases with pattern length. SEE ALSO
grep(1) DIAGNOSTICS
Exit status is 0 if any matches are found, 1 if none, 2 for syntax errors or inaccessible files. AUTHOR
Peter Bain (pdbain@wateng), with modifications suggested by John Gilmore BUGS
Only 100 patterns are allowed. Patterns may not contain newlines. If a line (delimited by newlines, and the beginning and end of the file) is longer than 8000 charcters (e.g. in a core dump), it will not be completely printed. If multiple patterns are specified, the order of the ouput lines is not necessarily the same as the order of the input lines. A line will be printed once for each different string on that line. The algorithm cannot count lines. The -n and -c work differently from fgrep. The -v, -i, and -b are not available. 4th Berkeley Distribution 8 July 1985 BM(PUBLIC)
All times are GMT -4. The time now is 11:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy