10-21-2015
If you want to find one of 10k items in a large file, there's no straight, easy way to avoid comparing each item against each line in file. What you can do is
break when found.
You could try a
binary search algorithm.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hello all
greeting for the day
i have a text file as the following
text.xml
abcd<FIELD>123.456</FIELD>efgh
i need to replace the value between <FIELD> and </FIELD> by using awk command.
please throw some light on this.
thank you very very much
Erik (5 Replies)
Discussion started by: erikshek
5 Replies
2. Shell Programming and Scripting
Hello!
I have text file:
From aaa@bbb Fri Jun 1 10:04:29 2010
--____OSPHWOJQGRPHNTTXKYGR____
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
My code '234565'. ... (2 Replies)
Discussion started by: candyme
2 Replies
3. Shell Programming and Scripting
Hello friends!
Help me pls to write correct awk and grep statements for my task:
I have got files with name filename.txt
It has such structure:
Start of file
FROM: address@domen.com (12...890) abc
DATE: 11/23/2009 on Std
SUBJECT: any subject
End of file
So, I must check,
if this file... (4 Replies)
Discussion started by: candyme
4 Replies
4. Shell Programming and Scripting
How to reverse search for a matched string in a file. Get line# of the first matched line. I am getting '2' into 'lineNum' variable.
But it feels like I am using too many commands. Is there a better more efficiant way to do this on Unix?
abc.log
aaaaaaaaaaaaa
bbbbbbbbbbbbb... (11 Replies)
Discussion started by: kchinnam
11 Replies
5. Shell Programming and Scripting
Hi,
I have an XML file with around 1 billion rows in it and i am trying to find the number of times a particular tag occurs in it. The solution i am using works but takes a lot of time (~1 hr) .Please help me with an efficient way to do this.
Lets say the input file is
<Root>
... (13 Replies)
Discussion started by: Sheel
13 Replies
6. Homework & Coursework Questions
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
Write a template main.c file via shell script to make it easier for yourself later.
The issue here isn't writing... (2 Replies)
Discussion started by: george3isme
2 Replies
7. Shell Programming and Scripting
Hi,
I am trying to populate an array with data from a text file. I have a working method using awk but it is too slow and inefficent. See below.
The text file has 70,000 lines. As awk is a line editor it reads each line of the file until it gets to the required line and then processes it.... (3 Replies)
Discussion started by: carlr
3 Replies
8. Shell Programming and Scripting
Hello,
Some time ago a helpful awk file was provided on the forum which I give below:
NR==FNR{A=$0;next}{for(j in A){split(A,P,"=");for(i=1;i<=NF;i++){if($i==P){$i=P}}}}1
While it works beautifully on English and Latin characters i.e. within the ASCII range of 127, the moment a character beyond... (6 Replies)
Discussion started by: gimley
6 Replies
9. Shell Programming and Scripting
Hi Friends,
I have a very big text file, that has code for multiple functions. I have scan through the file and write each function in seperate file. All functions starts with
BEGIN DSFNC
Identifier "ABCDDataValidationfnc"
and ends with
END DSFNC
I need create a file(using identifier)... (2 Replies)
Discussion started by: anandapani
2 Replies
10. Shell Programming and Scripting
Hi guys,
I have a text file named file1.txt that is formatted like this:
001 , ID , 20000
002 , Name , Brandon
003 , Phone_Number , 616-234-1999
004 , SSNumber , 234-23-234
005 , Model , Toyota
007 , Engine ,V8
008 , GPS , OFF
and I have file2.txt formatted like this:
... (2 Replies)
Discussion started by: An0mander
2 Replies
LEARN ABOUT DEBIAN
htpurge
htdig(1) General Commands Manual htdig(1)
NAME
htpurge - remove unused documents from the database (general maintenance script)
SYNOPSIS
htpurge [-][-a][-c configfile][-u][-v]
DESCRIPTION
Htpurge functions to remove specified URLs from the databases as well as bad URLs, unretrieved URLs, obsolete documents, etc. It is recom-
mended that htpurge be run after htdig to clean out any documents of this sort.
OPTIONS
- Take URL list from standard input (rather than specified with -u). Format of input file is one URL per line. -a Use alternate work
files. Tells htpurge to append .work to database files, causing a second copy of the database to be built. This allows the original
files to be used by htsearch during the run.
-c configfile
Use the specified configfile instead of the default.
-u URL Add this URL to the list of documents to remove. Must be specified multiple times if more than one URL are to be removed. Should nor
be used together with -.
-v Verbose mode. This increases the verbosity of the program. Using more than 2 is probably only useful for debugging purposes. The
default verbose mode (using only one -v) gives a nice progress report while digging.
FILES
/etc/htdig/htdig.conf
The default configuration file.
SEE ALSO
Please refer to the HTML pages (in the htdig-doc package) /usr/share/doc/htdig-doc/html/index.html and the manual pages htdigconfig(8) ,
htdig(1) and htmerge(1) for a detailed description of ht://Dig and its commands.
AUTHOR
This manual page was written by Robert Ribnitz, based on the HTML documentation of ht://Dig.
January 2004 htdig(1)