Sponsored Content
Top Forums Shell Programming and Scripting Searching for strings amongst non-uniform data Post 302581690 by gazza86 on Tuesday 13th of December 2011 10:29:48 PM
Old 12-13-2011
Searching for strings amongst non-uniform data

Hi Guys,

I have a source file which contains significant strings amongst a lot of dross in non-uniform format, I'd like to search the input file for any examples of data from my reference file, and then output any matches to a list (text file).

I've made something that achieves this, it's just very slow (The reference file contains hundreds of entries) - Is anyone able to suggest a faster / more efficient method of achieving the below output?

Many thanks,

Gary

Code:
#script.sh
cat referencefile.txt | while read HOST
do
check=`grep $HOST input.txt | wc -l`
if [  "$check" -ne 0 ]; then
echo $HOST >> output.txt
fi
done

referencefile.txt:
hostname1
hostname2
hostname3
hostname4
hostname5
hostname6
hostname7
hostname8
hostname9
hostname10

input.txt:
random text random text hostname1 random text
random text hostname3 random text
randomtexthostname6randomtext random text

output.txt:
hostname1
hostname3
hostname6

Last edited by gazza86; 12-13-2011 at 11:33 PM.. Reason: Post had wrapped funny for some reason.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

searching for strings/user IP addresses

Hi, I'm trying to write a script, which will perform the following actions. Pick up the IP address of the PC I have used to telnet into the SUN server. Export this. Run the rest of my script. I am struggling with the first part, I know the IP address can be displayed by the command... (2 Replies)
Discussion started by: 30694
2 Replies

2. Shell Programming and Scripting

searching for filenames with search strings in another file

Hi, I have 5 files in a directory. emp1_usage.txt emp2_usage.txt emp3_usage.txt emp4_usage.txt emp5_usage.txt I am using sqlldr to get the contents of the above 5 files and store it in a temp table and update my original table using temp table. for f in *emp*.txt do sqlldr... (3 Replies)
Discussion started by: pathanjalireddy
3 Replies

3. Solaris

higlighting strings while searching

Hello experts, i am using sun solaris 9 i try to searching string from a file using more command. I wish when i search a string it will higlight the string/strings from the file. Have any idea how to do it..? I use putty. br//purple (9 Replies)
Discussion started by: thepurple
9 Replies

4. Shell Programming and Scripting

Need help in searching 2 files for strings then perform an action

I have 2 files. I basically want to search both of them to see if the 1st column ($1) matches and if it matches then check to see if the 2nd column ($2) matches, then execute some code showing the results of the matches. File 1: AAA 123 misc blah BBB 456 CCC 789 File 2: ... (2 Replies)
Discussion started by: streetfighter2
2 Replies

5. Shell Programming and Scripting

reformating non-uniform strings

I have a set of free-form phone numbers that are not uniform and I want to reformat them into a standard uniform string. These are embedded at the end of a colon seperated file built by a large nawk + tr piping like such: XXXXX:YYYYY:ZZZZZ:(333)333-3333x33333 XXXXX:YYYYY:ZZZZZ:x44444... (9 Replies)
Discussion started by: lordsmiter
9 Replies

6. Shell Programming and Scripting

data searching and pasting with in the file

Hi All, I have .csv file in which I am trying to manipulate a column date, I started with awk but i am not sure how to do the below logic in . the file has a 23 columns and in the first row if the value is Trend and in the second column the value is Analysis then the program has to... (3 Replies)
Discussion started by: shruthidwh
3 Replies

7. Shell Programming and Scripting

Searching for string between two different strings

I want to the following results using unix script, can some one help ? Thanks Input: select col1, col2 from tab1 where ......... select col1,.....,coln from tab2, tab3 where.... select clo1,clo2,col3 from tab4 where... output (results) tab1 tab2 tab3 tab4 basically I need... (5 Replies)
Discussion started by: kanagalav
5 Replies

8. Shell Programming and Scripting

Searching for a list of strings in a file with Python

Hi guys, I'm trying to search for several strings, which I have in a .txt file line by line, on another file. So the idea is, take input.txt and search for each line in that file in another file, let's call it rules.txt. So far, I've been able to do this, to search for individual strings: ... (1 Reply)
Discussion started by: starriol
1 Replies

9. Programming

Hsearch() problem when searching for strings.

I have written a code in C where the objective is to search for strings. There are two files: 1. Database file which has more than one billion entries. This file is read in argv in the C code below. The format of the file is like this: a.txt apple b.txt candle c.txt glue 2. There is another... (1 Reply)
Discussion started by: shoaibjameel123
1 Replies

10. UNIX for Beginners Questions & Answers

Can I combine below mentioned grep commands using OR (when searching strings having spaces)

Command 1: $script | grep 'Write to ECC( SSID=MARGIN)' Command 2: $script | grep 'is not greater than existing logical processing' The above commands run my script and search the mentioned strings but I do not want to run my script twice. It is increasing run time. Can someone tell me... (3 Replies)
Discussion started by: Tanu
3 Replies
WRJPGCOM(1)						      General Commands Manual						       WRJPGCOM(1)

NAME
wrjpgcom - insert text comments into a JPEG file SYNOPSIS
wrjpgcom [ -replace ] [ -comment text ] [ -cfile name ] [ filename ] DESCRIPTION
wrjpgcom reads the named JPEG/JFIF file, or the standard input if no file is named, and generates a new JPEG/JFIF file on standard output. A comment block is added to the file. The JPEG standard allows "comment" (COM) blocks to occur within a JPEG file. Although the standard doesn't actually define what COM blocks are for, they are widely used to hold user-supplied text strings. This lets you add annotations, titles, index terms, etc to your JPEG files, and later retrieve them as text. COM blocks do not interfere with the image stored in the JPEG file. The maximum size of a COM block is 64K, but you can have as many of them as you like in one JPEG file. wrjpgcom adds a COM block, containing text you provide, to a JPEG file. Ordinarily, the COM block is added after any existing COM blocks; but you can delete the old COM blocks if you wish. OPTIONS
Switch names may be abbreviated, and are not case sensitive. -replace Delete any existing COM blocks from the file. -comment text Supply text for new COM block on command line. -cfile name Read text for new COM block from named file. If you have only one line of comment text to add, you can provide it on the command line with -comment. The comment text must be sur- rounded with quotes so that it is treated as a single argument. Longer comments can be read from a text file. If you give neither -comment nor -cfile, then wrjpgcom will read the comment text from standard input. (In this case an input image file name MUST be supplied, so that the source JPEG file comes from somewhere else.) You can enter multiple lines, up to 64KB worth. Type an end-of-file indicator (usually control-D) to terminate the comment text entry. wrjpgcom will not add a COM block if the provided comment string is empty. Therefore -replace -comment "" can be used to delete all COM blocks from a file. EXAMPLES
Add a short comment to in.jpg, producing out.jpg: wrjpgcom -c "View of my back yard" in.jpg > out.jpg Attach a long comment previously stored in comment.txt: wrjpgcom in.jpg < comment.txt > out.jpg or equivalently wrjpgcom -cfile comment.txt < in.jpg > out.jpg SEE ALSO
cjpeg(1), djpeg(1), jpegtran(1), rdjpgcom(1) AUTHOR
Independent JPEG Group 15 June 1995 WRJPGCOM(1)
All times are GMT -4. The time now is 04:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy