Sponsored Content
Full Discussion: Best match query
Top Forums Programming Best match query Post 302981335 by stomp on Saturday 10th of September 2016 09:29:52 AM
Old 09-10-2016
Hi,

maybe you create a function which figures out the count of equal numbers from the left. Let it for example be called: "get_equal_numbers(col_number,my_number)" See here fore documentation: CREATE FUNCTION examples

Then you may create a query like this:

Code:
SELECT col_number,rate,get_equal_numbers(col_number,my_number) AS equals 
FROM destinations
WHERE equals >= 1 
ORDER BY equals DESC

I'm not sure if this performes well.

The choice I normally would take is too read the whole table into an array and use a custom search function, which does the same as above.

Both solutions are not that efficient - especially when you have a lot of lookups to accomplish.

For speed and improving search effiency, I probably would build a hash like this(example in ruby-code):

Code:
#!/usr/bin/ruby

def get_routing_target(number)
   # number is a string
   routing_table = [
     "001" => "data1",
     "0012 => "data2",
     "00129" => "data3" ]
   for current_length in number.length .. 1
        if routing_table[number[0,current_length]] then
            return routing_table[number[0,current_length]
        end
   end
end

The last idea does not allow to have multiple routing targets for the same number. So this needs a bit adjustment to enable that("data" may be implemented as an array of multiple routing targets).

In general I think this may be a performance critical issue, which may cause very high load - depending on the number of requests that will be processed - on your server(DB or application server) if not designed well. So maybe it is better to have a program running as daemon, keeping the complete routing table in memory and reloading from time to time instead of reading it for every request.

UPDATE-1:

I tried it and created some test data(100k Records, with index on the number) and a mysql function:

Code:
DROP FUNCTION IF EXISTS equal_chars_from_left;
DELIMITER $$
CREATE FUNCTION equal_chars_from_left(wanted TEXT, current_field TEXT)
  RETURNS INT
BEGIN
  DECLARE current_length INT;
  SET current_length = LENGTH(wanted);

  count_loop: LOOP
    IF SUBSTRING(current_field,1,current_length) = SUBSTRING(wanted,1,current_length) THEN
        LEAVE count_loop;
    END IF;
    SET current_length = current_length - 1;

    IF current_length = 0 THEN
      LEAVE count_loop;
    END IF;
    
  END LOOP;
  RETURN current_length;
END;
$$
DELIMITER ;

The performance is as bad as expected.

Code:
mysql> select destination,rate,equal_chars_from_left("1212313",destination) as equals from test order by equals desc limit 10;
+----------------+------+--------+
| destination    | rate | equals |
+----------------+------+--------+
| 1212313893101  | 0.97 |      7 |
| 12123105591    | 0.09 |      6 |
| 12123          | 0.98 |      5 |
| 12123235501917 | 0.07 |      5 |
| 121283339945   | 0.19 |      4 |
| 121243075      | 0.53 |      4 |
| 121251         | 0.69 |      4 |
| 1212626040     | 0.23 |      4 |
| 1212914454156  | 0.72 |      4 |
| 121251042914   | 0.46 |      4 |
+----------------+------+--------+
10 rows in set (6,35 sec)

UPDATE 2:

This is a bit faster:

Code:
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '1212313%'  UNION
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '121231%'  UNION
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '12123%'  UNION
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '1212%'  UNION
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '121%'  UNION
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '12%'  UNION
SELECT destination,equal_chars_from_left('1212313',destination) as equals from test where destination like '1%' 
ORDER BY equals DESC LIMIT 10;

Code:
+----------------+--------+
| destination    | equals |
+----------------+--------+
| 1212313893101  |      7 |
| 12123105591    |      6 |
| 12123          |      5 |
| 12123235501917 |      5 |
| 12120247005    |      4 |
| 121283339945   |      4 |
| 12125241424986 |      4 |
| 1212426494209  |      4 |
| 1212116831     |      4 |
| 121284268      |      4 |
+----------------+--------+
10 rows in set (2,10 sec)


Last edited by stomp; 09-10-2016 at 05:36 PM..
This User Gave Thanks to stomp For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

add the output of a query to a variable to be used in another query

I would like to use the result of a query in another query. How do I redirect/add the output to another variable? $result = odbc_exec($connect, $query); while ($row = odbc_fetch_array($result)) { echo $row,"\n"; } odbc_close($connect); ?> This will output hostnames: host1... (0 Replies)
Discussion started by: hazno
0 Replies

2. UNIX and Linux Applications

mysql query all entries which 'dont' match

I am trying to query a list of hosts and extract all entries which 'dont' match. SELECT LOGS.host, GOODLIST.host FROM LOGS,db.GOODLIST WHERE (LOGS.host <> GOODLIST.host)When I use this query, it is very very slow. Matching the host with the GOODLIST.host works great and fast but when I use <>... (1 Reply)
Discussion started by: hazno
1 Replies

3. Shell Programming and Scripting

Query Oracle tables and return values to shell script that calls the query

Hi, I have a requirement as below which needs to be done viz UNIX shell script (1) I have to connect to an Oracle database (2) Exexute "SELECT field_status from table 1" query on one of the tables. (3) Based on the result that I get from point (2), I have to update another table in the... (6 Replies)
Discussion started by: balaeswari
6 Replies

4. Programming

sql query to match condition from other table (time sensitive)

I know little SQL and could really use a hand here. I need to get the fields last_name, first_name and email from the table users and only if the entery with the same pkey in the table addresses, in the city column is Seattle. So if the city in the addresses table is Seattle, go to the table... (2 Replies)
Discussion started by: computethis
2 Replies

5. Shell Programming and Scripting

Shell Script to execute Oracle query taking input from a file to form query

Hi, I need to query Oracle database for 100 users. I have these 100 users in a file. I need a shell script which would read this User file (one user at a time) & query database. For instance: USER CITY --------- ---------- A CITY_A B CITY_B C ... (2 Replies)
Discussion started by: DevendraG
2 Replies

6. Shell Programming and Scripting

Match pattern1 in file, match pattern2, substitute value1 in line

not getting anywhere with this an xml file contains multiple clients set up with same tags, different values. I need to parse the file for client foo, and change the value of tag "64bit" from false to true. cat clients.xml <Client type"FIX"> <ClientName>foo</ClientName>... (3 Replies)
Discussion started by: jack.bauer
3 Replies

7. Shell Programming and Scripting

Request: How to Parse dynamic SQL query to pad extra columns to match the fixed number of columns

Hello All, I have a requirement in which i will be given a sql query as input in a file with dynamic number of columns. For example some times i will get 5 columns, some times 8 columns etc up to 20 columns. So my requirement is to generate a output query which will have 20 columns all the... (7 Replies)
Discussion started by: vikas_trl
7 Replies

8. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

9. Shell Programming and Scripting

awk to print match or non-match and select fields/patterns for non-matches

In the awk below I am trying to output those lines that Match between file1 and file2, those Missing in file1, and those missing in file2. Using each $1,$2,$4,$5 value as a key to match on, that is if those 4 fields are found in both files the match, but if those 4 fields are not found then missing... (0 Replies)
Discussion started by: cmccabe
0 Replies

10. UNIX for Beginners Questions & Answers

Data match 2 files based on first 2 columns matching only and join if match

Hi, i have 2 files , the data i need to match is in masterfile and i need to pull out column 3 from master if column 1 and 2 match and output entire row to new file I have tried with join and awk and i keep getting blank outputs or same file is there an easier way than what i am... (4 Replies)
Discussion started by: axis88
4 Replies
All times are GMT -4. The time now is 06:28 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy