Sponsored Content
Top Forums Shell Programming and Scripting Shell script matching similar records Post 302864269 by kashik786 on Wednesday 16th of October 2013 07:41:26 AM
Old 10-16-2013
Shell script matching similar records

hello all,
I have requirement to identify similar records matching about 80% to 90%.I have to black list customers with multiple accounts.
The data is in the Oracle Database, but is there any way I can get the data into flat file and compare the strings and fetch similar matching records?
I have one table with
Code:
id 
Name
middle_name
address
city

I need to pull out name and address fields similar. And spaces or slight difference can be ignored. I have tried within database methods which is very slow. And I have only basic knowledge of shell scripting need help for a way to get compared result faster. I have very large data to compare..more than 10 millions..

Last edited by Scrutinizer; 10-17-2013 at 04:47 PM.. Reason: code tags
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

update records in a file using shell script...

Hi, I have a file with 6 columns. First 3 columns together make unique record. I have a variable ($v) which hold a value that is obtained by a caliculaion. I have to replace value in 5th columnn with the value of the variable ($v). The value $v is caliculated from col4 and col6 values. ... (2 Replies)
Discussion started by: new_learner
2 Replies

2. Shell Programming and Scripting

need shell script to read particular records from a file

i am reading an i/p file input.txt as below and want to read all filenames as in highlighted in bold below and put them in a different file output.txt. can someone help me with a shell script to do this? thanks in advance regards brad input.txt --------- START TYPE:OPT INIT_SEQ:01... (8 Replies)
Discussion started by: bradc
8 Replies

3. Shell Programming and Scripting

shell script similar to the command HEAD

hello please if anyone can help me make a shell script similar to the command HEAD and TAIL THANKS (3 Replies)
Discussion started by: BELLA86
3 Replies

4. Shell Programming and Scripting

Help with shell script in formatting the records.

I have a text file in the following format. can any one please help me in printing the output in userfriendly format mentioned below. Input. 1) /ss/abc/1/w/s/domainname/abc1/logname/ ########error################### ########error################### ########error###################... (2 Replies)
Discussion started by: vinny81
2 Replies

5. Shell Programming and Scripting

script to return records which are not matching

file1 abcd efgh ijkl mnop file2 mnop qrst uvwx xyza file3 1234 4567 (8 Replies)
Discussion started by: rajivrsk
8 Replies

6. UNIX for Dummies Questions & Answers

Matching and reporting near-similar lines in a file

Hi, I have a file with the lines as below: C_10_A05_T7 C_10_A06_SP6 C_10_B05_SP6 C_10_B05_T7 C_10_B01_SP6 C_10_B01_T7 C_12_G07_SP6 C_12_G11_SP6 C_12_G11_T7 C_2_H18_T7 C_2_I02_SP6 C_2_I02_T7 C_2_I13_SP6 C_2_I17_SP6 The four segments of each line are connected by '_' symbols. I... (7 Replies)
Discussion started by: Fahmida
7 Replies

7. Shell Programming and Scripting

Shell script to print date and no.of records

hi all, i want script to print the output in the following format filename yyyy/mm/dd count where count= no.of records in the file Thanks in advance hemanthsaikumar (11 Replies)
Discussion started by: hemanthsaikumar
11 Replies

8. Shell Programming and Scripting

Join all the lines matching similar pattern

I am trying to Join all the lines matching similar pattern. Example ; I wanted to join all the lines which has sam to a single line. In next line, i wanted to have all the lines with jones to a single line....etc > cat sample.txt sam 2012/11/23 sam 2012/12/5 sam 2012/12/5 jones... (2 Replies)
Discussion started by: evrurs
2 Replies

9. Shell Programming and Scripting

Delete db records from shell script

Hello Mates, I would request your help in a shell script, simply I need to delete some matching db table records (rows) to ones in a given file: ------------------------------ #!/bin/bash SQL="delete from numberlist where msidn='';" MYSQL_USER="<your-user>"... (4 Replies)
Discussion started by: EAGL€
4 Replies

10. Shell Programming and Scripting

Shell script to filter records in a zip file that contains matching columns from another file

Not sure if this is the correct forum for this question. I have two files. file1.zip, file2 Input: file1.zip col1, col2 , col3 a , b , 0:0:0:0:0:c436:9346:d40b x, y, 0:0:0:0:0:880:39f9:c9a7 m, n , 0:0:0:0:0:80c7:9161:fe00 file2.txt col1 c4:36:93:46:d4:0b... (1 Reply)
Discussion started by: anil.v
1 Replies
HWDB(7) 							       hwdb								   HWDB(7)

NAME
hwdb - Hardware Database DESCRIPTION
The hardware database is a key-value store for associating modalias-like keys to udev-property-like values. It is used primarily by udev to add the relevant properties to matching devices, but it can also be queried directly. HARDWARE DATABASE FILES
The hwdb files are read from the files located in the system hwdb directory /lib/udev/hwdb.d and the local administration directory /etc/udev/hwdb.d. All hwdb files are collectively sorted and processed in lexical order, regardless of the directories in which they live. However, files with identical filenames replace each other. Files in /etc have the highest priority and take precedence over files with the same name in /lib. This can be used to override a system-supplied hwdb file with a local file if needed; a symlink in /etc with the same name as a hwdb file in /lib, pointing to /dev/null, disables that hwdb file entirely. hwdb files must have the extension .hwdb; other extensions are ignored. Each hwdb file contains data records consisting of matches and associated key-value pairs. Every record in the hwdb starts with one or more match strings, specifying a shell glob to compare the lookup string against. Multiple match lines are specified in consecutive lines. Every match line is compared individually, and they are combined by OR. Every match line must start at the first character of the line. The match lines are followed by one or more key-value pair lines, which are recognized by a leading space character. The key name and value are separated by "=". An empty line signifies the end of a record. Lines beginning with "#" are ignored. In case multiple records match a given lookup string, the key-value pairs from all records are combined. If a key is specified multiple times, the value from the record with the highest priority is used (each key can have only a single value). The priority is higher when the record is in a file that sorts later lexicographically, and in case of records in the same file, later records have higher priority. The content of all hwdb files is read by systemd-hwdb(8) and compiled to a binary database located at /etc/udev/hwdb.bin, or alternatively /lib/udev/hwdb.bin if you want ship the compiled database in an immutable image. During runtime, only the binary database is used. EXAMPLES
Example 1. General syntax of hwdb files # /lib/udev/hwdb.d/example.hwdb # Comments can be placed before any records. This is a good spot # to describe what that file is used for, what kind of properties # it defines, and the ordering convention. # A record with three matches and one property mouse:*:name:*Trackball*: mouse:*:name:*trackball*: mouse:*:name:*TrackBall*: ID_INPUT_TRACKBALL=1 # A record with a single match and five properties mouse:usb:v046dp4041:name:Logitech MX Master: MOUSE_DPI=1000@166 MOUSE_WHEEL_CLICK_ANGLE=15 MOUSE_WHEEL_CLICK_ANGLE_HORIZONTAL=26 MOUSE_WHEEL_CLICK_COUNT=24 MOUSE_WHEEL_CLICK_COUNT_HORIZONTAL=14 Example 2. Overriding of properties # /lib/udev/hwdb.d/60-keyboard.hwdb evdev:atkbd:dmi:bvn*:bvr*:bd*:svnAcer*:pn* KEYBOARD_KEY_a1=help KEYBOARD_KEY_a2=setup KEYBOARD_KEY_a3=battery evdev:atkbd:dmi:bvn*:bvr*:bd*:svnAcer*:pn123* KEYBOARD_KEY_a2=wlan # /etc/udev/hwdb.d/70-keyboard.hwdb # disable wlan key on all at keyboards evdev:atkbd:* KEYBOARD_KEY_a2=reserved If the hwdb consists of those two files, a keyboard with the lookup string "evdev:atkbd:dmi:bvnAcer:bdXXXXX:bd08/05/2010:svnAcer:pn123" will match all three records, and end up with the following properties: KEYBOARD_KEY_a1=help KEYBOARD_KEY_a2=reserved KEYBOARD_KEY_a3=battery SEE ALSO
systemd-hwdb(8) systemd 237 HWDB(7)
All times are GMT -4. The time now is 01:35 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy