Sponsored Content
Top Forums Shell Programming and Scripting Finding missing records and Dups Post 302884965 by Saanvi1 on Wednesday 22nd of January 2014 02:33:43 PM
Old 01-22-2014
Finding missing records and Dups

I have a fixed width file. The records looks something similar to below:

Type ID SSN NAME .....AND SOME MORE FIELDS
A1 1234 .....
A1 1234 .....
B1 1234 .....
M2 4567 .....
M2 4567 .....
N2 4567 .....
N2 4567 .....
A1 9999
N2 9999



Now if A1 is present then B1 has to be present. Also if M2 is present then N2 has to be present.
So A1 and B1 record goes together. And M2 and N2 records goes together.

I am looking for TWO things:

1) I am looking for records where A1 record has missing B1 record and vise versa AND I am looking for records
where M2 record has missing N2 record and vice versa.
In other words, if the A1 and B1 combination is missing then throw that record in a file. If M2 and N2 combination is missing throw
that in a file

e.g: So in the above example it will be the records below as both of them has missing B1 and M2 records.

A1 9999
N2 9999

2) I am looking to find duplicate records and put it in a separate file. For Duplicate we look for A1 and 1234 in combination(Type and Id field together).
e.g: So I am looking to put A1 duplicates in one file, B1 duplicates in another file,M2 duplicates in another file and N2 duplicates in another.
so from above example A1 1234 in one file as it is duplicate and M2 4567 will go in one file as it is duplicate and N2 4567 in another file.

Really appreciate your help.

Thanks
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

deleting records with a missing field

I had to delete rows when a record was missing a field. My solution was cut -c 202-402 ${dataFile} | awk '{if (substr($0,103,30) !~ /^ *$/) print $0} >> ${workFile} The cut is because they came two records to a row. Anyone want to offer a more elegant solution? (2 Replies)
Discussion started by: gillbates
2 Replies

2. UNIX for Dummies Questions & Answers

using cat and grep to display missing records

Gentle Unix users, Can someone tell me how I can use a combination of the cat and grep command to display records that are in FileA but missing in FileB. cat FileA one line at a time and grep to see if it is in fileB. If it is ignore. If line is not in fileB display the line. Thanks in... (4 Replies)
Discussion started by: jxh461
4 Replies

3. UNIX for Advanced & Expert Users

Urgent: How can i get the missing records from one file out of two

Hi, I have two files say A and B, Both files have some common records few records which are unique to file A and unique to file B. Can anyone please help me out to find the records which are present in only B Please consider the files are of too large size. Thanks:confused: (1 Reply)
Discussion started by: Shiv@jad
1 Replies

4. UNIX for Advanced & Expert Users

Finding which file is missing

I was hoping someone ould help me with the following. I have 2 files in a directory FILEA and FILEB. i am running a process on these 2 files but before the process can run both FILEA and FILEB need to be present. If one or both the files are missing i need to know what file(s) is(are)... (10 Replies)
Discussion started by: SAMZ
10 Replies

5. Shell Programming and Scripting

Finding missing sequential file names

So, I've got a ton of files that I want to go through (ie something like 300,000), and they're all labeled sequentially. However I'm not 100% positive that they are all there. Is there any way of running through a sequence of numbers, checking if the file is in the folder, if not appending it... (2 Replies)
Discussion started by: Julolidine
2 Replies

6. Shell Programming and Scripting

finding missing items in file

I have a need to compare 2 files, then print results to file. Need to find items from file2 that are not found in file 1. thanks in advance! example: file 1: abcde=12 fffff=6 bbbb=35 file2: abcde=12 fffff=6 bbbb=35 ccccc=10 kkkkk=45 (8 Replies)
Discussion started by: discostu
8 Replies

7. Shell Programming and Scripting

Finding missing tags

I have a list containing strings. All strings should have either "smp" or "drw" else it is considered an error. I have written this code below. Any better ideas to tackle this? set fdrw = 0 set fsmp = 0 foreach f ($Lst) set fdrwtag = `echo $f | awk '/drw/'` set fsmptag = `echo $f | awk... (1 Reply)
Discussion started by: kristinu
1 Replies

8. Shell Programming and Scripting

Finding the records with a specified length

I have a sample txt file which has different variable lengths of 2,10,3,15. What is the command that I need use in order to get the record count that has length '3' Thanks (3 Replies)
Discussion started by: bobby1015
3 Replies

9. AIX

Need help finding missing drivers

I'm in the process of migrating a system to some newer hardware (Power 5 to Power 7). I've done these migrations in the past, and have not had any problems. But this system does not see the new network controllers on the Power 7 system. The system was running AIX 5.3 before, I've upgraded it to... (5 Replies)
Discussion started by: acascianelli
5 Replies

10. UNIX for Beginners Questions & Answers

Finding records NOT on another file

I have three files named ALL, MATCH, and DIFF. Match and diff have completely different records included in the "all" file, but the "all" file also has records not in either the Match or Diff files. I know I can sort all three files together, one unique and one without that option to show which... (5 Replies)
Discussion started by: wbport
5 Replies
UPSC(8) 							    NUT Manual								   UPSC(8)

NAME
upsc - example lightweight UPS client SYNOPSIS
upsc -l | -L [host] upsc ups [variable] upsc -c ups DESCRIPTION
upsc is provided as a quick way to poll the status of a UPS server. It can be used inside shell scripts and other programs that need UPS data but don't want to include the full interface. OPTIONS
-l host List all UPS names configured at host, one name per line. The hostname defaults to "localhost". You may optionally add a colon and a port number. -L host As above, list all UPS names configured at host, including their description provided by the remote upsd(8) from ups.conf(5). The hostname defaults to "localhost". You may optionally add a colon and a port number to override the default port. -c ups Lists each client connected on ups, one name per line. ups Display the status of that UPS. The format for this option is upsname[@hostname[:port]]. The default hostname is "localhost". variable Display the value of this variable only. By default, upsc retrieves the list of variables from the server and then displays the value for each. This may be useful in shell scripts to save an additional pipe into grep. EXAMPLES
To list all variables on an UPS named "myups" on a host called "mybox", with upsd(8) running on port 1234: $ upsc myups@mybox:1234 battery.charge: 100.0 battery.voltage: 13.9 battery.voltage.nominal: 13.6 . . . To list the UPSes configured on this system, along with their descriptions: $ upsc -L apc: Back-UPS 500 ppro2: Patriot Pro II To retrieve the status for all UPSes connected to mybox, using Bourne-shell syntax: $ for UPS in `upsc -l mybox:1234`; do upsc $UPS ups.status done To list clients connected on "myups": $ upsc -c myups 127.0.0.1 ::1 192.168.1.2 DIAGNOSTICS
upsc will either print a list of UPS names, a list of all supported variables and their values on the UPS, or an error message. If you receive an error, make sure you have specified a valid UPS on the command line, that upsd(8) is really running on the other host and that no firewalls are blocking you. HISTORY
Earlier versions of this program used the upsfetch library and UDP sockets to talk to upsd. This version of upsc uses the new upsclient library, which only talks TCP. This is why upsct no longer exists. SEE ALSO
upsd(8) INTERNET RESOURCES
The NUT (Network UPS Tools) home page: http://www.networkupstools.org/ Network UPS Tools 05/21/2012 UPSC(8)
All times are GMT -4. The time now is 05:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy