01-22-2014
Finding missing records and Dups
I have a fixed width file. The records looks something similar to below:
Type ID SSN NAME .....AND SOME MORE FIELDS
A1 1234 .....
A1 1234 .....
B1 1234 .....
M2 4567 .....
M2 4567 .....
N2 4567 .....
N2 4567 .....
A1 9999
N2 9999
Now if A1 is present then B1 has to be present. Also if M2 is present then N2 has to be present.
So A1 and B1 record goes together. And M2 and N2 records goes together.
I am looking for TWO things:
1) I am looking for records where A1 record has missing B1 record and vise versa AND I am looking for records
where M2 record has missing N2 record and vice versa.
In other words, if the A1 and B1 combination is missing then throw that record in a file. If M2 and N2 combination is missing throw
that in a file
e.g: So in the above example it will be the records below as both of them has missing B1 and M2 records.
A1 9999
N2 9999
2) I am looking to find duplicate records and put it in a separate file. For Duplicate we look for A1 and 1234 in combination(Type and Id field together).
e.g: So I am looking to put A1 duplicates in one file, B1 duplicates in another file,M2 duplicates in another file and N2 duplicates in another.
so from above example A1 1234 in one file as it is duplicate and M2 4567 will go in one file as it is duplicate and N2 4567 in another file.
Really appreciate your help.
Thanks
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I had to delete rows when a record was missing a field. My solution was
cut -c 202-402 ${dataFile} | awk '{if (substr($0,103,30) !~ /^ *$/) print $0} >> ${workFile}
The cut is because they came two records to a row.
Anyone want to offer a more elegant solution? (2 Replies)
Discussion started by: gillbates
2 Replies
2. UNIX for Dummies Questions & Answers
Gentle Unix users,
Can someone tell me how I can use a combination of the cat and grep command to display records that are in FileA but missing in FileB.
cat FileA one line at a time and grep to see if it is in fileB. If it is ignore.
If line is not in fileB display the line.
Thanks in... (4 Replies)
Discussion started by: jxh461
4 Replies
3. UNIX for Advanced & Expert Users
Hi,
I have two files say A and B, Both files have some common records few records which are unique to file A and unique to file B.
Can anyone please help me out to find the records which are present in only B
Please consider the files are of too large size.
Thanks:confused: (1 Reply)
Discussion started by: Shiv@jad
1 Replies
4. UNIX for Advanced & Expert Users
I was hoping someone ould help me with the following.
I have 2 files in a directory FILEA and FILEB. i am running a process on these 2 files but before the process can run both FILEA and FILEB need to be present.
If one or both the files are missing i need to know what file(s) is(are)... (10 Replies)
Discussion started by: SAMZ
10 Replies
5. Shell Programming and Scripting
So, I've got a ton of files that I want to go through (ie something like 300,000), and they're all labeled sequentially. However I'm not 100% positive that they are all there.
Is there any way of running through a sequence of numbers, checking if the file is in the folder, if not appending it... (2 Replies)
Discussion started by: Julolidine
2 Replies
6. Shell Programming and Scripting
I have a need to compare 2 files, then print results to file. Need to find items from file2 that are not found in file 1. thanks in advance!
example:
file 1:
abcde=12
fffff=6
bbbb=35
file2:
abcde=12
fffff=6
bbbb=35
ccccc=10
kkkkk=45 (8 Replies)
Discussion started by: discostu
8 Replies
7. Shell Programming and Scripting
I have a list containing strings. All strings should have either "smp" or "drw" else it is considered an error. I have written this code below. Any better ideas to tackle this?
set fdrw = 0
set fsmp = 0
foreach f ($Lst)
set fdrwtag = `echo $f | awk '/drw/'`
set fsmptag = `echo $f | awk... (1 Reply)
Discussion started by: kristinu
1 Replies
8. Shell Programming and Scripting
I have a sample txt file which has different variable lengths of 2,10,3,15.
What is the command that I need use in order to get the record count that has length '3'
Thanks (3 Replies)
Discussion started by: bobby1015
3 Replies
9. AIX
I'm in the process of migrating a system to some newer hardware (Power 5 to Power 7). I've done these migrations in the past, and have not had any problems. But this system does not see the new network controllers on the Power 7 system. The system was running AIX 5.3 before, I've upgraded it to... (5 Replies)
Discussion started by: acascianelli
5 Replies
10. UNIX for Beginners Questions & Answers
I have three files named ALL, MATCH, and DIFF. Match and diff have completely different records included in the "all" file, but the "all" file also has records not in either the Match or Diff files.
I know I can sort all three files together, one unique and one without that option to show which... (5 Replies)
Discussion started by: wbport
5 Replies
UPSC(8) NUT Manual UPSC(8)
NAME
upsc - example lightweight UPS client
SYNOPSIS
upsc -l | -L [host]
upsc ups [variable]
upsc -c ups
DESCRIPTION
upsc is provided as a quick way to poll the status of a UPS server. It can be used inside shell scripts and other programs that need UPS
data but don't want to include the full interface.
OPTIONS
-l host
List all UPS names configured at host, one name per line. The hostname defaults to "localhost". You may optionally add a colon and a
port number.
-L host
As above, list all UPS names configured at host, including their description provided by the remote upsd(8) from ups.conf(5). The
hostname defaults to "localhost". You may optionally add a colon and a port number to override the default port.
-c ups
Lists each client connected on ups, one name per line.
ups
Display the status of that UPS. The format for this option is upsname[@hostname[:port]]. The default hostname is "localhost".
variable
Display the value of this variable only. By default, upsc retrieves the list of variables from the server and then displays the value
for each. This may be useful in shell scripts to save an additional pipe into grep.
EXAMPLES
To list all variables on an UPS named "myups" on a host called "mybox", with upsd(8) running on port 1234:
$ upsc myups@mybox:1234
battery.charge: 100.0
battery.voltage: 13.9
battery.voltage.nominal: 13.6
. . .
To list the UPSes configured on this system, along with their descriptions:
$ upsc -L
apc: Back-UPS 500
ppro2: Patriot Pro II
To retrieve the status for all UPSes connected to mybox, using Bourne-shell syntax:
$ for UPS in `upsc -l mybox:1234`; do
upsc $UPS ups.status
done
To list clients connected on "myups":
$ upsc -c myups
127.0.0.1
::1
192.168.1.2
DIAGNOSTICS
upsc will either print a list of UPS names, a list of all supported variables and their values on the UPS, or an error message. If you
receive an error, make sure you have specified a valid UPS on the command line, that upsd(8) is really running on the other host and that
no firewalls are blocking you.
HISTORY
Earlier versions of this program used the upsfetch library and UDP sockets to talk to upsd. This version of upsc uses the new upsclient
library, which only talks TCP. This is why upsct no longer exists.
SEE ALSO
upsd(8)
INTERNET RESOURCES
The NUT (Network UPS Tools) home page: http://www.networkupstools.org/
Network UPS Tools 05/21/2012 UPSC(8)