Sponsored Content
Top Forums Shell Programming and Scripting Filter uniq field values (non-substring) Post 302900604 by vgersh99 on Wednesday 7th of May 2014 06:08:20 PM
Old 05-07-2014
Quote:
Originally Posted by alister
That approach only tests potential substrings against some of the strings which precede it. As a consequence, the first string in the file is always included.

Consider the result if the first line in the OP's data sample were deleted.

Regards,
Alister
very good point - thank you!
here's a diff approach:
Code:
awk '{for(i in a) if ($2 ~ i) delete a[i];a[$2]=$0}END {for (i in a) print a[i]}' myFile

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Uniq using only the first field

Hi all, I have a file that contains a list of codes (shown below). I want to 'uniq' the file using only the first field. Anyone know an easy way of doing it? Cheers, Dave ##### Input File ##### 1xr1 1xws 1yxt 1yxu 1yxv 1yxx 2o3p 2o63 2o64 2o65 1xr1 1xws 1yxt 1yxv 1yxx 2o3p 2o63 2o64... (8 Replies)
Discussion started by: Digby
8 Replies

2. UNIX for Dummies Questions & Answers

How to uniq third field in a file

Hi ; I have a question regarding the uniq command in unix How do I uniq 3rd field in a file ? original file : zoom coord 39 18652 39 18652 zoom coord 39 18653 39 18653 zoom coord 39 18818 39 18818 zoom coord 39 18840 39 18840 zoom coord 41 15096 41 15096 zoom... (1 Reply)
Discussion started by: babycakes
1 Replies

3. Shell Programming and Scripting

How to use uniq on a certain field?

How can I use uniq on a certain field or what else could I use? If I want to use uniq on the second field and the output would remove one of the lines with a 5. bob 5 hand jane 3 leg jon 4 head chris 5 lungs (1 Reply)
Discussion started by: Bandit390
1 Replies

4. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

5. Shell Programming and Scripting

Uniq based on first field

Hi New to unix. I want to display only the unrepeated lines from a file using first field. Ex: 1234 uname1 status1 1235 uname2 status2 1234 uname3 status3 1236 uname5 status5 I used sort filename | uniq -u output: 1234 uname1 status1 1235 uname2 status2 1234 uname3 status3 1236... (10 Replies)
Discussion started by: venummca
10 Replies

6. Shell Programming and Scripting

Sort field and uniq

I have a flatfile A.txt 2012/12/04 14:06:07 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:07:22 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 17:13:27 |trees|Boards 2, 3|denver|mekong|mekong12 2012/12/04 14:07:39 |rain|Boards 1|tampa|merced|merced11 How do i sort and get... (3 Replies)
Discussion started by: sabercats
3 Replies

7. Shell Programming and Scripting

Printing uniq first field with the the highest second field

Hi All, I am searching for a script which will produce an output file with the uniq first field with the second field having highest value among all the duplicates.. The output file will produce only the uniqs which are duplicate 3 times.. Input file X 9 B 5 A 1 Z 9 T 4 C 9 A 4... (13 Replies)
Discussion started by: ailnilanjan
13 Replies

8. Shell Programming and Scripting

Grok filter to extract substring from path and add to host field in logstash

Hii, I am reading data from files by defining path as *.log etc, Files names are like app1a_test2_heep.log , cdc2a_test3_heep.log etc How to configure logstash so that the part of string that is string before underscore (app1a, cdc2a..) should be grepped and added to host field and... (7 Replies)
Discussion started by: Ravi Kishore
7 Replies

9. Shell Programming and Scripting

HELP - uniq values per column

Hi All, I am trying to output uniq values per column. see file below. can you please assist? Thank you in advance. cat names joe allen ibm joe smith ibm joe allen google joe smith google rachel allen google desired output is: joe allen google rachel smith ibm (5 Replies)
Discussion started by: Apollo
5 Replies

10. Shell Programming and Scripting

awk to update field using matching value in file1 and substring in field in file2

In the awk below I am trying to set/update the value of $14 in file2 in bold, using the matching NM_ in $12 or $9 in file2 with the NM_ in $2 of file1. The lengths of $9 and $12 can be variable but what is consistent is the start pattern will always be NM_ and the end pattern is always ;... (2 Replies)
Discussion started by: cmccabe
2 Replies
TPM QUOTE 
TOOLS(8) TPM QUOTE TOOLS(8) NAME
TPM Quote Tools PROGRAMS
tpm_mkuuid, tpm_mkaik, tpm_loadkey, tpm_unloadkey, tpm_getpcrhash, tpm_updatepcrhash, tpm_getquote, tpm_verifyquote DESCRIPTION
TPM Quote Tools is a collection of programs that provide support for TPM based attestation using the TPM quote operation. A TPM contains a set of Platform Configuration Registers (PCRs). In a well configured machine, some of these registers are set to known values during the boot up process or at other times. For example, a PCR might contain the hash of a boot loader in memory before it is run. The TPM quote operation is used to authoritatively verify the contents of a TPM's Platform Configuration Registers (PCRs). During provi- sioning, a composite hash of a selected set of PCRs is computed. The TPM quote operation produces a composite hash that can be compared with the one computed while provisioning. To use the TPM quote operation, keys must be generated. During provisioning, an Attestation Identity Key (AIK) is generated for each TPM, and the public part of the key is made available to entities that validate quotes. The TPM quote operation returns signed data and a signature. The data that is signed contains the PCRs selected for the operation, the composite hash for the selected PCRs, and a nonce provided as input, and used to prevent replay attacks. At provisioning time, the data that is signed is stored, not just the composite hash. The signature is discarded. An entity that wishes to evaluate a machine generates a nonce, and sends it along with the set of PCR used to generate the composite PCR hash at provisioning time. For this use of the TPM quote operation, the signed data is ignored, and the signature returned is used to val- idate the state of the TPM's PCRs. Given the signature, the evaluating entity replaces the nonce in the signed data generated at provi- sioning time, and checks to see if the signature is valid for the data. If so, this check ensures the selected PCRs contain values that match the ones measured during provisioning. A typical scenario for an enterprise using these tools follows. The tools expect AIKs to be referenced via one enterprise-wide Universally Unique Identifier (UUID). The program tpm_mkuuid creates one. For each machine being checked, an AIK is created using tpm_mkaik. The key blob produced is bound to the UUID on its machine using tpm_loadkey. The public key associated with the AIK is sent to the entities that verify quotes. Finally, the expected PCR composite hash is obtained using tpm_getpcrhash. When the expected PCR values change, a new hash can be generated with tpm_updatepcrhash. The program to obtain a quote, and thus measure the current state of the PCRs is tpm_getquote. The program that verifies the quote describes the same PCR composite hash as was measured initially is tpm_verifyquote. SEE ALSO
tpm_mkuuid(8), tpm_mkaik(8), tpm_loadkey(8), tpm_unloadkey(8), tpm_getpcrhash(8), tpm_updatepcrhash(8), tpm_getquote(8), tpm_verifyquote(8) Oct 2010 TPM QUOTE TOOLS(8)
All times are GMT -4. The time now is 08:05 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy