Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: validating a file based on conditions
Top Forums Shell Programming and Scripting validating a file based on conditions Post 302272702 by radoulov on Wednesday 31st of December 2008 12:12:12 PM
Old 12-31-2008
Quote:
Originally Posted by Christoph Spohr
Hi radoulov,

another one of your astonishing awk scripts. Could you go in
some detail how it works.

Regards

Chris
Astonishing Smilie,
thank you!

I'll try to explain.
Code followed by comments.

Code:
{
  u[$1] = k[$1,$2]++ ? x : 1      
  }

While reading the input build an associative array named u (unique) keyed by $1. The values are build/chosen based on the following expression:

Code:
k[$1,$2]++ ? x : 1

If the value of the auto incremented associative array k, build en passant with $1 SUBSEP $2 as keys, is different than 0 (i.e. true in boolean context), i.e. already seen (remember Ed Morton's !arr[val]++?),
then return and assign the value of the variable x (never used and auto initialized -> null -> 0 in numeric context -> false in boolean context, if I had written 0, it would have been clearer Smilie), otherwise return and assign the value 1 (the opposite of the previous).

Code:
END { 
  for (_ in u) if (u[_])
      print _
    }

After reading all the input, print only those u keys whose values are true when evaluated in boolean context (which equal to 1).

Happy holidays!
Last edited by radoulov; 12-31-2008 at 01:18 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Validating input based on fixed number of fields

Yes, i did... let me state my problem in more detail Inputs: I have one input CSV file And, i have stored no. of comma each line should in a variable. e.g. $ cat cmt.csv this, is a ,comma ,count test1 ,,this, is a ,comma ,count test2 this, is a ,comma ,count test3... (6 Replies)
Discussion started by: Dipali
6 Replies

2. UNIX for Dummies Questions & Answers

How to get remove duplicate of a file based on many conditions

Hii Friends.. I have a huge set of data stored in a file.Which is as shown below a.dat: RAO 1869 12 19 0 0 0.00 17.9000 82.3000 10.0 0 0.00 0 3.70 0.00 0.00 0 0.00 3.70 4 NULL LEE 1870 4 11 1 0 0.00 30.0000 99.0000 0.0 0 0.00 0 0.00 0.00 0.00 0 ... (3 Replies)
Discussion started by: reva
3 Replies

3. Shell Programming and Scripting

Extract file records based on some field conditions

Hello Friends, I have a file(InputFile.csv) with the following columns(the columns are pipe-delimited): ColA|ColB|ColC|ColD|ColE|ColF Now for this file, I have to get those records which fulfil the following condition: If "ColB" is NOT NULL and "ColD" has values one of the following... (9 Replies)
Discussion started by: mehimadri
9 Replies

4. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file based on certain conditions

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (1 Reply)
Discussion started by: Vivekit82
1 Replies

5. Shell Programming and Scripting

Split File based on different conditions

I need to split the file Conditions: Ignore any record that either starts with 1 or 9 Split the file at position 404 , if position 404 is abc or def then write all the records in a file > File 1 , the remaining records should go in to a file > File 2 Further I want to split the... (7 Replies)
Discussion started by: protech
7 Replies

6. Shell Programming and Scripting

Create new file with increment column based on conditions

Hello, Using bash script, i need to process the following file: 887,86,,2013-11-06,1,10030,5,2,0,200,, 887,86,,2013-11-05,1,10030,5,2,0,199,, 887,138,,2013-11-06,1,10031,6,2,0,1610612736,, 887,164,,2013-11-06,1,10000,0,2,0,36000,, and to create a new file such as the below ... (2 Replies)
Discussion started by: JonhyDeep
2 Replies

7. Shell Programming and Scripting

Help with Creating file based on conditions

Can anyone please assist? I have a .txt file(File1.txt) and a property file(propertyfile.txt) . I have to read the vales from the property file and .txt file and create the output file(outputfile.txt) mentioned in the attachment. For each record in .txt file,the below mentioned values shall be... (20 Replies)
Discussion started by: vinus
20 Replies

8. Shell Programming and Scripting

awk to filter file based on seperate conditions

The below awk will filter a list of 30,000 lines in the tab-delimited file. What I am having trouble with is adding a condition to SVTYPE=CNV that will only print that line if CI= must be >.05 . The other condition to add is if SVTYPE=Fusion, then in order to print that line READ_COUNT must... (3 Replies)
Discussion started by: cmccabe
3 Replies

9. Shell Programming and Scripting

awk to update file based on 5 conditions

I am trying to use awk to update the below tab-delimited file based on 5 different rules/conditions. The final output is also tab-delimited and each line in the file will meet one of the conditions. My attemp is below as well though I am not very confident in it. Thank you :). Condition 1: The... (10 Replies)
Discussion started by: cmccabe
10 Replies

10. Shell Programming and Scripting

Search and replace value based on certain conditions in a fixed width file

Hi Forum. I tried searching for a solution using the internet search but I haven't been able to find any solution for what I'm trying to accomplish. I have a fixed width column file where I need to search for any occurrences of "D0" in col pos.#1-2, 10-11, 20-21 and replaced it with "XD". ... (2 Replies)
Discussion started by: pchang
2 Replies

LEARN ABOUT FREEBSD

hx509_peer_info_alloc

hx509 certificate selecting functions(3)			Heimdalx509library			  hx509 certificate selecting functions(3)

NAME

       hx509 certificate selecting functions -

   Functions
       int hx509_peer_info_alloc (hx509_context context, hx509_peer_info *peer)
       void hx509_peer_info_free (hx509_peer_info peer)
       int hx509_peer_info_set_cert (hx509_peer_info peer, hx509_cert cert)
       int hx509_peer_info_add_cms_alg (hx509_context context, hx509_peer_info peer, const AlgorithmIdentifier *val)
       int hx509_peer_info_set_cms_algs (hx509_context context, hx509_peer_info peer, const AlgorithmIdentifier *val, size_t len)

Detailed Description
Function Documentation
   int hx509_peer_info_add_cms_alg (hx509_context context, hx509_peer_info peer, const AlgorithmIdentifier * val)
       Add an additional algorithm that the peer supports.

       Parameters:
	   context A hx509 context.
	   peer the peer to set the new algorithms for
	   val an AlgorithmsIdentier to add

       Returns:
	   An hx509 error code, see hx509_get_error_string().

   int hx509_peer_info_alloc (hx509_context context, hx509_peer_info * peer)
       Allocate a new peer info structure an init it to default values.

       Parameters:
	   context A hx509 context.
	   peer return an allocated peer, free with hx509_peer_info_free().

       Returns:
	   An hx509 error code, see hx509_get_error_string().

   void hx509_peer_info_free (hx509_peer_info peer)
       Free a peer info structure.

       Parameters:
	   peer peer info to be freed.

   int hx509_peer_info_set_cert (hx509_peer_info peer, hx509_cert cert)
       Set the certificate that remote peer is using.

       Parameters:
	   peer peer info to update
	   cert cerificate of the remote peer.

       Returns:
	   An hx509 error code, see hx509_get_error_string().

   int hx509_peer_info_set_cms_algs (hx509_context context, hx509_peer_info peer, const AlgorithmIdentifier * val, size_t len)
       Set the algorithms that the peer supports.

       Parameters:
	   context A hx509 context.
	   peer the peer to set the new algorithms for
	   val array of supported AlgorithmsIdentiers
	   len length of array val.

       Returns:
	   An hx509 error code, see hx509_get_error_string().

Version 1.5.2							    11 Jan 2012 			  hx509 certificate selecting functions(3)
All times are GMT -4. The time now is 03:45 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy