Sponsored Content
Top Forums Shell Programming and Scripting Issues with filtering duplicate records using gawk script Post 302305849 by vidyadhar85 on Friday 10th of April 2009 12:07:24 AM
Old 04-10-2009
whats the problem you are facing??
1.Trade file will be sorted in ascending order on first two columns(COB date and trade id)
2.Trade file will be sorted in descending order on first two columns(COB date and trade id)
i didn't understand above two condition..
why do you want it to sort it in both ways ??
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Filtering duplicate lines

Does anybody know a command that filters duplicate lines out of a file. Similar to the uniq command but can handle duplicate lines no matter where they occur in a file? (9 Replies)
Discussion started by: AreaMan
9 Replies

2. UNIX for Dummies Questions & Answers

Filtering records of a file based on a value of a column

Hi all, I would like to extract records of a file based on a condition. The file contains 47 fields, and I would like to extract only those records that match a certain value in one of the columns, e.g. COL1 COL2 COL3 ............... COL47 1 XX 45 ... (4 Replies)
Discussion started by: risk_sly
4 Replies

3. Linux

Need awk script for removing duplicate records

I have huge txt file having millions of trade data. For e.g Trade.txt (first 8 lines in the file is header info) COB_DATE,TRADE_ID,SOURCE_SYSTEM_TRADE_ID,TRADE_GROUP_ID, TRADE_TYPE,DEALER_NAME,EXTERNAL_COUNTERPARTY_ID, EXTERNAL_COUNTERPARTY_NAME,DB_COUNTERPARTY_ID,... (6 Replies)
Discussion started by: nmumbarkar
6 Replies

4. Shell Programming and Scripting

Filtering issues with multiple columns in a single file

Hi, I am new to unix and would greatly appreciate some help. I have a file containing multiple colums containing different sets of data e.g. File 1: John Ireland 27_December_69 Mary England 13_March_55 Mike France 02_June_80 I am currently using the awk... (10 Replies)
Discussion started by: crunchie
10 Replies

5. Shell Programming and Scripting

Filtering Issues Using sed and awk

Hi, I am currently using the sed and awk commands to filter a file that has multiple sets of data in different columns. An example of part of the file i am filtering is as follows; Sat Oct 2 07:42:45 2010 01:33:46 R1_CAR_12.34 Sun Oct 3 13:09:53 2010 00:02:34 R2_BUS_56.78 Sun... (4 Replies)
Discussion started by: crunchie
4 Replies

6. Shell Programming and Scripting

Filtering issues while using nawk

Hi, I am currently filtering a file that has multiple sets of data. An example of some of the data is as follows; Sat Oct 2 07:42:45 2010 01:33:46 R1_CAR_12.34 Sun Oct 3 13:09:53 2010 00:02:34 R2_BUS_56.78 Sun Oct 3 21:11:39 2010 00:43:21 R3_TRAIN_COACH_90.12 Mon Oct 4... (1 Reply)
Discussion started by: crunchie
1 Replies

7. Linux

Need awk script for removing duplicate records

I have log file having Traffic line 2011-05-21 15:11:50.356599 TCP (6), length: 52) 10.10.10.1.3020 > 10.10.10.254.50404: 2011-05-21 15:11:50.652739 TCP (6), length: 52) 10.10.10.254.50404 > 10.10.10.1.3020: 2011-05-21 15:11:50.652558 TCP (6), length: 89) 10.10.10.1.3020 >... (1 Reply)
Discussion started by: Rastamed
1 Replies

8. Shell Programming and Scripting

Perl: filtering lines based on duplicate values in a column

Hi I have a file like this. I need to eliminate lines with first column having the same value 10 times. 13 18 1 + chromosome 1, 122638287 AGAGTATGGTCGCGGTTG 13 18 1 + chromosome 1, 128904080 AGAGTATGGTCGCGGTTG 13 18 1 - chromosome 14, 13627938 CAACCGCGACCATACTCT 13 18 1 + chromosome 1,... (5 Replies)
Discussion started by: polsum
5 Replies

9. Shell Programming and Scripting

Need help with filtering records in a file

Hi, I have following records in a file more file1.txt setting applicaction ABC for user setting applicaction CDE for user setting applicaction XXX for user logging applicaction XXX for user I need to filter out records which have strings " setting... (5 Replies)
Discussion started by: manid
5 Replies

10. UNIX for Beginners Questions & Answers

Filtering records of a csv file based on a value of a column

Hi, I tried filtering the records in a csv file using "awk" command listed below. awk -F"~" '$4 ~ /Active/{print }' inputfile > outputfile The output always has all the entries. The same command worked for different users from one of the forum links. content of file I was... (3 Replies)
Discussion started by: sunilmudikonda
3 Replies
SORT-DCTRL(1)						       Debian user's manual						     SORT-DCTRL(1)

NAME
sort-dctrl - sort Debian control files SYNOPSIS
sort-dctrl [options] [ file ... ] sort-dctrl --copying | --help | --version | -ChV DESCRIPTION
The sort-dctrl program sorts Debian control files according to specified criteria. A Debian control (dctrl) file is a semistructured single-table database stored in a machine-parseable text file. Such a database consists of a set of records; each record is a mapping from field names to field content. Textually, records are separated by empty lines, while each field is encoded as one or more nonempty lines inside a record. A field starts with its name, followed by a colon, followed by the field content. The colon must reside on the first line of the field, and the first line must start with no whitespace. Subsequent lines, in contrast, always start with linear whitespace (one or more space or tab characters). The sort-dctrl program recognizes two field types: string fields and version fields. Version fields act also as numeric fields. String fields are compared according to strict lexicographical octet-by-octet comparison, after ignoring any initial whitespace after the colon. Version fields are parsed and compared as Debian version numbers. When comparing version numbers, if a field content does not in fact con- form to the version number syntax, it compares less than any field content that does conform, and equal to any other nonconforming field content. The sort-dctrl program assumes all fields are string fields unless told otherwise. You can specify arbitrary number of keys for sorting, using the -k option. The keys are interpreted in a descending order of priority: the first key specified is primary, the second key specified is secondary, and so on. If two records compare equal under the primary key, then they are compared under the secondary key, and so on. If no keys are specified, a default key is assumed (the "Package" field with no mod- ifiers). OPTIONS
-k keyspec, --key-spec=keyspec Specify one or more keys for sorting. You may specify this option any number of times. The keyspec argument consists of a comma- separated list of key specifications. Each key specification consists of the name of the field that serves as the key, optionally followed by a colon and key modifiers. The following key modifiers are supported: r Invert the comparison for this key, reversing the sorting order. v Treat this field as a version number field. n Treat this field as numeric, which currently is synonymous with v. -q, --quiet, --silent Output nothing to the standard output stream. Instead, exit immediately after finding the first match. -l level, --errorlevel=level Set log level to level. level is one of fatal, important, informational and debug, but the last may not be available, depending on the compile-time options. These categories are given here in order; every message that is emitted when fatal is in effect, will be emitted in the important error level, and so on. The default is important. -V, --version Print out version information. -C, --copying Print out the copyright license. This produces much output; be sure to redirect or pipe it somewhere (such as your favourite pager). -h, --help Print out a help summary. EXAMPLES
Here are some sample invocations of the program: sort-dctrl /var/lib/dpkg/available Output the dpkg available file sorted by the package name. sort-dctrl -k Version:v /var/lib/dpkg/available Output the dpkg available file sorted in ascending order of version numbers. sort-dctrl -k Version:vr /var/lib/dpkg/available Output the dpkg available file sorted in descending order of version numbers. sort-dctrl -k Package,Version:v /var/lib/dpkg/available Output the dpkg available file sorted primarily in ascending order of package names and secondarily in descending order of version numbers. sort-dctrl -k Installed-Size:n,Size:nr /var/lib/dpkg/available Output the dpkg available file sorted primarily in ascending order of installation sizes and secondarily in descending order of package sizes. AUTHOR
The program and this manual page were written by Antti-Juhani Kaijanaho <ajk@debian.org>. SEE ALSO
Debian Policy Manual. Published as the Debian package debian-policy. Also available in the Debian website. grep-dctrl(1) Debian Project 2005-06-08 SORT-DCTRL(1)
All times are GMT -4. The time now is 05:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy