Sponsored Content
Top Forums Shell Programming and Scripting Delete unique rows - optimize script Post 302701425 by pamu on Sunday 16th of September 2012 03:41:04 AM
Old 09-16-2012
Code:
[4] A response
[1] C request
[1] C response
[3] D request
[2] C request
[2] C response
[5] E request

I think you want all the response items as per your request?????

Code:
#/bin/bash
req=$(mktemp)
res=$(mktemp)
new=$(mktemp)
tmp=$(mktemp)
grep request  $1 | sort | uniq > $req   #to get unique request items.
grep response $1 > $res
for id in "$(cat $req | awk '{ print $1,$2}')" # Get whole part as - [1] C,[2] C -- Makes it more unique
do    
    #id=$(echo $id | tr -d "[]")   # I think its better to search with [1] C to get more unique results.. that's why commented this.
    grep "$id" $res > $tmp 
    if [[ -s $tmp ]]
    then
 grep "$id" $req >> $new
 cat $tmp >> $new 
    fi
done
mv $new $2
rm $req $res $tmp


Last edited by pamu; 09-16-2012 at 04:58 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

optimize the script

Hi, I have this following script below. Its searching a log file for 2 string and if found then write the strings to success.txt and If not found write strings to failed.txt . if one found and not other...then write found to success.txt and not found to failed.txt. I want to optimize this... (3 Replies)
Discussion started by: amitrajvarma
3 Replies

2. Shell Programming and Scripting

Script to delete older versions of unique files

I have directory where new sub directories and files being created every few minutes. The directories are like abc_date, def_date, ghi_date. I am looking to keep the latest 2 unique directories and delete everything else. Here is what I have so far This gives me unique names excluding the... (5 Replies)
Discussion started by: zzstore
5 Replies

3. Shell Programming and Scripting

Shell script to count unique rows in a CSV

HI All, I have a CSV file of 30 columns separated by ,. I want to get a count of all unique rows written to a flat file. The CSV file is around 5000 rows The first column is a time stamp and I need to exclude while counting unique Thanks, Ravi (4 Replies)
Discussion started by: Nani369
4 Replies

4. UNIX for Dummies Questions & Answers

Delete rows with unique value for specific column

Hi all I have a file which looks like this 1234|1|Jon|some text|some text 1234|2|Jon|some text|some text 3453|5|Jon|some text|some text 6533|2|Kate|some text|some text 4567|3|Chris|some text|some text 4567|4|Maggie|some text|some text 8764|6|Maggie|some text|some text My third column is my... (9 Replies)
Discussion started by: A-V
9 Replies

5. UNIX for Dummies Questions & Answers

Extract unique combination of rows from text files

Hi Gurus, I have 100 tab-delimited text files each with 21 columns. I want to extract only 2nd and 5th column from each text file. However, the values in both 2bd and 5th column contain duplicate values but the combination of these values in a row are not duplicate. I want to extract only those... (3 Replies)
Discussion started by: Unilearn
3 Replies

6. UNIX and Linux Applications

Script to delete few rows from a file and then update header

HJKL1Name00014300010800000418828124201 L201207022012070228XAM 00000000031795404 001372339540000000000000000000000 COOLTV KEYA Zx00 xI-50352202553 00000000 00000000 G000000000000 00000000 ... (10 Replies)
Discussion started by: mirwasim
10 Replies

7. Shell Programming and Scripting

Unique extraction of rows

I do have a tab delimited file of the following format: 431 kat1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 432 kat2 2 NA NA NA NA NA NA NA NA NA NA NA NA NA 433 KATe NA 3 NA NA 6 NA NA NA 10 11 NA NA NA NA 542 Kaed 2 NA NA NA NA NA NA NA NA NA NA NA NA NA 543 hkwuy NA NA NA NA 6 NA NA NA NA 11 NA NA... (11 Replies)
Discussion started by: Kanja
11 Replies

8. Shell Programming and Scripting

Script to delete rows in a file

Hi All, I am new to UNIX . Please help me in writing code to delete all records from the file where all columns after cloumn 5 in file is either 0, #MI or NULL. Initial 5 columns are string e.g. "alsod" "1FEV2" "wjwroe" " wsse" "hd3" 1 2 34 #Mi "malasl" "wses" "trwwwe" " wsse" "hd3" 1 2 0... (4 Replies)
Discussion started by: alok2082
4 Replies

9. UNIX for Dummies Questions & Answers

Removing rows that contain non-unique column entry

Background: I have a file of thousands of potential SSR primers from Batch Primer 3. I can't use primers that will contain the same sequence ID or sequence as another primer. I have some basic shell scripting skills, but not enough to handle this. What you need to know: I need to remove the... (1 Reply)
Discussion started by: msatseqs
1 Replies

10. Shell Programming and Scripting

Help Optimize the Script Further

Hi All, I have written a new script to check for DB space and size of dump log file before it can be imported into a Oracle DB. I'm relatively new to shell scripting. Please help me optimize this script further. (0 Replies)
Discussion started by: narayanv
0 Replies
DB2_STATISTICS(3)							 1							 DB2_STATISTICS(3)

db2_statistics - Returns a result set listing the index and statistics for a table

SYNOPSIS
resource db2_statistics (resource $connection, string $qualifier, string $schema, string $table-name, bool $unique) DESCRIPTION
Returns a result set listing the index and statistics for a table. PARAMETERS
o $connection - A valid connection to an IBM DB2, Cloudscape, or Apache Derby database. o $qualifier - A qualifier for DB2 databases running on OS/390 or z/OS servers. For other databases, pass NULL or an empty string. o $schema - The schema that contains the targeted table. If this parameter is NULL, the statistics and indexes are returned for the schema of the current user. o $table_name - The name of the table. o $unique - An integer value representing the type of index information to return. o $0 - Return only the information for unique indexes on the table. o $1 - Return the information for all indexes on the table. RETURN VALUES
Returns a statement resource with a result set containing rows describing the statistics and indexes for the base tables matching the specified parameters. The rows are composed of the following columns: +--------------+---------------------------------------------------+ | Column name | | | | | | | Description | | | | +--------------+---------------------------------------------------+ | TABLE_CAT | | | | | | | The catalog that contains the table. The value is | | | NULL if this table does not have catalogs. | | | | | TABLE_SCHEM | | | | | | | Name of the schema that contains the table. | | | | | TABLE_NAME | | | | | | | Name of the table. | | | | | NON_UNIQUE | | | | | | | An integer value representing whether the index | | | prohibits unique values, or whether the row rep- | | | resents statistics on the table itself: | | | | | | box, tab (|); c | c | . T{ Return value | | | | | | Parameter type | | | | +--------------+---------------------------------------------------+ |0 (SQL_FALSE) | | | | | | | The index allows duplicate values. | | | | |1 (SQL_TRUE) | | | | | | | The index values must be unique. | | | | | | | | NULL | | | | | | | This row is statistics information for the table | | | itself. | | | | +--------------+---------------------------------------------------+ T} T{ INDEX_QUALIFIER T} |T{ A string value representing the qualifier that would have to be prepended to INDEX_NAME to fully qualify the index. T} T{ INDEX_NAME T} |T{ A string representing the name of the index. T} T{ TYPE T} |T{ An integer value representing the type of information contained in this row of the result set: +------------------------+---------------------------------------------------+ | Return value | | | | | | | Parameter type | | | | +------------------------+---------------------------------------------------+ | 0 (SQL_TABLE_STAT) | | | | | | | The row contains statistics about the table | | | itself. | | | | |1 (SQL_INDEX_CLUSTERED) | | | | | | | The row contains information about a clustered | | | index. | | | | | 2 (SQL_INDEX_HASH) | | | | | | | The row contains information about a hashed | | | index. | | | | | 3 (SQL_INDEX_OTHER) | | | | | | | The row contains information about a type of | | | index that is neither clustered nor hashed. | | | | +------------------------+---------------------------------------------------+ T} T{ ORDINAL_POSITION T} |T{ The 1-indexed position of the column in the index. NULL if the row contains statistics information about the table itself. T} T{ COLUMN_NAME T} |T{ The name of the column in the index. NULL if the row contains statistics information about the table itself. T} T{ ASC_OR_DESC T} |T{ A if the column is sorted in ascending order, D if the column is sorted in descending order, NULL if the row contains statistics informa- tion about the table itself. T} T{ CARDINALITY T} |T{ If the row contains information about an index, this column contains an integer value representing the number of unique values in the index. If the row contains information about the table itself, this column contains an integer value representing the number of rows in the table. T} T{ PAGES T} |T{ If the row contains information about an index, this column contains an integer value representing the number of pages used to store the index. If the row contains information about the table itself, this column contains an integer value representing the number of pages used to store the table. T} T{ FILTER_CONDITION T} |T{ Always returns NULL. T} SEE ALSO
db2_column_privileges(3), db2_columns(3), db2_foreign_keys(3), db2_primary_keys(3), db2_procedure_columns(3), db2_procedures(3), db2_spe- cial_columns(3), db2_table_privileges(3), db2_tables(3). PHP Documentation Group DB2_STATISTICS(3)
All times are GMT -4. The time now is 10:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy