Sponsored Content
Full Discussion: filter based on column value
Top Forums Shell Programming and Scripting filter based on column value Post 51612 by rraajjiibb on Tuesday 25th of May 2004 07:40:21 AM
Old 05-25-2004
filter based on column value

I have a file with colon separated values.. the sample is attached below.
No of fields in each record/line is dependent on the value of field53.
What I need to do is to design a special filter based on specific requirement of some match of values in particular column or combination of columns.

I give an example..

my query may demand me to extarct all the lines that match the criterion

field7=2230982599 AND field8=1232600248

I want the field no and the matching pattern to be variable...

Hope I have clearly stated the problem.

pART OF THE FILE LOOKS LIKE
PACE | PCMD | 21.0 | 040507 | 18 |
3;10115950;611261;106440;3;8d2d9522;2230982599;1232600248;0;65;2;1;43;65;2;1;43;
1641;65;7;0;23811450;2;2;2;2;2;1;0;3;3;994;518;7;0;2;2;1;1;0;140;0;0;0;4;3;0;0;0
;0;0;0;2;25;27;
3;10115951;612309;1100;1;8d31de39;2230983737;1232602026;0;33;2;1;34;33;2;1;34;0;
0;0;0;982014290;2;2;2;0;2;1;0;3;3;995;823;7;0;2;0;1;2;0;56;0;0;0;0;0;0;0;0;0;0;0
;1;25;
3;10115952;612347;0;59;0;2233498694;0;2233498694;0;0;0;0;0;0;0;0;1885;230;7;0;0;
0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;1;8;0;0;0;0;0;0;0;0;0;0;0;0;4802828128;0;
3;10115953;611938;40797;3;44f2a266;2234358432;1075033834;0;12;2;2;2;12;2;2;2;617
;69;7;1;02234204762;2;2;1;1;1;1;0;3;3;993;894;7;0;2;2;1;1;0;84;0;0;0;3;3;0;0;0;0
;0;0;2;25;27;

................
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

intent: df -kh | filter based on capacity (used space) column where % > 85

I want to accomplish this in sh, however if the capability exists only in other shells elsewhere that's acceptable. % df -kh Filesystem size used avail capacity Mounted on ... /dev/dsk/c0t0d0s1 103G 102G 23M 100% /export/DISK15 ... # output... (5 Replies)
Discussion started by: ProGrammar
5 Replies

2. Shell Programming and Scripting

Filter the column and print the result based on condition

Hi all This is my output of the some SQL Query TABLESPACE_NAME FILE_NAME TOTALSPACE FREESPACE USEDSPACE Free ------------------------- ------------------------------------------------------- ---------- --------- ---------... (2 Replies)
Discussion started by: jhon
2 Replies

3. Shell Programming and Scripting

awk filter based on column value (variable value)

Hi, I have a requirement to display/write the 3rd column from a file based on the value in the column 3. Ex: Data in the File (comma delimited) ID,Value,Description 1,A,Active 1,I,Inactive 2,S,Started 1,N,None 2,C,Completed 2,F,Failed I need to first get a list of all Unique IDs in... (7 Replies)
Discussion started by: kiranredz
7 Replies

4. Linux

Filter a .CSV file based on the 5th column values

I have a .CSV file with the below format: "column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10 "12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Discussion started by: dhruuv369
2 Replies

5. Shell Programming and Scripting

Need to filter data based on sysdate

I have a file from which I need to filter out certain lines when field 17 is less than sysdate. The file has date in YYYYMMDD HH:MI:SS format. Sample file is as below: PRUM,67016800 ,CC ,C1,67016800 ,00,Y,Y,2 ,US,BX,BOX ... (5 Replies)
Discussion started by: mady135
5 Replies

6. Shell Programming and Scripting

Filter on one column and then perform conditional calculations on another column with a Linux script

Hi, I have a file (stats.txt) with columns like in the example below. Destination IP address, timestamp, TCP packet sequence number and packet length. destIP time seqNo packetLength 1.2.3.4 0.01 123 500 1.2.3.5 0.03 44 1500 1.3.2.5 0.08 44 1500 1.2.3.4 0.44... (12 Replies)
Discussion started by: Zooma
12 Replies

7. Shell Programming and Scripting

UNIX command -Filter rows in fixed width file based on column values

Hi All, I am trying to select the rows in a fixed width file based on values in the columns. I want to select only the rows if column position 3-4 has the value AB I am using cut command to get the column values. Is it possible to check if cut -c3-4 = AB is true then select only that... (2 Replies)
Discussion started by: ashok.k
2 Replies

8. Shell Programming and Scripting

Filter Row Based On Max Column Value After Group BY

Hello Team, Need your expertise on following: Here is the set of data: C1|4|C1SP1|A1|C1BP1|T1 C1|4|C1SP2|A1|C1BP2|T2 C2|3|C2SP1|A2|C2BP1|T2 C3|3|C3SP1|A3|C3BP1|T2 C2|2|C2SP2|A2|C2BP2|T1 I need to filter above date base on following two steps: 1. Group them by column 1 and 4 2.... (12 Replies)
Discussion started by: angshuman
12 Replies

9. Shell Programming and Scripting

Filter Data based on codes

Hello, I have a question on how to filter the data on multiple columns. The problem is I have one table with 25 columns, 4500 rows. I need to filter out the data based on some codes like 'XXXX', I have 25 codes to check on that table. My requirement is that which ever row has this code I... (1 Reply)
Discussion started by: sandeep309
1 Replies

10. Shell Programming and Scripting

Filter tab file based on column value

Hello I have a tab text file with many columns and have to filter rows ONLY if column 22 has the value of '0', '1', '2' or '3' (out of 0-5). If Column 22 has value '0','1', '2' or '3' (highlighted below), then remove anything less than 10 and greater 100 (based on column 5) AND remove anything... (1 Reply)
Discussion started by: nans
1 Replies
DBS_UPDATE(1p)						User Contributed Perl Documentation					    DBS_UPDATE(1p)

NAME
dbs_update - Update SQL Databases DESCRIPTION
dbs_update is an utility to update SQL databases from text files. FORMAT OF THE TEXT FILES dbs_update assumes that each line of the input contains a data record and that the field within the records are separated by tabulators. You can tell dbs_update about the input format with the --format option. The first field of the data record is used as table specification. These consists of the table name and optionally the index of starting column, separated by a dot. Alternatively dbs_update can read the column names from the first line of input (see the -h/--headline option). These can even be aliases for the real column names (see the -m/--map option). COMMAND LINE PARAMETERS
Required command line parameters are the DBI driver ("Pg" for Postgres or "mysql" for MySQL) and the database name. The third parameter is optionally and specifies the database user and/or the host where the database resides ("racke", "racke@linuxia.de" or "@linuxia.de"). OPTIONS
--cleanse Removes all records which remain unaffected from the update process. The same result as deleting all records from the table first and then running dbs_update, but the table is not empty in the meantime. -c COLUMN,COLUMN,..., --columns=COLUMN,COLUMN,... Update only the table columns given by the COLUMN parameters. To exclude columns from the update prepend "!" or "^" to the parameters. --rows=ROW,ROW,... Update only the input rows given by the ROW parameters. The first row is 1 where headlines doesn't count. To exclude rows from the update prepend "!" or "^" to the parameters. -f FILE, --file=FILE Reads records from file FILE instead of from standard input. --format=FORMAT[SEPCHAR] Assumes FORMAT as format for the input. Only CSV can be specified for now, default is TAB. The default field separator for CSV is a comma, you may change this by appending the separator to the format. -h, --headline Reads the column names from the first line of the input instead of dedicting them from the database layout. Requires the -t/--table option. -k COUNT, -k KEY,KEY,..., --keys=COUNT, --keys=KEY,KEY,... Specifies the keys for the table(s) either as the number of columns used as keys or by specifying them explicitly as comma separated argu- ments to the option. This is used for the detection of existing records. -m ALIASDEF, --map=ALIASDEF Maps the names found in the first line of input to the actual column names in the database. The alias and the column name are separated with "=" signs and the different entries are separated by ";" signs, e.g. "Art-No.=code;Short Description=shortdescr'". --map-filter=FILTER Applies a filter to the column names read from the input file. Currently there is only the "lc" filter available. --match-sql=FIELD:{STATEMENT} Updates only records where the value of the column FIELD is in the result set of the SQL statement STATEMENT, e.g. "category:{select dis- tinct name from categories}". -o, --update-only Updates existing database entries only, stops if it detects new ones. -r ROUTINE, --routine=ROUTINE Applies ROUTINE to any data record. ROUTINE must be a subroutine. dbs_update passes the table name and a hash reference to this subrou- tine. The keys of the hash are the column names and the values are the corresponding field values. If the return value of ROUTINE is not a truth value, the data record will be skipped. "sub {my ($table, $valref) = @_; unless (defined $$valref{country} && $$valref{country} !~ /S/) { $$valref{country} = "Germany"; } 1; }" --skipbadlines Lines not matching the assumed format are ignored. Without this option, dbs_update simply stops. -t TABLE, --table=TABLE Uses TABLE as table name for all records instead of the first field name. AUTHOR
Stefan Hornburg (Racke), racke@linuxia.de SEE ALSO
perl(1), DBIx::Easy(3) perl v5.8.8 2007-02-01 DBS_UPDATE(1p)
All times are GMT -4. The time now is 05:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy