05-25-2004
filter based on column value
I have a file with colon separated values.. the sample is attached below.
No of fields in each record/line is dependent on the value of field53.
What I need to do is to design a special filter based on specific requirement of some match of values in particular column or combination of columns.
I give an example..
my query may demand me to extarct all the lines that match the criterion
field7=2230982599 AND field8=1232600248
I want the field no and the matching pattern to be variable...
Hope I have clearly stated the problem.
pART OF THE FILE LOOKS LIKE
PACE | PCMD | 21.0 | 040507 | 18 |
3;10115950;611261;106440;3;8d2d9522;2230982599;1232600248;0;65;2;1;43;65;2;1;43;
1641;65;7;0;23811450;2;2;2;2;2;1;0;3;3;994;518;7;0;2;2;1;1;0;140;0;0;0;4;3;0;0;0
;0;0;0;2;25;27;
3;10115951;612309;1100;1;8d31de39;2230983737;1232602026;0;33;2;1;34;33;2;1;34;0;
0;0;0;982014290;2;2;2;0;2;1;0;3;3;995;823;7;0;2;0;1;2;0;56;0;0;0;0;0;0;0;0;0;0;0
;1;25;
3;10115952;612347;0;59;0;2233498694;0;2233498694;0;0;0;0;0;0;0;0;1885;230;7;0;0;
0;0;0;0;0;0;0;0;0;0;0;0;0;0;0;1;8;0;0;0;0;0;0;0;0;0;0;0;0;4802828128;0;
3;10115953;611938;40797;3;44f2a266;2234358432;1075033834;0;12;2;2;2;12;2;2;2;617
;69;7;1;02234204762;2;2;1;1;1;1;0;3;3;993;894;7;0;2;2;1;1;0;84;0;0;0;3;3;0;0;0;0
;0;0;2;25;27;
................
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I want to accomplish this in sh, however if the capability exists only in other shells elsewhere that's acceptable.
% df -kh
Filesystem size used avail capacity Mounted on
...
/dev/dsk/c0t0d0s1 103G 102G 23M 100% /export/DISK15
...
# output... (5 Replies)
Discussion started by: ProGrammar
5 Replies
2. Shell Programming and Scripting
Hi all
This is my output of the some SQL Query
TABLESPACE_NAME FILE_NAME TOTALSPACE FREESPACE USEDSPACE Free
------------------------- ------------------------------------------------------- ---------- --------- ---------... (2 Replies)
Discussion started by: jhon
2 Replies
3. Shell Programming and Scripting
Hi,
I have a requirement to display/write the 3rd column from a file based on the value in the column 3.
Ex: Data in the File (comma delimited)
ID,Value,Description
1,A,Active
1,I,Inactive
2,S,Started
1,N,None
2,C,Completed
2,F,Failed
I need to first get a list of all Unique IDs in... (7 Replies)
Discussion started by: kiranredz
7 Replies
4. Linux
I have a .CSV file with the below format:
"column 1","column 2","column 3","column 4","column 5","column 6","column 7","column 8","column 9","column 10
"12310","42324564756","a simple string with a , comma","string with or, without commas","string 1","USD","12","70%","08/01/2013",""... (2 Replies)
Discussion started by: dhruuv369
2 Replies
5. Shell Programming and Scripting
I have a file from which I need to filter out certain lines when field 17 is less than sysdate. The file has date in YYYYMMDD HH:MI:SS format.
Sample file is as below:
PRUM,67016800 ,CC ,C1,67016800 ,00,Y,Y,2 ,US,BX,BOX ... (5 Replies)
Discussion started by: mady135
5 Replies
6. Shell Programming and Scripting
Hi,
I have a file (stats.txt) with columns like in the example below. Destination IP address, timestamp, TCP packet sequence number and packet length.
destIP time seqNo packetLength
1.2.3.4 0.01 123 500
1.2.3.5 0.03 44 1500
1.3.2.5 0.08 44 1500
1.2.3.4 0.44... (12 Replies)
Discussion started by: Zooma
12 Replies
7. Shell Programming and Scripting
Hi All,
I am trying to select the rows in a fixed width file based on values in the columns.
I want to select only the rows if column position 3-4 has the value AB
I am using cut command to get the column values. Is it possible to check if cut -c3-4 = AB is true then select only that... (2 Replies)
Discussion started by: ashok.k
2 Replies
8. Shell Programming and Scripting
Hello Team,
Need your expertise on following:
Here is the set of data:
C1|4|C1SP1|A1|C1BP1|T1
C1|4|C1SP2|A1|C1BP2|T2
C2|3|C2SP1|A2|C2BP1|T2
C3|3|C3SP1|A3|C3BP1|T2
C2|2|C2SP2|A2|C2BP2|T1
I need to filter above date base on following two steps:
1. Group them by column 1 and 4
2.... (12 Replies)
Discussion started by: angshuman
12 Replies
9. Shell Programming and Scripting
Hello,
I have a question on how to filter the data on multiple columns. The problem is I have one table with 25 columns, 4500 rows. I need to filter out the data based on some codes like 'XXXX', I have 25 codes to check on that table. My requirement is that which ever row has this code I... (1 Reply)
Discussion started by: sandeep309
1 Replies
10. Shell Programming and Scripting
Hello
I have a tab text file with many columns and have to filter rows ONLY if column 22 has the value of '0', '1', '2' or '3' (out of 0-5).
If Column 22 has value '0','1', '2' or '3' (highlighted below), then remove anything less than 10 and greater 100 (based on column 5) AND remove anything... (1 Reply)
Discussion started by: nans
1 Replies
LEARN ABOUT DEBIAN
dbs_update
DBS_UPDATE(1p) User Contributed Perl Documentation DBS_UPDATE(1p)
NAME
dbs_update - Update SQL Databases
DESCRIPTION
dbs_update is an utility to update SQL databases from text files.
FORMAT OF THE TEXT FILES
dbs_update assumes that each line of the input contains a data record and that the field within the records are separated by tabulators.
You can tell dbs_update about the input format with the --format option.
The first field of the data record is used as table specification. These consists of the table name and optionally the index of starting
column, separated by a dot.
Alternatively dbs_update can read the column names from the first line of input (see the -h/--headline option). These can even be aliases
for the real column names (see the -m/--map option).
COMMAND LINE PARAMETERS
Required command line parameters are the DBI driver ("Pg" for Postgres or "mysql" for MySQL) and the database name. The third parameter is
optionally and specifies the database user and/or the host where the database resides ("racke", "racke@linuxia.de" or "@linuxia.de").
OPTIONS
--cleanse
Removes all records which remain unaffected from the update process. The same result as deleting all records from the table first and then
running dbs_update, but the table is not empty in the meantime.
-c COLUMN,COLUMN,..., --columns=COLUMN,COLUMN,...
Update only the table columns given by the COLUMN parameters. To exclude columns from the update prepend "!" or "^" to the parameters.
--rows=ROW,ROW,...
Update only the input rows given by the ROW parameters. The first row is 1 where headlines doesn't count. To exclude rows from the update
prepend "!" or "^" to the parameters.
-f FILE, --file=FILE
Reads records from file FILE instead of from standard input.
--format=FORMAT[SEPCHAR]
Assumes FORMAT as format for the input. Only CSV can be specified for now, default is TAB. The default field separator for CSV is a comma,
you may change this by appending the separator to the format.
-h, --headline
Reads the column names from the first line of the input instead of dedicting them from the database layout. Requires the -t/--table option.
-k COUNT, -k KEY,KEY,..., --keys=COUNT, --keys=KEY,KEY,...
Specifies the keys for the table(s) either as the number of columns used as keys or by specifying them explicitly as comma separated argu-
ments to the option. This is used for the detection of existing records.
-m ALIASDEF, --map=ALIASDEF
Maps the names found in the first line of input to the actual column names in the database. The alias and the column name are separated
with "=" signs and the different entries are separated by ";" signs, e.g. "Art-No.=code;Short Description=shortdescr'".
--map-filter=FILTER
Applies a filter to the column names read from the input file. Currently there is only the "lc" filter available.
--match-sql=FIELD:{STATEMENT}
Updates only records where the value of the column FIELD is in the result set of the SQL statement STATEMENT, e.g. "category:{select dis-
tinct name from categories}".
-o, --update-only
Updates existing database entries only, stops if it detects new ones.
-r ROUTINE, --routine=ROUTINE
Applies ROUTINE to any data record. ROUTINE must be a subroutine. dbs_update passes the table name and a hash reference to this subrou-
tine. The keys of the hash are the column names and the values are the corresponding field values. If the return value of ROUTINE is not a
truth value, the data record will be skipped.
"sub {my ($table, $valref) = @_;
unless (defined $$valref{country} && $$valref{country} !~ /S/) {
$$valref{country} = "Germany";
}
1; }"
--skipbadlines
Lines not matching the assumed format are ignored. Without this option, dbs_update simply stops.
-t TABLE, --table=TABLE
Uses TABLE as table name for all records instead of the first field name.
AUTHOR
Stefan Hornburg (Racke), racke@linuxia.de
SEE ALSO
perl(1), DBIx::Easy(3)
perl v5.8.8 2007-02-01 DBS_UPDATE(1p)