Normalize text field or sort values


 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Normalize text field or sort values
# 1  
Old 04-20-2020
1. Could you please clarify?
injest => ingest
normalize => sort ?

2. Could you provide a short example of your input data?

3. Could you provide your script trial?
If you run it on the example input data, would goes wrong?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

awk GSUB read field values from multiple text files

My program run without error. The problem I am having. The program isn't outputting field values with the column headers to file.txt. Each of the column headers in file.txt has no data. MEMSIZE SECOND SASFoundation Filename The output results in file.txt should show: ... (1 Reply)
Discussion started by: dellanicholson
1 Replies

2. Shell Programming and Scripting

awk to parse field and include the text of 1 pipe in field 4

I am trying to parse the input in awk to include the |gc= in $4 but am not able to. The below is close: awk so far: awk '{sub(/\|]+]++/, ""); print }' input.txt Input chr1 955543 955763 AGRN-6|pr=2|gc=75 0 + chr1 957571 957852 AGRN-7|pr=3|gc=61.2 0 + chr1 970621 ... (7 Replies)
Discussion started by: cmccabe
7 Replies

3. Shell Programming and Scripting

Sort file by field 1 that has text as well as a number

I am using the below awk that results in the below output: awk '{k=$1 OFS $2; s+=$4; c++} END{for(i in s) print i, s/c}' input.txt > output.txt output.txt chr20:43625799-43625957 STK4:exon.6;STK4:exon.7 310.703 chr20:36770455-36770611 TGM2:exon.6;TGM2:exon.7 614.756... (5 Replies)
Discussion started by: cmccabe
5 Replies

4. Shell Programming and Scripting

Read record from the text file contain multiple separated values & assign those values to variables

I have a file containing multiple values, some of them are pipe separated which are to be read as separate values and some of them are single value all are these need to store in variables. I need to read this file which is an input to my script Config.txt file name, first path, second... (7 Replies)
Discussion started by: ketanraut
7 Replies

5. Shell Programming and Scripting

Awk Search text string in field, not all in field.

Hello, I am using awk to match text in a tab separated field and am able to do so when matching the exact word. My problem is that I would like to match any sequence of text in the tab-separated field without having to match it all. Any help will be appreciated. Please see the code below. awk... (3 Replies)
Discussion started by: rocket_dog
3 Replies

6. Shell Programming and Scripting

awk, comma as field separator and text inside double quotes as a field.

Hi, all I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes. sample input: for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies

7. Shell Programming and Scripting

adding field values if field matches

hi i have file as below , i want to add duplicate records like bell_bb to one record with valuve as 15 ( addition of both ) any oneline awk script to achive this ? header 0 CAMPAIGN_NAME 1 Bell_BB 14 Bell_MONTHLY 803 SOLO_UNBEATABLE 644 Bell_BB 1 Bell_MONTHLY 25 SOLO_UNBEATABLE... (4 Replies)
Discussion started by: raghavendra.cse
4 Replies

8. Shell Programming and Scripting

Sort alpha on 1st field, numerical on 2nd field (sci notation)

I want to sort alphabetically on the first field and sort in descending numerical order on the 2nd field. With a normal "sort -r -n" it does this: abc ||| 5e-05 ||| bla abc ||| 3 ||| ble def ||| 1 ||| abc def ||| 0.2 ||| def As you can see it ignores the fact that 5e-05 is actually 0.00005... (1 Reply)
Discussion started by: FrancoisCN
1 Replies

9. Shell Programming and Scripting

Find top N values for field X based on field Y's value

I want to find the top N entries for a certain field based on the values of another field. For example if N=3, we want the 3 best values for each entry: Entry1 ||| 100 Entry1 ||| 95 Entry1 ||| 30 Entry1 ||| 80 Entry1 ||| 50 Entry2 ||| 40 Entry2 ||| 20 Entry2 ||| 10 Entry2 ||| 50... (1 Reply)
Discussion started by: FrancoisCN
1 Replies

10. Shell Programming and Scripting

How to sort a field in a file having date values

Hi All, I am having a pipe delimited file .In this file the 3rd column is having date values.I need to get the min date and max date from that file. I have used cut -d '|' test.dat -f 3|sort -u But it is not sorting the date .How to sort the date column using unix commands Thanks ... (4 Replies)
Discussion started by: risshanth
4 Replies
Login or Register to Ask a Question
DBI::ProfileData(3)					User Contributed Perl Documentation				       DBI::ProfileData(3)

NAME
DBI::ProfileData - manipulate DBI::ProfileDumper data dumps SYNOPSIS
The easiest way to use this module is through the dbiprof frontend (see dbiprof for details): dbiprof --number 15 --sort count This module can also be used to roll your own profile analysis: # load data from dbi.prof $prof = DBI::ProfileData->new(File => "dbi.prof"); # get a count of the records in the data set $count = $prof->count(); # sort by longest overall time $prof->sort(field => "longest"); # sort by longest overall time, least to greatest $prof->sort(field => "longest", reverse => 1); # exclude records with key2 eq 'disconnect' $prof->exclude(key2 => 'disconnect'); # exclude records with key1 matching /^UPDATE/i $prof->exclude(key1 => qr/^UPDATE/i); # remove all records except those where key1 matches /^SELECT/i $prof->match(key1 => qr/^SELECT/i); # produce a formatted report with the given number of items $report = $prof->report(number => 10); # clone the profile data set $clone = $prof->clone(); # get access to hash of header values $header = $prof->header(); # get access to sorted array of nodes $nodes = $prof->nodes(); # format a single node in the same style as report() $text = $prof->format($nodes->[0]); # get access to Data hash in DBI::Profile format $Data = $prof->Data(); DESCRIPTION
This module offers the ability to read, manipulate and format DBI::ProfileDumper profile data. Conceptually, a profile consists of a series of records, or nodes, each of each has a set of statistics and set of keys. Each record must have a unique set of keys, but there is no requirement that every record have the same number of keys. METHODS
The following methods are supported by DBI::ProfileData objects. $prof = DBI::ProfileData->new(File => "dbi.prof") $prof = DBI::ProfileData->new(Files => [ "dbi.prof.1", "dbi.prof.2" ]) Creates a a new DBI::ProfileData object. Takes either a single file through the File option or a list of Files in an array ref. If multiple files are specified then the header data from the first file is used. $copy = $prof->clone(); Clone a profile data set creating a new object. $header = $prof->header(); Returns a reference to a hash of header values. These are the key value pairs included in the header section of the DBI::ProfileDumper data format. For example: $header = { Path => '[ DBIprofile_Statement, DBIprofile_MethodName ]', Program => 't/42profile_data.t', }; Note that modifying this hash will modify the header data stored inside the profile object. $nodes = $prof->nodes() Returns a reference the sorted nodes array. Each element in the array is a single record in the data set. The first seven elements are the same as the elements provided by DBI::Profile. After that each key is in a separate element. For example: $nodes = [ [ 2, # 0, count 0.0312958955764771, # 1, total duration 0.000490069389343262, # 2, first duration 0.000176072120666504, # 3, shortest duration 0.00140702724456787, # 4, longest duration 1023115819.83019, # 5, time of first event 1023115819.86576, # 6, time of last event 'SELECT foo FROM bar' # 7, key1 'execute' # 8, key2 # 6+N, keyN ], # ... ]; Note that modifying this array will modify the node data stored inside the profile object. $count = $prof->count() Returns the number of items in the profile data set. $prof->sort(field => "field") $prof->sort(field => "field", reverse => 1) Sorts data by the given field. Available fields are: longest total count shortest The default sort is greatest to smallest, which is the opposite of the normal Perl meaning. This, however, matches the expected behav- ior of the dbiprof frontend. $count = $prof->exclude(key2 => "disconnect") $count = $prof->exclude(key2 => "disconnect", case_sensitive => 1) $count = $prof->exclude(key1 => qr/^SELECT/i) Removes records from the data set that match the given string or regular expression. This method modifies the data in a permanent fashion - use clone() first to maintain the original data after exclude(). Returns the number of nodes left in the profile data set. $count = $prof->match(key2 => "disconnect") $count = $prof->match(key2 => "disconnect", case_sensitive => 1) $count = $prof->match(key1 => qr/^SELECT/i) Removes records from the data set that do not match the given string or regular expression. This method modifies the data in a perma- nent fashion - use clone() first to maintain the original data after match(). Returns the number of nodes left in the profile data set. $Data = $prof->Data() Returns the same Data hash structure as seen in DBI::Profile. This structure is not sorted. The nodes() structure probably makes more sense for most analysis. $text = $prof->format($nodes->[0]) Formats a single node into a human-readable block of text. $text = $prof->report(number => 10) Produces a report with the given number of items. AUTHOR
Sam Tregar <sam@tregar.com> COPYRIGHT AND LICENSE
Copyright (C) 2002 Sam Tregar This program is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself. perl v5.8.0 2002-12-01 DBI::ProfileData(3)