04-19-2020
Normalize text field or sort values
Hello so I am trying to "normalize" data so that it will upload correctly into a postgreql db.
The problem I am running into is that I formatted my db to injest the data in the order of which it is normalized.
However in certain data sets several fields of information are not available, so when I normalize those with my script it gets thrown off, and the values are not injested into the correct column in the DB.
So the options I have are
1. when I normalize the information, sort it so the first fields are the ones that are always present or
2. When i normalize the information insert the ,, for the missing fields in order to keep everything in the correct column.
Either way I have no idea how to do this and would appreciate any suggestions.
THANK YOU SO MUCH FOR YOUR TIME
Last edited by vbe; 04-20-2020 at 03:06 AM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi All,
I am having a pipe delimited file .In this file the 3rd column is having date values.I need to get the min date and max date from that file.
I have used
cut -d '|' test.dat -f 3|sort -u
But it is not sorting the date .How to sort the date column using unix commands
Thanks ... (4 Replies)
Discussion started by: risshanth
4 Replies
2. Shell Programming and Scripting
I want to find the top N entries for a certain field based on the values of another field.
For example if N=3, we want the 3 best values for each entry:
Entry1 ||| 100
Entry1 ||| 95
Entry1 ||| 30
Entry1 ||| 80
Entry1 ||| 50
Entry2 ||| 40
Entry2 ||| 20
Entry2 ||| 10
Entry2 ||| 50... (1 Reply)
Discussion started by: FrancoisCN
1 Replies
3. Shell Programming and Scripting
I want to sort alphabetically on the first field and sort in descending numerical order on the 2nd field. With a normal "sort -r -n" it does this:
abc ||| 5e-05 ||| bla
abc ||| 3 ||| ble
def ||| 1 ||| abc
def ||| 0.2 ||| def
As you can see it ignores the fact that 5e-05 is actually 0.00005... (1 Reply)
Discussion started by: FrancoisCN
1 Replies
4. Shell Programming and Scripting
hi
i have file as below , i want to add duplicate records like bell_bb to one record with valuve as 15 ( addition of both )
any oneline awk script to achive this ?
header 0
CAMPAIGN_NAME 1
Bell_BB 14
Bell_MONTHLY 803
SOLO_UNBEATABLE 644
Bell_BB 1
Bell_MONTHLY 25
SOLO_UNBEATABLE... (4 Replies)
Discussion started by: raghavendra.cse
4 Replies
5. Shell Programming and Scripting
Hi, all
I need to get fields in a line that are separated by commas, some of the fields are enclosed with double quotes, and they are supposed to be treated as a single field even if there are commas inside the quotes.
sample input:
for this line, 5 fields are supposed to be extracted, they... (8 Replies)
Discussion started by: kevintse
8 Replies
6. Shell Programming and Scripting
Hello, I am using awk to match text in a tab separated field and am able to do so when matching the exact word. My problem is that I would like to match any sequence of text in the tab-separated field without having to match it all. Any help will be appreciated. Please see the code below.
awk... (3 Replies)
Discussion started by: rocket_dog
3 Replies
7. Shell Programming and Scripting
I have a file containing multiple values, some of them are pipe separated which are to be read as separate values and some of them are single value all are these need to store in variables.
I need to read this file which is an input to my script
Config.txt
file name, first path, second... (7 Replies)
Discussion started by: ketanraut
7 Replies
8. Shell Programming and Scripting
I am using the below awk that results in the below output:
awk '{k=$1 OFS $2; s+=$4; c++} END{for(i in s) print i, s/c}' input.txt > output.txt output.txt
chr20:43625799-43625957 STK4:exon.6;STK4:exon.7 310.703
chr20:36770455-36770611 TGM2:exon.6;TGM2:exon.7 614.756... (5 Replies)
Discussion started by: cmccabe
5 Replies
9. Shell Programming and Scripting
I am trying to parse the input in awk to include the |gc= in $4 but am not able to. The below is close:
awk so far:
awk '{sub(/\|]+]++/, ""); print }' input.txt Input
chr1 955543 955763 AGRN-6|pr=2|gc=75 0 +
chr1 957571 957852 AGRN-7|pr=3|gc=61.2 0 +
chr1 970621 ... (7 Replies)
Discussion started by: cmccabe
7 Replies
10. UNIX for Beginners Questions & Answers
My program run without error. The problem I am having.
The program isn't outputting field values with the column headers to file.txt.
Each of the column headers in file.txt has no data.
MEMSIZE SECOND SASFoundation Filename
The output results in file.txt should show:
... (1 Reply)
Discussion started by: dellanicholson
1 Replies
LEARN ABOUT CENTOS
po4a-normalize
PO4A-NORMALIZE(1p) Po4a Tools PO4A-NORMALIZE(1p)
NAME
po4a-normalize - normalize a documentation file by parsing it in po4a, and writing it back
SYNOPSIS
po4a-normalize -f fmt master.doc
DESCRIPTION
The po4a (PO for anything) project goal is to ease translations (and more interestingly, the maintenance of translations) using gettext
tools on areas where they were not expected like documentation.
The po4a-normalize script is a debugging tool used to make sure that po4a don't change the document when it's not supposed to. Only use it
if you're developing a new module, or if you doubt the sanity of the tools.
The generated document will be written to po4a-normalize.output while the generated PO file will be written to po4a-normalize.po. No way to
change that ;)
OPTIONS
-o, --option
Extra option(s) to pass to the format plugin. Specify each option in the 'name=value' format. See the documentation of each plugin for
more information about the valid options and their meanings.
-b, --blank
Create an blank translated document. The generated translated document will be generated assuming all messages are translated by a
space or new line.
This is useful to check what parts of the document cannot be translated.
-h, --help
Show a short help message.
--help-format
List the documentation formats understood by po4a.
-f, --format
Format of the documentation you want to handle. Use the --help-format option to see the list of available formats.
-M, --master-charset
Charset of the file containing the document to translate.
-V, --version
Display the version of the script and exit.
SEE ALSO
po4a-gettextize(1), po4a-translate(1), po4a-updatepo(1), po4a(7)
AUTHORS
Denis Barbier <barbier@linuxfr.org>
Nicolas Francois <nicolas.francois@centraliens.net>
Martin Quinson (mquinson#debian.org)
COPYRIGHT AND LICENSE
Copyright 2002-2012 by SPI, inc.
This program is free software; you may redistribute it and/or modify it under the terms of GPL (see the COPYING file).
Po4a Tools 2014-06-10 PO4A-NORMALIZE(1p)