02-21-2020
Hi Neo,
Apologies... Let me try to answer your queries:
Quote:
When building a new system task, we should clearly know the interface; and you have not provided any of the details.
It is an existing system. Current high level architecture:
Source system (Reuters, Bloomberg etc) ---> provide transaction details in some flat files to an internal team in raw format [.dat files] ---> That team apply some transformation logic to generate required files for our application [Infra: Hadoop, Unix] ----> we load these files [Infra: Unix, Oracle] ---> apply ETL transformation logic [Infra: Unix, Oracle] ---> Run rules/logic [Infra: Unix, Oracle] ---> generate alerts [Infra: Unix, Oracle] .
Quote:
Do these number come from a DB query?
Yes, Oracle SQL query (Manual run in Database to check on daily basis. SQL output data provided as a sample csv file in previous post)
.
Now I would like to verify the data volume just before ETL transformation layer.
Please let me know if I'm able to provide required answer.
Cheers,
~ Saps
9 More Discussions You Might Find Interesting
1. HP-UX
HI all,
I've been having issues with printers and plotters disabling with no reason at all...It could happen to the same workstation once a day or once a week..There is nothing consistant..I stop and restart the spooler, only then can I re-enable the disabled printer....
If anyone is... (0 Replies)
Discussion started by: Shutdown
0 Replies
2. Shell Programming and Scripting
I need to zip all the files within a directory on a daily basis and store them as a zip file within that directory with the date in its naming convention
I have file extentions .log .lst and .out's within that directory
How do i do this task
zip -mqj mydir/allzip$YYMMDDDD.zip mydir/*.*
... (5 Replies)
Discussion started by: ramky79
5 Replies
3. AIX
Hi!
Can anyone help me on how I can do a basic check on the Unix filesystems / physical volumes and logical volumes?
What items should I check, like where do I look at in smit? Or are there commands that I should execute?
I need to do this as I was informed by IBM that there seems to be... (1 Reply)
Discussion started by: chipahoys
1 Replies
4. Shell Programming and Scripting
Need help on this script.
the month is not changing to February...
#!/bin/bash
for X in `seq 1 100`
do
DATE=`date +%Y-%m-%d "--date=${X} day ago"`
Y=`date +%Y`
M=`date +%m`
D=`date +%d "--date=${X} day ago"`
DIR=/home/LogBackup
for i in `seq 1 `
do
if ;then
# ... (1 Reply)
Discussion started by: alelee
1 Replies
5. UNIX for Advanced & Expert Users
Hi with the help of Gabriel canepa, i have just edited filename only in his code. The help which i got and he helped is
1) I have around 22 logs and each log should be updated in the last 24 hours from the current timestamp.
2) It should check for ERROR message (not error,Error) in the log and... (2 Replies)
Discussion started by: Kalaihari
2 Replies
6. UNIX for Dummies Questions & Answers
Hi Gurus,
Is there any solution for tracking the script(.sh) files triggered on daily or monthly basis from source - Datastage (Routines)
Needs to find out if this scripts are running on daily
just want to know that is there anything to track
Thanks in Advance (2 Replies)
Discussion started by: SeenuGuddu
2 Replies
7. Shell Programming and Scripting
Hi!
Please I need help on the a script that would pick one file in a directory, change its name, them change its permissions, them move to a different directory, but has to be done on a daily basis, and the file that is being moved to its final destination has to have the following format:... (7 Replies)
Discussion started by: fretagi
7 Replies
8. Shell Programming and Scripting
Hi all
I need to count files on a daily basis and send the output via email, what I am currently doing its simply running the following:ls -lrt *.cdr | awk '{if(($6 == "Jul") && ($7 == "13")) print $0}' | wc -l, which in this case it will count all files with extension "cdr" from the month of... (13 Replies)
Discussion started by: fretagi
13 Replies
9. Programming
What's the difference between BEGIN TRANSACTION vs START TRANSACTION?
Also goes for COMMIT TRANSACTION vs COMMIT? (0 Replies)
Discussion started by: dan139
0 Replies
LEARN ABOUT DEBIAN
sql::translator::parser::oracle
SQL::Translator::Parser::Oracle(3pm) User Contributed Perl Documentation SQL::Translator::Parser::Oracle(3pm)
NAME
SQL::Translator::Parser::Oracle - parser for Oracle
SYNOPSIS
use SQL::Translator;
use SQL::Translator::Parser::Oracle;
my $translator = SQL::Translator->new;
$translator->parser("SQL::Translator::Parser::Oracle");
DESCRIPTION
From http://www.ss64.com/ora/table_c.html:
CREATE [GLOBAL TEMPORARY] TABLE [schema.]table (tbl_defs,...)
[ON COMMIT {DELETE|PRESERVE} ROWS]
[storage_options | CLUSTER cluster_name (col1, col2,... )
| ORGANIZATION {HEAP [storage_options]
| INDEX idx_organized_tbl_clause}]
[LOB_storage_clause][varray_clause][nested_storage_clause]
partitioning_options
[[NO]CACHE] [[NO]MONITORING] [PARALLEL parallel_clause]
[ENABLE enable_clause | DISABLE disable_clause]
[AS subquery]
tbl_defs:
column datatype [DEFAULT expr] [column_constraint(s)]
table_ref_constraint
storage_options:
PCTFREE int
PCTUSED int
INITTRANS int
MAXTRANS int
STORAGE storage_clause
TABLESPACE tablespace
[LOGGING|NOLOGGING]
idx_organized_tbl_clause:
storage_option(s) [PCTTHRESHOLD int]
[COMPRESS int|NOCOMPRESS]
[ [INCLUDING column_name] OVERFLOW [storage_option(s)] ]
nested_storage_clause:
NESTED TABLE nested_item STORE AS storage_table
[RETURN AS {LOCATOR|VALUE} ]
partitioning_options:
Partition_clause {ENABLE|DISABLE} ROW MOVEMENT
Column Constraints (http://www.ss64.com/ora/clause_constraint_col.html)
CONSTRAINT constrnt_name {UNIQUE|PRIMARY KEY} constrnt_state
CONSTRAINT constrnt_name CHECK(condition) constrnt_state
CONSTRAINT constrnt_name [NOT] NULL constrnt_state
CONSTRAINT constrnt_name REFERENCES [schema.]table[(column)]
[ON DELETE {CASCADE|SET NULL}] constrnt_state
constrnt_state
[[NOT] DEFERRABLE] [INITIALLY {IMMEDIATE|DEFERRED}]
[RELY | NORELY] [USING INDEX using_index_clause]
[ENABLE|DISABLE] [VALIDATE|NOVALIDATE]
[EXCEPTIONS INTO [schema.]table]
Note that probably not all of the above syntax is supported, but the grammar was altered to better handle the syntax created by
DDL::Oracle.
AUTHOR
Ken Youens-Clark <kclark@cpan.org>.
SEE ALSO
SQL::Translator, Parse::RecDescent, DDL::Oracle.
perl v5.14.2 2012-01-18 SQL::Translator::Parser::Oracle(3pm)