Sponsored Content
Full Discussion: awk use to generate report
Top Forums UNIX for Beginners Questions & Answers awk use to generate report Post 303039795 by vedanta on Tuesday 15th of October 2019 03:32:14 PM
Old 10-15-2019
awk use to generate report

Hi ,
In a directory list of ddl files are stored in the given format above. Above is the sample ddl file. The ddl file name is same as that of table name ie email_notifications.ddl
I want to generate below report using awk utility reading all the ddl files stored in /ddl path
Code:
Desired output:
table name, last_modified_time, totalsize
email_notifications,1559883505,8765717674
email_bounced, 1559883505,999999

Also, would like to know if the modified time in unix can be converted to actual date time instead of unix time

Note: It may happen that in other DDL files TBLPROPERTIES information may or may not available.


here is one of the file sample format:
Code:
CREATE TABLE `test.email_notifications`(
  `email_address_id` decimal(16,0), 
  `individual_id` decimal(16,0), 
  `email_address` string, 
  `last_bounce_date` string 
)
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY '|' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://apps/hive/db/test.db/email_notifications'
TBLPROPERTIES (
  'last_modified_by'='ops', 
  'last_modified_time'='1559883505', 
  'numFiles'='1', 
  'numRows'='0', 
  'rawDataSize'='0', 
  'skip.header.line.count'='1', 
  'totalSize'='8765717674', 
  'transient_lastDdlTime'='1559883505')

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help generate report from log files

Hi Expert, I have some confusing to generate report from the log file as shown below: filename :test1.log start_time date end_time number code P 000029.621 20070823 000029 12134567890 111111111111 00 0 000 003... (4 Replies)
Discussion started by: bucci
4 Replies

2. Shell Programming and Scripting

Generate report in HTML file from Oracle DB

Hi Team, I need a suggestion/comments for my below requirement. I have a procedure which performs some DDL operations & loads data into a Oracle table. This status contains Audit data. What i wanted to do is, once the procedure is completed (daily), shell script should retrive the data from the... (4 Replies)
Discussion started by: Amit.Sagpariya
4 Replies

3. Shell Programming and Scripting

generate a report

Hi Please help me to resolve the below query. My shell script has generated a file output.file like below ******************************** DROP TABLE GPS_CONTACT_DETAILS DB20000I The SQL command completed successfully. CREATE TABLE GPS_CONTACT_DETAILS ( CONTACT_ID ... (8 Replies)
Discussion started by: sailaja_80
8 Replies

4. AIX

Re-Generate error report

Hi Is there a way to re-generate AIX error report after clearing it? I mean if I did errclear 0, Does the error log get erased? Is there a command that will get entries back in errpt? I hope that made sense. (5 Replies)
Discussion started by: Dardeer
5 Replies

5. Shell Programming and Scripting

generate a report

I am trying to generate a report for a file called phone_book awk -f {phone_book} why does this not work? Nothing happens at all. (2 Replies)
Discussion started by: gustave
2 Replies

6. Shell Programming and Scripting

generate report based on data in files.

Hi All, I need to develop a shell script which does sanity check of a data file, as below. 1. For DATE columns, it should check if date is given in proper format or not? For example, if format of date is expected as DD-MON-YYYY HH24:MI:SS and we received the date in formation like DDMMYYYY HH24,... (1 Reply)
Discussion started by: ace_friends22
1 Replies

7. Shell Programming and Scripting

Need a script that generate a report from a combined sudoers

Hello Gurus Is anyone aware of a script/utility that can be used to generate a report from a combined "sudoers" file for more than 100 servers in one go please? Basic query: Which particular user or an user group can execute root equivalent commands or has the ability for e.g. to do a... (1 Reply)
Discussion started by: kapil514
1 Replies

8. Shell Programming and Scripting

Generate a Summary report

Hi All, Script to meet my requirement might be simpler for UINIX experts.. :) I need to generate an summary report in .txt file using shell script I have Reject directory in Unix server which contains all reject files for three diff categories- Presentation, Chapter and Scene Following... (3 Replies)
Discussion started by: Sakthikalluri
3 Replies

9. Shell Programming and Scripting

Generate .csv/ xls file report

There can be thousand of .ksh in a specific directory where sql files are called from ksh. Requirement is to loop through all the files content and generate a report like below: Jobname Type type sqlname gemd1970 sql daily tran01 gemw1971 sql weekly ... (6 Replies)
Discussion started by: vedanta
6 Replies

10. Shell Programming and Scripting

To get older than last 7days records using awk scripting to generate report and send email

Hello All, I have need as below: 1--> I need to get all users(who submit jobs) and their details by using below command: qstat -u \* output of the above command looks line below: job-ID prior name user-id state "submit/start at" queue jclass slots ja-task-ID... (5 Replies)
Discussion started by: VasuKukkapalli
5 Replies
Test::Regression(3pm)					User Contributed Perl Documentation				     Test::Regression(3pm)

NAME
Test::Regression - Test library that can be run in two modes; one to generate outputs and a second to compare against them VERSION
Version 0.05 SYNOPSIS
use Test::Regression; ok_regression(sub {return "hello world"}, "t/out/hello_world.txt"); DESCRIPTION
Using the various Test:: modules you can compare the output of a function against what you expect. However if the output is complex and changes from version to version, maintenance of the expected output could be costly. This module allows one to use the test code to generate the expected output, so that if the differences with model output are expected, one can easily refresh the model output. EXPORT
ok_regression FUNCTIONS
ok_regression This function requires two arguments: a CODE ref and a file path. The CODE ref is expected to return a SCALAR string which can be compared against previous runs. If the TEST_REGRESSION_GEN is set to a true value, then the CODE ref is run and the output written to the file. Otherwise the output of the file is compared against the contents of the file. There is a third optional argument which is the test name. ENVIRONMENT VARIABLES
TEST_REGRESSION_GEN If the TEST_REGRESSION_GEN environment file is unset or false in a perl sense, then the named output files must exist and be readable and the test will run normally comparing the outputs of the CODE refs against the contents of those files. If the environment variable is true in a perl sense, then model output files will be overwritten with the output of the CODE ref. AUTHOR
Nicholas Bamber, "<nicholas at periapt.co.uk>" BUGS
Please report any bugs or feature requests to "bug-test-regression at rt.cpan.org", or through the web interface at <http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Regression>. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. testing of STDERR The testing of stderr from this module is not as thorough as I would like. Test::Builder::Tester allows turning off of stderr checking but not matching by regular expression. Handcrafted efforts currently fall foul of Test::Harness. Still it is I believe adequately tested in terms of coverage. SUPPORT
You can find documentation for this module with the perldoc command. perldoc Test::Regression You can also look for information at: o RT: CPAN's request tracker <http://rt.cpan.org/NoAuth/Bugs.html?Dist=Test-Regression> o AnnoCPAN: Annotated CPAN documentation <http://annocpan.org/dist/Test-Regression> o CPAN Ratings <http://cpanratings.perl.org/d/Test-Regression> o Search CPAN <http://search.cpan.org/dist/Test-Regression/> ACKNOWLEDGEMENTS
Some documentation improvements have been suggested by toolic (http://perlmonks.org/?node_id=622051). Thanks to Filip GraliXski for pointing out I need to test against output of zero length and providing a patch. COPYRIGHT &; LICENSE Copyright 2009-10 Nicholas Bamber. This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License. See http://dev.perl.org/licenses/ for more information. perl v5.10.1 2010-08-29 Test::Regression(3pm)
All times are GMT -4. The time now is 02:47 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy