Sponsored Content
Full Discussion: awk use to generate report
Top Forums UNIX for Beginners Questions & Answers awk use to generate report Post 303039795 by vedanta on Tuesday 15th of October 2019 03:32:14 PM
Old 10-15-2019
awk use to generate report

Hi ,
In a directory list of ddl files are stored in the given format above. Above is the sample ddl file. The ddl file name is same as that of table name ie email_notifications.ddl
I want to generate below report using awk utility reading all the ddl files stored in /ddl path
Code:
Desired output:
table name, last_modified_time, totalsize
email_notifications,1559883505,8765717674
email_bounced, 1559883505,999999

Also, would like to know if the modified time in unix can be converted to actual date time instead of unix time

Note: It may happen that in other DDL files TBLPROPERTIES information may or may not available.


here is one of the file sample format:
Code:
CREATE TABLE `test.email_notifications`(
  `email_address_id` decimal(16,0), 
  `individual_id` decimal(16,0), 
  `email_address` string, 
  `last_bounce_date` string 
)
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY '|' 
STORED AS INPUTFORMAT 
  'org.apache.hadoop.mapred.TextInputFormat' 
OUTPUTFORMAT 
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
LOCATION
  'hdfs://apps/hive/db/test.db/email_notifications'
TBLPROPERTIES (
  'last_modified_by'='ops', 
  'last_modified_time'='1559883505', 
  'numFiles'='1', 
  'numRows'='0', 
  'rawDataSize'='0', 
  'skip.header.line.count'='1', 
  'totalSize'='8765717674', 
  'transient_lastDdlTime'='1559883505')

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help generate report from log files

Hi Expert, I have some confusing to generate report from the log file as shown below: filename :test1.log start_time date end_time number code P 000029.621 20070823 000029 12134567890 111111111111 00 0 000 003... (4 Replies)
Discussion started by: bucci
4 Replies

2. Shell Programming and Scripting

Generate report in HTML file from Oracle DB

Hi Team, I need a suggestion/comments for my below requirement. I have a procedure which performs some DDL operations & loads data into a Oracle table. This status contains Audit data. What i wanted to do is, once the procedure is completed (daily), shell script should retrive the data from the... (4 Replies)
Discussion started by: Amit.Sagpariya
4 Replies

3. Shell Programming and Scripting

generate a report

Hi Please help me to resolve the below query. My shell script has generated a file output.file like below ******************************** DROP TABLE GPS_CONTACT_DETAILS DB20000I The SQL command completed successfully. CREATE TABLE GPS_CONTACT_DETAILS ( CONTACT_ID ... (8 Replies)
Discussion started by: sailaja_80
8 Replies

4. AIX

Re-Generate error report

Hi Is there a way to re-generate AIX error report after clearing it? I mean if I did errclear 0, Does the error log get erased? Is there a command that will get entries back in errpt? I hope that made sense. (5 Replies)
Discussion started by: Dardeer
5 Replies

5. Shell Programming and Scripting

generate a report

I am trying to generate a report for a file called phone_book awk -f {phone_book} why does this not work? Nothing happens at all. (2 Replies)
Discussion started by: gustave
2 Replies

6. Shell Programming and Scripting

generate report based on data in files.

Hi All, I need to develop a shell script which does sanity check of a data file, as below. 1. For DATE columns, it should check if date is given in proper format or not? For example, if format of date is expected as DD-MON-YYYY HH24:MI:SS and we received the date in formation like DDMMYYYY HH24,... (1 Reply)
Discussion started by: ace_friends22
1 Replies

7. Shell Programming and Scripting

Need a script that generate a report from a combined sudoers

Hello Gurus Is anyone aware of a script/utility that can be used to generate a report from a combined "sudoers" file for more than 100 servers in one go please? Basic query: Which particular user or an user group can execute root equivalent commands or has the ability for e.g. to do a... (1 Reply)
Discussion started by: kapil514
1 Replies

8. Shell Programming and Scripting

Generate a Summary report

Hi All, Script to meet my requirement might be simpler for UINIX experts.. :) I need to generate an summary report in .txt file using shell script I have Reject directory in Unix server which contains all reject files for three diff categories- Presentation, Chapter and Scene Following... (3 Replies)
Discussion started by: Sakthikalluri
3 Replies

9. Shell Programming and Scripting

Generate .csv/ xls file report

There can be thousand of .ksh in a specific directory where sql files are called from ksh. Requirement is to loop through all the files content and generate a report like below: Jobname Type type sqlname gemd1970 sql daily tran01 gemw1971 sql weekly ... (6 Replies)
Discussion started by: vedanta
6 Replies

10. Shell Programming and Scripting

To get older than last 7days records using awk scripting to generate report and send email

Hello All, I have need as below: 1--> I need to get all users(who submit jobs) and their details by using below command: qstat -u \* output of the above command looks line below: job-ID prior name user-id state "submit/start at" queue jclass slots ja-task-ID... (5 Replies)
Discussion started by: VasuKukkapalli
5 Replies
SELECT 
INTO(7) SQL Commands SELECT INTO(7) NAME
SELECT INTO - define a new table from the results of a query SYNOPSIS
[ WITH [ RECURSIVE ] with_query [, ...] ] SELECT [ ALL | DISTINCT [ ON ( expression [, ...] ) ] ] * | expression [ [ AS ] output_name ] [, ...] INTO [ TEMPORARY | TEMP ] [ TABLE ] new_table [ FROM from_item [, ...] ] [ WHERE condition ] [ GROUP BY expression [, ...] ] [ HAVING condition [, ...] ] [ WINDOW window_name AS ( window_definition ) [, ...] ] [ { UNION | INTERSECT | EXCEPT } [ ALL ] select ] [ ORDER BY expression [ ASC | DESC | USING operator ] [ NULLS { FIRST | LAST } ] [, ...] ] [ LIMIT { count | ALL } ] [ OFFSET start [ ROW | ROWS ] ] [ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLY ] [ FOR { UPDATE | SHARE } [ OF table_name [, ...] ] [ NOWAIT ] [...] ] DESCRIPTION
SELECT INTO creates a new table and fills it with data computed by a query. The data is not returned to the client, as it is with a normal SELECT. The new table's columns have the names and data types associated with the output columns of the SELECT. PARAMETERS
TEMPORARY or TEMP If specified, the table is created as a temporary table. Refer to CREATE TABLE [create_table(7)] for details. new_table The name (optionally schema-qualified) of the table to be created. All other parameters are described in detail under SELECT [select(7)]. NOTES
CREATE TABLE AS [create_table_as(7)] is functionally similar to SELECT INTO. CREATE TABLE AS is the recommended syntax, since this form of SELECT INTO is not available in ECPG or PL/pgSQL, because they interpret the INTO clause differently. Furthermore, CREATE TABLE AS offers a superset of the functionality provided by SELECT INTO. Prior to PostgreSQL 8.1, the table created by SELECT INTO included OIDs by default. In PostgreSQL 8.1, this is not the case -- to include OIDs in the new table, the default_with_oids configuration variable must be enabled. Alternatively, CREATE TABLE AS can be used with the WITH OIDS clause. EXAMPLES
Create a new table films_recent consisting of only recent entries from the table films: SELECT * INTO films_recent FROM films WHERE date_prod >= '2002-01-01'; COMPATIBILITY
The SQL standard uses SELECT INTO to represent selecting values into scalar variables of a host program, rather than creating a new table. This indeed is the usage found in ECPG (see in the documentation) and PL/pgSQL (see in the documentation). The PostgreSQL usage of SELECT INTO to represent table creation is historical. It is best to use CREATE TABLE AS for this purpose in new code. SEE ALSO
CREATE TABLE AS [create_table_as(7)] SQL - Language Statements 2010-05-14 SELECT INTO(7)
All times are GMT -4. The time now is 04:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy