Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Prepare file run report

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Prepare file run report
# 1  
Old 10-27-2014
Prepare file run report
I have a requirement to prepare a report.

We validate some incoming data fields and create validation_error reports which will contain records which do not pass validation. Once files are processed they will all be dropped under one folder.

Code:
EMPLOYEE_20140915.txt
EMPLOYEE_20140915.txt.error.dat
DEPARTMENT_20140911.txt
DEPARTMENT_20140911.txt.error.dat
PRSNL_INFO_20140911.txt
SAL_HIKE_20140911.txt
SALARY_20140911.txt
SALARY_20140911.txt.error.dat

As seen above, of the 5 files, 3 of them have rejects and 2 were processed fine. I need to prepare a report which looks like:
Code:
FILE NAME	  Status
-------------------------------------
PRSNL_INFO	- Processed successfully
SAL_HIKE	- Processed successfully
EMPLOYEE 	- Has error resubmit
DEPARTMENT	- Has error resubmit
SALARY		- Has error resubmit

Can someone help me how I can go about creating the report?
# 2  
Old 10-27-2014
Not the slightest attempt from your side? Try
Code:
ls *.txt |
awk     'BEGIN          {print "FILE NAME\tStatus\n-------------------------------------"}
                        {gsub(/_[0-9]*$/, "", $1); OUT[$1]}
         /error/        {OUT[$1]++}
         END            {for (i in OUT) print i, OUT[i]?"- has error":"- success"}
        ' FS="." OFS="\t"
FILE NAME     Status
-------------------------------------
SALARY    - has error
DEPARTMENT    - has error
EMPLOYEE      - has error
PRSNL_INFO    - success
SAL_HIKE      - success

# 3  
Old 10-27-2014
Thanks RudiC.

I am a beginner in Unix and still learning. I tried a few things but did not get anywhere so wanted to get help.

Can you help me understand what this code is doing? I seem to be getting a success for all the files.

Code:
EMPLOYEE_20140915.txt
EMPLOYEE_20140915.txt.error.dat
DEPARTMENT_20140911.txt
DEPARTMENT_20140911.txt.error.dat
PRSNL_INFO_20140911.txt
SAL_HIKE_20140911.txt
SALARY_20140911.txt
SALARY_20140911.txt.error.dat

ls *.txt | awk     'BEGIN          {print "FILE NAME\tStatus\n-------------------------------------"}
                        {gsub(/_[0-9]*$/, "", $1); OUT[$1]}
         /error/        {OUT[$1]++}
         END            {for (i in OUT) print i, OUT[i]?"- has error":"- success"}
        ' FS="." OFS="\t"
FILE NAME       Status
-------------------------------------
PRSNL_INFO      - success
EMPLOYEE        - success
DEPARTMENT      - success
SAL_HIKE        - success
SALARY  - success

# 4  
Old 10-28-2014
What's your system and awk version? Please post the output of ls *.txt without editing.
# 5  
Old 10-28-2014
Please find below details:

Code:
dev/processed>ls *.txt
DEPARTMENT_20140911.txt  EMPLOYEE_20140915.txt  PRSNL_INFO_20140911.txt  SALARY_20140911.txt  SAL_HIKE_20140911.txt

Code:
AWK version:
dev/processed>awk -W version
GNU Awk 3.1.7
Copyright (C) 1989, 1991-2009

Linux version: 2.6.32-358.11.1.el6.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux

Thanks!
# 6  
Old 10-28-2014
Please use code tags as required by forum rules!

So - where are the error files? No error files - no errors - all success!
# 7  
Old 10-28-2014
Error files are also in the same folder:

Processed files:
Code:
dev/processed>ls *.txt
DEPARTMENT_20140911.txt  EMPLOYEE_20140915.txt  PRSNL_INFO_20140911.txt  SALARY_20140911.txt  SAL_HIKE_20140911.txt

Error files:
Code:
dev/processed>ls *err*
DEPARTMENT_20140911.txt.error.dat  EMPLOYEE_20140915.txt.error.dat  SALARY_20140911.txt.error.dat

I get all files as processed successfully:
Code:
dev/processed>ls *.txt | awk     'BEGIN          {print "FILE NAME\tStatus\n-------------------------------------"}
>	{gsub(/_[0-9]*$/, "", $1); OUT[$1]}
>   /error/        {OUT[$1]++}
>   END            {for (i in OUT) print i, OUT[i]?"- has error":"- success"}
>   ' FS="." OFS="\t"
FILE NAME       Status
-------------------------------------
PRSNL_INFO      - success
EMPLOYEE        - success
DEPARTMENT      - success
SAL_HIKE        - success
SALARY  - success
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script run in a case statement call to run a php file, also Perl

Linux System having all Perl, Python, PHP (and Ruby) installed From a Shell script, can call a Perl, Python, PHP (or Ruby ?) file eg eg a Shell script run in a case statement call to run a php file, also Perl or/and Python file??? Like #!/usr/bin/bash .... .... case $INPUT_STRING... (1 Reply)
Discussion started by: hoyanet
1 Replies

2. Shell Programming and Scripting

Dynamically prepare the condition using flat file sdata

I have a file in which the data looks like A,B My output should be T1.A=T2.A AND T1.B=T2.B If my input is A,B,C My output should be T1.A=T2.A AND T1.B=T2.B AND T1.C=T2.C Currently I am automating my databae script and strucked with this part.Not able to search the... (2 Replies)
Discussion started by: bikky6
2 Replies

3. Red Hat

Modprobe prepare new IP address

Hello, I m working on virtualization and saved the templates in virtual server. On creating the new Virtual machine or linux system, is there a way where during booting, it should prompt for new IP address, gateway, DNS and hostname? Or is there any configuration in linux where we can modify... (5 Replies)
Discussion started by: alnhk
5 Replies

4. UNIX for Dummies Questions & Answers

prepare a perl tuts

Hi I am working on perl for last 3 yrs and as part of knowledge sharing i have been told to conduct sessions with other teams.Only 1 session with basic details. My problem is i am not sure what all should i cover in this tut ? because i learned perl on my own. so i am not sure where to start... (4 Replies)
Discussion started by: zedex
4 Replies

5. Shell Programming and Scripting

How can i prepare a file by comparing two other files?

File 1 data: TestA TestB TestC File 2 data: TestA TestD TestE My Output File (pick all from both and create a file without duplicate data) ---------------------------------------------------------------------- TestA TestB TestC (3 Replies)
Discussion started by: manmohanpv
3 Replies

6. Shell Programming and Scripting

need help on unix script to run report

Hi All, I have a report which I am running from unix shell script using CONCSUB utility. Just for testing I put the order number in shell script as ord_low and ord_high. But Actually what is going to happen is that the order numbers will be in a file say a.txt and this CONCSUB... (2 Replies)
Discussion started by: isingh786
2 Replies

7. HP-UX

Unix sheel script to run report

Hi All, I have a report which I am running from unix shell script using CONCSUB utility. Just for testing I put the order number in shell script as ord_low and ord_high. But Actually what is going to happen is that the order numbers will be in a file say a.txt and this CONCSUB should... (0 Replies)
Discussion started by: isingh786
0 Replies

8. Shell Programming and Scripting

prepare log for access

I need help with one of my log files I got following format: This is only a smal part of the file ! ......... -------------------------------------- 2003-08-05 12:23:13.939781 logNo : 1380008 Server started - Activate; 10.48.4.51 -------------------------------------- 2003-08-05 ... (4 Replies)
Discussion started by: joerg
4 Replies
Login or Register to Ask a Question