Script to Gather data from logs and export to a CSV file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to Gather data from logs and export to a CSV file
# 8  
Old 11-23-2016
Yes cfg is the configuration file. also not I change the full path to ./ while testing here so it is currently looking for ./File_YYYMMDD* input files not /path has you originally had.

Have you checked the Error_Report_YYYMMDD.csv file?

When you say nothing is appearing does that include the headings?

If you are just getting headings and not data records, this means the your Pattern set on line 1 in PAT variable is not matching any line.

If you get lines with all blank totals then PAT matches the line fields 28-30 mismatch OR no error codes match.
This User Gave Thanks to Chubler_XL For This Post:
# 9  
Old 11-23-2016
Another thought, if it is not completing and you are sending files with millions of records, try a single smaller test file first.
# 10  
Old 11-24-2016
Got it.

All clear. Using the same criteria as yours I manage to achieve the desired output. A few lines of the code is very complex, but will help me improve.

I just need to do some small changes and the report will be perfect.

I am considering this as Solved. And I really appreciate your help .

Once again, thank you very much.

---------- Post updated at 04:21 AM ---------- Previous update was at 12:56 AM ----------

Just another question.

Script is working properly, but I don't know how to avoid this line :

Code:
2001,Server1,TIMESTAMP:00:00,,,,,,,,,,,,,,,,,,,,
2001,Server1,2016-11-21,20:00:00,,,,,,,,,,,,,,,,,,,,
2001,Server1,2016-11-21,21:00:00,,,,,,,,,,,,,,,,,,,,

Any ideia ?

Thanks again !
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Linux

Parsing - export html table data as .csv file?

Hi all, Is there any out there have a brilliant idea on how to export html table data as .csv or write to txt file with separated comma and also get the filename of link from every table and put one line per rows each table. Please see the attached html and PNG of what it looks like. ... (7 Replies)
Discussion started by: lxdorney
7 Replies

2. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

3. Shell Programming and Scripting

Shell script to extract data from csv file

Hi everyone, I have a csv file which has data with different heading and column names as below. Static Data Ingested ,,,,,,,,,,,,Known Explained Rejections Column_1,column_2,Column_3,Column_4,,Column_6,Column_7,,% Column_8,,Column_9 ,Column_10 ,... (14 Replies)
Discussion started by: Vivekit82
14 Replies

4. UNIX for Dummies Questions & Answers

Shell script to extract data from csv file

Hi Guys, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 7 columns having values say column 1,column 2.....column 7 as below along with their values. Name, Address,... (7 Replies)
Discussion started by: Vivekit82
7 Replies

5. Shell Programming and Scripting

Run script to export the data to ixf file in loop

Hi, I am trying to export the data to an .ixf file. I have read the table names from a .dat file and those table name should be passed to the select * from schema.TABLENAME query . I am trying the below loop while read TABLE; do db2 EXPORT TO ~/data_export/$TABLE.ixf OF IXF MESSAGES... (5 Replies)
Discussion started by: vikyalex4
5 Replies

6. Shell Programming and Scripting

Exporting data as a CSV file from Unix shell script

Friends...This is the first time i am trying the report generation using shell script... any suggestions are welcome. Is there a way to set the font size & color when i am exporting the data from unix shell script as a CSV file ? The following sample data is saved as a .csv file in the... (2 Replies)
Discussion started by: appu2176
2 Replies

7. Web Development

script to load data from csv file

hello i want a script to load the data line by line from a csv file into a mysql table (3 Replies)
Discussion started by: srpa01red
3 Replies

8. Shell Programming and Scripting

FTP script to gather logs.

Hi, I am currently working on a FTP script which would ftp to extranet servers from intranet server and gather logs based on timestamps in archive logs and by using a wildcard character in the present logs. I have the following in place AAA="Extranet Server 1:Instance Extranet Server... (2 Replies)
Discussion started by: openspark
2 Replies

9. Shell Programming and Scripting

Shell Script to Load data into the database using a .csv file and .ctl file

Since i'm new to scripting i'm findind it difficult to code a script. The script has to be an executable with 2 paramters passed to it.The Parameters are 1. The Control file name(.ctl file) 2. The Data file name(.csv file) Does anybody have an idea about it? :confused: (3 Replies)
Discussion started by: Csmani
3 Replies

10. Shell Programming and Scripting

unix script to export data from csv file to oracle database

Hello people, Need favour. The problem I have is that, I need to develop a unix shell script that performs recurring exports of data from a csv file to an oracle database. Basically, the csv file contains just the first name and last name will be dumped to an Unix server. The data from these... (3 Replies)
Discussion started by: vinayagan
3 Replies
Login or Register to Ask a Question