Script to Grep column 3 from csv file generated yesterday


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to Grep column 3 from csv file generated yesterday
# 1  
Old 01-24-2011
Script to Grep column 3 from csv file generated yesterday

Hello,

Can any one please assist how to scirpt it:

Every day a new log file is create and I want to process only the one generated yesterday and get the data of column 3 and 6.

For example today's date is 24 then I want to get the data of log file created on 23rd.

Log Files in Path_dir:\\log are in this format:
Test Log 2011-01-24.csv
Test Log 2011-01-23.csv


Each file has the same format:
Name code source server serial group
abc 123 tty1 ser1 xy12 234
john 1233 tty2 ser1 xy12 56


Appreciate your response.
# 2  
Old 01-24-2011
Assuming we have "no , text in '"', then sed or awk can do this. Hopefulle, 5 ',' on every line!
Code:
 
sed '
  s/^[^,]*,[^,]*,\([^,]*\),[^,]*,[^,]*,\([^,]*\).*/\1 \2/
 '


Last edited by DGPickett; 01-24-2011 at 06:04 PM.. Reason: field 6
# 3  
Old 01-24-2011
Code:
cat `ls -r | head -2 | tail -1` | cut -d',' -f3,6

finds the 2nd most recent file (yesterday)
displays it
cutting out only the 3rd and 6th fields
# 4  
Old 01-26-2011
Thanks Joeyg. The command works from CLI, I am trying to script it:

Can you please check the code:

Log files in c:\\log are
Test Log 2011-01-24.csv
Test Log 2011-01-23.csv

Code:
 
#!c:\perl\bin\perl.exe
 
#File Directory
$file_dir="C:\\log";
 
# Get yesterdays file
$last_file=`ls -r $file_dir | head -2 | tail -1`;

#Get Column3 and column6
$all=`cat "$last_file" | cut -d',' -f3,6 | sort | uniq`;
print "$all";

When I run the script, I get the following error:
C:\files\script>perl yesterday.pl
cat: Test Log 2011-01-23.csv: No such file or directory
# 5  
Old 01-26-2011
Try like this,

Code:
$all=`cat "$file_dir\\$last_file" | cut -d',' -f3,6 | sort | uniq`;

# 6  
Old 01-26-2011
It only executes the first half but
cut -d',' -f3,6 | sort | uniq does not execute....

Thanks
# 7  
Old 01-26-2011
Could this help ?
Code:
$all=`cut -d "," -f3,6 $file_dir'\\'$last_file | sort | uniq`;

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Python or Shell script to Grep strings from input file and output in csv format

Hi Experts, I am writing a python script to grep string from file and display output in csv file as in attached screenshot https://drive.google.com/file/d/1gfUUdfmQma33tz65NskThYDhkZUGQO0H/view Input file(result_EPFT_config_device) Below is the python script i have prepared as of... (1 Reply)
Discussion started by: as7951
1 Replies

2. Shell Programming and Scripting

Need awk or Shell script to compare Column-1 of two different CSV files and print if column-1 matche

Example: I have files in below format file 1: zxc,133,joe@example.com cst,222,xyz@example1.com File 2 Contains: hxd hcd jws zxc cst File 1 has 50000 lines and file 2 has around 30000 lines : Expected Output has to be : hxd hcd jws (5 Replies)
Discussion started by: TestPractice
5 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

Script for extracting data from csv file based on column values.

Hi all, I am new to shell script.I need your help to write a shell script. I need to write a shell script to extract data from a .csv file where columns are ',' separated. The file has 5 columns having values say column 1,column 2.....column 5 as below along with their valuesm.... (3 Replies)
Discussion started by: Vivekit82
3 Replies

5. Shell Programming and Scripting

Script to grep for a string in log files generated in last 15 minutes.

Dear Guru's I've a requirment to grep for a string in series of log files that are getting generated almost every minute. I'm looking to schedule a script every 15 mountes,in order to check if the error string has been generated in any of the log files generated in last 15 minutes. Please... (3 Replies)
Discussion started by: rajivatnova
3 Replies

6. Shell Programming and Scripting

Perl code to grep a particular column in CSV format

Hi I want to grep a column 6 & column 7 from a CSV Format file & then i have to find the difference between these columns as these both columns contains date & time in 7/7/2012 9:20 this format . So kindly help me out ASAP. But please kindly dis xls has to be converted in csv format as may... (5 Replies)
Discussion started by: Prateek@123
5 Replies

7. Shell Programming and Scripting

Help with selecting column with awk for a txt file generated by excel

I am new to scripting/programming, so I apologize for any novice questions. I have a tab delimited text file that was saved from excel xls file. I am trying to select only the third column using awk command. My command line is as below: cat test.txt | awk '{print $3}' However, above... (8 Replies)
Discussion started by: SangLad
8 Replies

8. Shell Programming and Scripting

Retrieve logs generated in last 10 mins from a log file using 'grep' command

HI All, I have a log file where the logs will be in the format as given below: 2011-05-25 02:32:51 INFO PROCESS STARTING 2011-05-25 02:32:52 INFO PROCESS STARTED . . . I want to retrieve only the logs which are less than 5 mins older than current time using grep... (3 Replies)
Discussion started by: rvhg16
3 Replies

9. UNIX for Dummies Questions & Answers

Option in sql script to include column headers when spooling file to .csv format

Can anyone help me how to include COLUMN HEADER when spooling file to .CSV format through SQL statement. Thanks, Akbar (4 Replies)
Discussion started by: s1a2m3
4 Replies

10. Programming

how to add CRLF support for CSV file generated in unix

Helo, my server side system is running on a redhat linux o.s. I have c program on the server which export list file into CSV format. now, I want that my program on server side is able to add support for CRLF(carriage return Line feed)into csv file format. so how do i write C program whcih... (3 Replies)
Discussion started by: amitpansuria
3 Replies
Login or Register to Ask a Question