validation of data using filter (awk or other that works...) in csv files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting validation of data using filter (awk or other that works...) in csv files
# 1  
Old 03-05-2008
validation of data using filter (awk or other that works...) in csv files

Hello People

I have the following file.csv:

date,string,float,number,boolean
20080303,abc,1.5,123,Y
20080304,abc,1.2,345,N
20080229,nvh,1.4,098,Y
20080319,ugy,1.9,586,N
20080315,gyh,2.4,345,Y
20080316,erf,3.1,932,N

I need to filter the date field where I have a data bigger than I pass (this date IŽll receive and store into a variable) reject the data and remove it from my file. So today is 20080305 then I receive this date and store in a variable. My file need to have only the rows with the dates 20080303, 20080304, 20080229 according to my file above.

IŽve tried to remove it using the command :
awk -F, 'BEGIN {OFS=","} { if ($1 = ${date}) print $0}' file.csv > ex.csv

However it looks like that the awk do not take variables into account.

Does anyone have any ideia?

Regards
# 2  
Old 03-05-2008
To remove the rows with the current date in the first field:

Code:
awk -v var=${date "+%Y%m%d"} 'BEGIN{FS=OFS=","} $1==var{next}1' file.csv > ex.csv

Regards
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Can we filter the below log data into CSV format?

HI , I m looking for help here!!! Can we filter the below log data into CSV format ? timestamp INFO <text > - Some text Drive .. Need a format of separate field such as 1 2 3 4 ... (2 Replies)
Discussion started by: MohSalNiz
2 Replies

2. Shell Programming and Scripting

Filter using awk in CSV files

Hello Gentlemen, Finding difficulties to play with my Input files:confused: . Your guidance will certainly help as always. After converting to csv file from XLSM file, I am getting some extra ""(double quote) characters which I want to terminate inside shell script and process it further. ... (6 Replies)
Discussion started by: pradyumnajpn10
6 Replies

3. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

4. Shell Programming and Scripting

awk filter by columns of file csv

Hi, I would like extract some lines from file csv using awk , below the example: I have the file test.csv with in content below. FLUSSO;COD;DATA_LAV;ESITO ULL;78;17/09/2013;OL ULL;45;05/09/2013;Apertura NP;45;13/09/2013;Riallineamento ULLNP;78;17/09/2013;OL NPG;14;12/09/2013;AperturaTK... (6 Replies)
Discussion started by: giankan
6 Replies

5. Shell Programming and Scripting

awk : Filter a set of data to parse header line and last field of multiple same match.

Hi Experts, I have a data with multiple entry , I want to filter PKG= & the last column "00060110" or "00088150" in the output file: ############################################################################################### PKG= P8SDB :: VGS = vgP8SOra vgP8SDB1 vgP8S001... (5 Replies)
Discussion started by: rveri
5 Replies

6. UNIX for Dummies Questions & Answers

Extracting data from PDF files into CSV file

Hi, I have several hundreds of PDFfiles number 01.pdf, 02.pdf, 03.pdf, etc in one folder. These are vey long documentd with a lot of information (text, tables, figures, etc). I need to extract the information asociated with one disease in particular (Varicella). The information I need to... (5 Replies)
Discussion started by: Xterra
5 Replies

7. UNIX for Advanced & Expert Users

Time validation in a csv file

Hi, Im having a hard time in creating a script with the following conditions below. I have a csv file generated which is updated every 5 mins and it contains a timestamp in it. for example: time data 00:00 1 00:05 0 00:10 6 00:15 3 however, there is a time that... (4 Replies)
Discussion started by: mdap
4 Replies

8. Shell Programming and Scripting

CSV file parsing and validation

I have a CSV file that needs to through two seperate processes (in the end there will be 2 files (Dload.unl and Tload.unl and we'll say the input file name is mass.csv). I have a processfile() function that will call the process Dload funtion. In Dload I want to read mass.csv into Dload and then... (1 Reply)
Discussion started by: dolo21taf
1 Replies

9. UNIX for Advanced & Expert Users

awk data validation

Hi , This is a general doubt.... Is there any way to do data validation inside an awk script.. Let me make it more clear.... I have a string variable inside awk script .. Is there any way to check whether it is number or string etc... Thanks in advance. Shihab (1 Reply)
Discussion started by: shihabvk
1 Replies

10. Shell Programming and Scripting

Perl - extracting data from .csv files

PROJECT: Extracting data from an employee timesheet. The timesheets are done in excel (for user ease) and then converted to .csv files that look like this (see color code key below): ,,,,,,,,,,,,,,,,,,, 9/14/2003,<-- Week Ending,,,,,,,,,,,,,,,,,, Craig Brennan,,,,,,,,,,,,,,,,,,,... (3 Replies)
Discussion started by: kregh99
3 Replies
Login or Register to Ask a Question