Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Filter using awk in CSV files
Top Forums Shell Programming and Scripting Filter using awk in CSV files Post 303008016 by pradyumnajpn10 on Saturday 25th of November 2017 10:59:39 AM
Old 11-25-2017
parse all csv files in directory and process it for Java command
Hi Rudic,

I know this must be silly for you but somewhere I might've over-explained or misguided you. Sorry about that.
Now I tried with all your suggestion and Step 1 and Step 2 and overcame it. Of-course your expertise one liner which I love is most welcome
Code:
#!/bin/bash

#Step 1
awk -F, '{gsub(/^"|"" "$/, "", $16); print $16}' AWS.CSV > out

#Step 2

while IFS= read -r line; do
  #Process each line 3 times with xargs(first try failed)
  # seq 3 | xargs -I{} $line

  for i in `seq 1 3`;
   do
    #Record the start time
    if [ $i -eq 1 ]; then
	start_time=$(date);# need to change later to get certain date format
    fi
    #Record the end time
    if [ $i -eq 3 ]; then
        end_time=$(date); 
    fi
    
    $line
   done < out

#Step 3
#Process all the CSV files starting with exec=1 run and run those files with java command
find . -maxdepth 1 -type f -name "exec=1*.csv" -exec 'java -classpath ./:${CLASSPATH} execGetResult {}' \;

Somehow managed to do step 3.But not sure about the syntax.
Step 3:
Find all the CSV files which is starting with exec=1*.csv and execute java command for each file found like below
Ex:
Code:
java -classpath ./:${CLASSPATH}  execGetResult exec=1_one.csv
java -classpath ./:${CLASSPATH}  execGetResult exec=1_two.csv

I am doing something like below to search the csv files in current diretory but struggling with syntax to embed java command for each csv files
Code:
find . -maxdepth 1 -type f -name "exec=1*.csv" -exec 'java -classpath ./:${CLASSPATH} execGetResult {}' \;

Rgds,
PD
Last edited by pradyumnajpn10; 11-25-2017 at 01:38 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

csv files (with quoted commas) and awk

I have a file as follows: 1,"This is field 2",3,4,5 2,"This is field 2 it can contain one , comma",3,4,5 3,"This is field 2 it also, can, contain, more",3,4,5 4,"This is field 2 without extra commas",3,4,5 and i pass this through to awk: awk -F, ' { if (... (3 Replies)
Discussion started by: Cranie
3 Replies

2. Shell Programming and Scripting

validation of data using filter (awk or other that works...) in csv files

Hello People I have the following file.csv: date,string,float,number,boolean 20080303,abc,1.5,123,Y 20080304,abc,1.2,345,N 20080229,nvh,1.4,098,Y 20080319,ugy,1.9,586,N 20080315,gyh,2.4,345,Y 20080316,erf,3.1,932,N I need to filter the date field where I have a data bigger than I... (1 Reply)
Discussion started by: Rafael.Buria
1 Replies

3. Shell Programming and Scripting

how to give multiple csv files as input in awk

Hi All, I am new to shell scripting..My problem is i want to give multiple csv files as input to awk script and process the data into one file.. My input file is File1 File2 File3 Product Location Period SalesPrice A x 8/11/2010 ... (7 Replies)
Discussion started by: kvth
7 Replies

4. Shell Programming and Scripting

Merge 2 csv files with awk

I have 2 files pipe delimted and want to merge them based on a key e.g file 1 123$aaa$yyy$zzz 345$xab$yzy$zyz 456$sss$ttt$foo 799$aaa$ggg$dee file 2 123$hhh 345$ddd 456$xxx 888$zzz so if the key is the first field, and the result should be the common key between file 1 and 2 (6 Replies)
Discussion started by: loloAix
6 Replies

5. Shell Programming and Scripting

Using AWK to match CSV files with duplicate patterns

Dear awk users, I am trying to use awk to match records across two moderately large CSV files. File1 is a pattern file with 173,200 lines, many of which are repeated. The order in which these lines are displayed is important, and I would like to preserve it. File2 is a data file with 456,000... (3 Replies)
Discussion started by: isuewing
3 Replies

6. Shell Programming and Scripting

awk filter by columns of file csv

Hi, I would like extract some lines from file csv using awk , below the example: I have the file test.csv with in content below. FLUSSO;COD;DATA_LAV;ESITO ULL;78;17/09/2013;OL ULL;45;05/09/2013;Apertura NP;45;13/09/2013;Riallineamento ULLNP;78;17/09/2013;OL NPG;14;12/09/2013;AperturaTK... (6 Replies)
Discussion started by: giankan
6 Replies

7. UNIX for Beginners Questions & Answers

awk assistance - Comparing 2 csv files

Hello all, I have searched high and low for a solution to this, many have come really close but not quite what I'm after. I have 2 files. One contains GUID's, for example: 8121E002-96FE-4C9C-BC5A-6AFF20DACECD 84468F30-F3B7-418B-81F0-0908E80792BF A second file, contains a path to the... (8 Replies)
Discussion started by: tirmUK
8 Replies

8. Shell Programming and Scripting

Modify csv-files with awk

Hello everyone! I have thousands of csv files I have to import into a Database table. As usually the files aren't perfect. For example they have a different number of columns and some weird columns. The second problem is, that I have to add 3 parts of the filename into 3 rows in the... (6 Replies)
Discussion started by: elRonaldo
6 Replies

9. UNIX for Beginners Questions & Answers

awk on csv files

wk on csv files Hi I have two csv files 1 ) keys.csv with data as below XX1,offsite XX2,offsite XX8,none XX3,offsite XX4,onsite XX7,none XX5,onsite XX6,onsite XX15,none 2) source.csv file with data like 1,0123,ppp,XX1 2,0122,sss,XX2 3,1239,yyy,XX8 4,567,kkk,XX5 (2 Replies)
Discussion started by: zozoo
2 Replies

10. Shell Programming and Scripting

Can we filter the below log data into CSV format?

HI , I m looking for help here!!! Can we filter the below log data into CSV format ? timestamp INFO <text > - Some text Drive .. Need a format of separate field such as 1 2 3 4 ... (2 Replies)
Discussion started by: MohSalNiz
2 Replies

LEARN ABOUT DEBIAN

libglpk-java

libglpk-java(3) 					       libglpk-java overview						   libglpk-java(3)

NAME

       libglpk-java - GNU Linear Programming Kit Java Binding

DESCRIPTION

       The  GNU  Linear  Programming  Kit  (GLPK)  package supplies a solver for large scale linear programming (LP) and mixed integer programming
       (MIP).  The GLPK project is hosted at http://www.gnu.org/software/glpk.

       It has two mailing lists:
       - help-glpk@gnu.org and
       - bug-glpk@gnu.org.

       To subscribe to one of these lists, please, send an empty mail with a Subject: header line of just "subscribe" to the list.

       GLPK provides a library written in C and a standalone solver.  The source code provided at ftp://gnu.ftp.org/gnu/glpk/ contains	the  docu-
       mentation of the library in file doc/glpk.pdf.

       The Java platform provides the Java Native Interface (JNI) to integrate non-Java language libraries into Java applications.

       Project GLPK for Java delivers a Java Binding for GLPK. It is hosted at http://glpk-java.sourceforge.net/.

       To report problems and suggestions concerning GLPK for Java, please, send an email to the author at xypron.glpk@gmx.de.

ARCHITECTURE

       A GLPK for Java application will consist of the following
       - the GLPK library
       - the GLPK for Java JNI library
       - the GLPK for Java class library
       - the application code.

GLPK LIBRARY

       The GLPK library can be compiled from source code. Follow the instructions in file INSTALL provided in the source distribution. Precompiled
       packages are available in many Linux distributions.

       The usual installation path for the library is /usr/local/lib/libglpk.so.  The library has to be in the search path for binaries.

GLPK FOR JAVA JNI LIBRARY

       The GLPK for Java JNI library can be compiled from source code. Follow the instructions in file INSTALL provided in  the  source  distribu-
       tion.

       The  usual installation path for the library is /usr/local/lib/jni/libglpk-java.so.  The library has to be in the search path for binaries.
       Specify the library path upon invocation of the application, e.g.
       java -Djava.library.path=/usr/local/libjni

GLPK FOR JAVA CLASS LIBRARY

       The source code to compile the GLPK for Java class library is provided at http://glpk-java.sourceforge.net.

       The GLPK for Java class library can be compiled from source code. Follow the instructions in file INSTALL provided in the source  distribu-
       tion.  The usual installation path for the library is /usr/local/share/java/glpk-java.jar.

       The library has to be in the CLASSPATH. Specify the classpath upon invocation of the application, e.g.
       java -classpath /usr/local/share/java/glpk-java.jar;.

SEE ALSO

       Further documentation and examples can be found in the documentation path, which defaults to /usr/local/share/doc/libglpk-java.

version 1.0.18						       September 10th, 2011						   libglpk-java(3)
All times are GMT -4. The time now is 05:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy