Please become accustomed to deploy utmost care when specifying your request, esp. constraints like "Exactly (no more and no less than) the GUIDs in file1", and give people a chance to understand what be test.csv, and what *.csv.
Why, for example, not just simply
I guess because you want to discriminate file2 with the values in file1. How about
Code:
awk -F "[,]" 'NR==FNR{A[$1]; next} { for (a in A) {if ($2 ~ a) print a, $2}}' OFS=, file[12]
84468F30-F3B7-418B-81F0-0908E80792BF,"TEST/DATA/84468F30-F3B7-418B-81F0-0908E80792BF.pdf"
8121E002-96FE-4C9C-BC5A-6AFF20DACECD,"TEST/DATA/8121E002-96FE-4C9C-BC5A-6AFF20DACECD.pdf"
As you can see, 1BCE1E40-D1BE-4DC1-8A0C-9EB236F56944 is in both the first csv and the 2nd csv. The last line of the 2nd csv it is embedded into a double GUID. I want the output to be:
Which is, except for the space, wherever that comes from, exactly what my second suggestion would yield - given the input files are served in the correct order. The shell will expand file[12] to file1 file2
Quote:
RudiC, for your suggestion, where do I pass in the first filename?
Now, there are two possibilities to arrange two file names: file1 file2 or file2 file1. Which did you test - only one will yield an (THE) output?
That works great thank you. I have ran a few test's on smaller files, and its fine,
I have 2 further queries that extend this. First, the lookup file contains 1.87 million rows, and the CSV file to check it against, contains 400,000 rows (there are around 40 csv files to chec).
Running it takes a very long time, so I'm wondering if that might change the command at all, or if it just needs to run overnight.
The second query, is can I use the awk command above, but instead of searching in a csv, can i search a directory? If we take the example above:
2 million lines is quite something, and you have to read all into memory. Checking against 40 * 0.4 E6 (16 million) will take its time. Not sure if the system will be already into swapping with data amounts like those. Try cutting the 2 million in half or quarter.
Not sure if I understand your second query. Your usage of "csv" is not quite consistent and obvious to me. awk needs files to operate upon, not directories. If you open, read, and close 40 (or so) files for every single line read from the (lookup / test / csv / GUID) file, you'll thrash your file system. Not clever.
Last edited by RudiC; 07-18-2018 at 05:51 AM..
Reason: Corrected an unclear formulation.
I have two csv files and im trying to compare them. e.g.
SAMPLE DATA:
file one:
ZipCode Name
20878 Washington
10023 Missouri
20304 Maryland
file two:
ID Name City ZipCode
11654 ... (11 Replies)
Hi All,
My requirement is to write a shell script to compare two large csv files.
I've created sample files for explaining my problem
i.e., a.csv and b.csv
contents of files:
-----------------
a.csv
------
Type,Memory (Kb),Location HD,Size (Mb),Serial #
XT,640,D402,0,MG0010... (2 Replies)
Hello,
I have about 10 csv files which range from csv1 - csv10.
Each csv file has same type/set of tabs and we have around 5-6 tabs for each of the csv file which have slightly different content(data).
A sample of CSV1 is shown below:
Joins: Data related to Joins, it can be any number of... (2 Replies)
Hi All,
I want to remove the rows from File1.csv by comparing the columns/fields in the File2.csv. I only need the records whose first column is same and the second column is different for the same record in both files.Here is an example on what I need.
File1.csv:
RAJAK|ACTIVE|1... (2 Replies)
Hi All,
I want to remove the rows from File1.csv by comparing a column/field in the File2.csv. If both columns matches then I want that row to be deleted from File1 using shell script(awk). Here is an example on what I need.
File1.csv:
RAJAK,ACTIVE,1
VIJAY,ACTIVE,2
TAHA,ACTIVE,3... (6 Replies)
Hi!
I'm just new to shell scripting n simple tasks looks so tough in initial stage. i need to write a script which will read a property file, property file will be containing count of the csv files, and in a folder(same folder) there will be respective csv files.
like Property file
data1=100... (3 Replies)
EDIT: My problems have been solved thanks to the help of bartus11 and pravin27
This code is just to help me learn. It serves no purpose other than that.
Here's a sample csv that I'm working with -
#listofpeeps.csv
Jackie Chan,1954,M
Chuck Norris,1930,M
Bruce Lee,1940,M
This code is... (13 Replies)
Hi All,
I've two .csv files as below
file1.csv
abc, tdf, 223, tpx
jgsd, tex, 342, rpy
a, jdjdsd, 423, djfkld
Where as file2.csv is the new version of file1.csv with some added fields in the end of each line and some additional lines.
lfj, eru, 98, jkldj, 39, jdkj9
abc, tdf, 223, tpx,... (3 Replies)
07-04-2018
