Finding total distinct count from multiple csv files through UNIX script
Hi All ,
I have multiple pipe delimited csv files are present in a directory.I need to find out distinct count on a column on those files and need the total distinct
count on all files.
We can't merge all the files here as file size are huge in millions.I have tried in below way for each files.
I need to automate the above procedure (ex: by putting each files count in a temporary file)as multiple csv files are present in the directory as I need the total distinct count on a column on all those files.Can anyone please help me with this.
Last edited by Don Cragun; 08-28-2017 at 04:00 AM..
Reason: Change HTML tags to CODE tags.
Hi,
I have a challenging task,in which i have to find the duplicate files by its name and size,then i need to take anyone of the file.Then i need to open the file and find for more than one pattern and count of that pattern.
Note:These are the samples of two files,but i can have more... (2 Replies)
Ok, another fun hiccup in my UNIX learning curve. I am trying to count the number of occurrences of an IP address across multiple files named example.hits. I can extract the number of occurrences from the files individually but when you use grep -c with multiple files you get the output similar to... (5 Replies)
Seems like can use awk and perl command. But I don't have the idea to write the command line. Thanks for all of your advise.
For example, if I have the file whose content are:
Sample 1. ATAGCAGAGGGAGTGAAGAGGTGGTGGGAGGGAGCT
Sample 2. ACTTTTATTTGAATGTAATATTTGGGACAATTATTC
Sample 3.... (1 Reply)
how to count the total number of lines of all the files under a directory using perl script..
I mean if I have 10 files under a directory then I want to count the total number of lines of all the 10 files contain. Please help me in writing a perl script on this. (5 Replies)
Please advice how can we search for a string say (abc) in multiple files and to get total occurrence of that searched string. (Need number of records that exits in period of time).
File look like this (read as filename.yyyymmdd)
a.20100101
b.20100108
c.20100115
d.20100122
e.20100129... (2 Replies)
i want to find the no:of occurrences of a word in a file
cat 1.txt
unix script unix script
unix script unix script unix script unix script
unix script unix script unix
unix
script
unix script unix script now i want to find , how many times 'unix' was occurred
please help me
thanks... (6 Replies)
Hi Guys,
I need to write a script to compare the count of two csv files each having 5 columns.
Everyday a csv file is recived.
Now we need to compare the count of todays csv file with yesterday's csv file and if the total count of records is same in todays csv file and yesterday csv file out... (3 Replies)
Hi,
Very good wishes to all!
Please help to provide the shell script for generating the record counts in filed wise from the .csv file
My question:
Source file:
Field1 Field2 Field3
abc 12f sLm
1234 hjd 12d
Hyd 34
Chn
My target file should generate the .csv file with the... (14 Replies)
Hi,
I have a .dat file with contents like the below:
Input file
============SEQ NO-1: COLUMN1==========
9835619
7152815
============SEQ NO-2: COLUMN2 ==========
7615348
7015548
9373086
============SEQ NO-3: COLUMN3===========
9373086
Expected Output: (I just... (1 Reply)
Hello All,
just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table.
Can you please suggest why? or any better idea?
export FILE="/abc/autom/file/geo_JOB.csv"
Export= `sqlplus -s dev01/password@dEV3... (16 Replies)
Discussion started by: Hope
16 Replies
LEARN ABOUT DEBIAN
bb-csvinfo.cgi
BB-CSVINFO.CGI(1) General Commands Manual BB-CSVINFO.CGI(1)NAME
bb-csvinfo.cgi - CGI program to show host information from a CSV file
SYNOPSIS
bb-csvinfo.cgi
DESCRIPTION
bb-csvinfo.cgi is invoked as a CGI script via the bb-csvinfo.sh CGI wrapper. Based on the parameters it receives, it searches a comma-
separated file for the matching host, and presents the information found as a table.
bb-csvinfo.cgi is passed a QUERY_STRING environment variable with the following parameters:
key (string to search for, typically hostname)
column (columnnumber to search - default 0)
db (name of the CSV database file in $BBHOME/etc/, default hostinfo.csv)
delimiter (delimiter character for columns, default semi-colon)
CSV files are easily created from e.g. spreadsheets, by exporting them in CSV format. You should have one host per line, with the first
line containing the column headings. Despite their name, the default delimiter for CSV files is the semi-colon - if you need a different
delimiter, invoke bb-csvinfo.cgi with the "delimiter=<character>" in the query string.
Example usage
This example shows how you can use the bb-csvinfo CGI. It assumes you have a CSV-formatted file with information about the hosts stored as
$BBHOME/etc/hostinfo.csv, and the hostname is in the first column of the file.
Use with the bbgen --docurl
The --docurl option to bbgen(1) sets up all of the hostnames on your Xymon webpages to act as links to a CGI script. To invoke the
bb-csvinfo CGI script, run bbgen with the option
--docurl=/cgi-bin/bb-csvinfo.sh?db=hostinfo.csv&key=%s
SEE ALSO bb-hosts(5), hobbitserver.cfg(5), bbgen(1)Xymon Version 4.2.3: 4 Feb 2009 BB-CSVINFO.CGI(1)