08-28-2017
Hi ,
Please find below my response :
All the csv files are not sorted on field 38 but field 38 contains only numerical values.
Each distinct value appear only in one file.Each distinct value won't be there in multiple files.
We are not processing 3 files only.It can be around 100-120 files.
As all the input files(around 120) can contains data in millions.If system data size exceeds 400gb ,there is chance if system blown up ,that's why we not merging all the files.
We need total distinct count on field 38 on all the files present on the directory.
As I'm relatively new to the scripting area ,can you please help me with the above requirement.Your help will be very much appreciated.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I have a challenging task,in which i have to find the duplicate files by its name and size,then i need to take anyone of the file.Then i need to open the file and find for more than one pattern and count of that pattern.
Note:These are the samples of two files,but i can have more... (2 Replies)
Discussion started by: jerome Sukumar
2 Replies
2. UNIX for Dummies Questions & Answers
Ok, another fun hiccup in my UNIX learning curve. I am trying to count the number of occurrences of an IP address across multiple files named example.hits. I can extract the number of occurrences from the files individually but when you use grep -c with multiple files you get the output similar to... (5 Replies)
Discussion started by: MrAd
5 Replies
3. Shell Programming and Scripting
Seems like can use awk and perl command. But I don't have the idea to write the command line. Thanks for all of your advise.
For example, if I have the file whose content are:
Sample 1. ATAGCAGAGGGAGTGAAGAGGTGGTGGGAGGGAGCT
Sample 2. ACTTTTATTTGAATGTAATATTTGGGACAATTATTC
Sample 3.... (1 Reply)
Discussion started by: patrick chia
1 Replies
4. Shell Programming and Scripting
how to count the total number of lines of all the files under a directory using perl script..
I mean if I have 10 files under a directory then I want to count the total number of lines of all the 10 files contain. Please help me in writing a perl script on this. (5 Replies)
Discussion started by: adityam
5 Replies
5. Shell Programming and Scripting
Please advice how can we search for a string say (abc) in multiple files and to get total occurrence of that searched string. (Need number of records that exits in period of time).
File look like this (read as filename.yyyymmdd)
a.20100101
b.20100108
c.20100115
d.20100122
e.20100129... (2 Replies)
Discussion started by: zooby
2 Replies
6. Shell Programming and Scripting
i want to find the no:of occurrences of a word in a file
cat 1.txt
unix script unix script
unix script unix script unix script unix script
unix script unix script unix
unix
script
unix script unix script now i want to find , how many times 'unix' was occurred
please help me
thanks... (6 Replies)
Discussion started by: mahesh1987
6 Replies
7. Shell Programming and Scripting
Hi Guys,
I need to write a script to compare the count of two csv files each having 5 columns.
Everyday a csv file is recived.
Now we need to compare the count of todays csv file with yesterday's csv file and if the total count of records is same in todays csv file and yesterday csv file out... (3 Replies)
Discussion started by: Vivekit82
3 Replies
8. Shell Programming and Scripting
Hi,
Very good wishes to all!
Please help to provide the shell script for generating the record counts in filed wise from the .csv file
My question:
Source file:
Field1 Field2 Field3
abc 12f sLm
1234 hjd 12d
Hyd 34
Chn
My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies
9. Shell Programming and Scripting
Hi,
I have a .dat file with contents like the below:
Input file
============SEQ NO-1: COLUMN1==========
9835619
7152815
============SEQ NO-2: COLUMN2 ==========
7615348
7015548
9373086
============SEQ NO-3: COLUMN3===========
9373086
Expected Output: (I just... (1 Reply)
Discussion started by: MS06
1 Replies
10. UNIX for Beginners Questions & Answers
Hello All,
just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table.
Can you please suggest why? or any better idea?
export FILE="/abc/autom/file/geo_JOB.csv"
Export= `sqlplus -s dev01/password@dEV3... (16 Replies)
Discussion started by: Hope
16 Replies
LEARN ABOUT ULTRIX
indxbib
lookbib(1) General Commands Manual lookbib(1)
Name
indxbib, lookbib - build inverted index for a bibliography, lookup bibliographic references
Syntax
indxbib database...
lookbib database
Description
The makes an inverted index to the named databases (or files) for use by and These files contain bibliographic references (or other kinds
of information) separated by blank lines.
A bibliographic reference is a set of lines, constituting fields of bibliographic information. Each field starts on a line beginning with
a ``%'', followed by a key-letter, then a blank, and finally the contents of the field, which may continue until the next line starting
with ``%''.
The command is a shell script that calls and The first program, truncates words to 6 characters, and maps upper case to lower case. It
also discards words shorter than 3 characters, words among the 100 most common English words, and numbers (dates) < 1900 or > 2000. These
parameters can be changed. The second program, inv, creates an entry file (.ia), a posting file (.ib), and a tag file (.ic), all in the
working directory.
The command uses an inverted index made by to find sets of bibliographic references. It reads keywords typed after the ``>'' prompt on the
terminal, and retrieves records containing all these keywords. If nothing matches, nothing is returned except another ``>'' prompt.
It is possible to search multiple databases, as long as they have a common index made by In that case, only the first argument given to is
specified to
If does not find the index files (the .i[abc] files), it looks for a reference file with the same name as the argument, without the suf-
fixes. It creates a file with a '.ig' suffix, suitable for use with It then uses this fgrep file to find references. This method is sim-
pler to use, but the .ig file is slower to use than the .i[abc] files, and does not allow the use of multiple reference files.
Files
x.ia, x.ib, x.ic, where x is the first argument, or if these are not present, then x.ig, x
See Also
addbib(1), lookbib(1), refer(1), roffbib(1), sortbib(1),
lookbib(1)