Hi Don ,
PFB the sample input files in a directory.
Code:
[omnidevint@sftp311 full_01jan13_31aug15]$ ls -ltr
total 800068
-rw-rw-r--. 1 omnidevint omnidevint 97184275 Aug 25 15:38 Final_Customer_Data_2013_2015_0.csv
-rw-rw-r--. 1 omnidevint omnidevint 88463636 Aug 25 15:38 Final_Customer_Data_2013_2015_1.csv
-rw-rw-r--. 1 omnidevint omnidevint 89536908 Aug 25 15:38 Final_Customer_Data_2013_2015_2.csv
-rw-rw-r--. 1 omnidevint omnidevint 107937776 Aug 25 15:38 Final_Customer_Data_2013_2015_3.csv
-rw-rw-r--. 1 omnidevint omnidevint 161423700 Aug 25 15:38 Final_Customer_Data_2013_2015_4.csv
-rw-rw-r--. 1 omnidevint omnidevint 129165088 Aug 25 15:38 Final_Customer_Data_2013_2015_5.csv
-rw-rw-r--. 1 omnidevint omnidevint 117823518 Aug 25 15:38 Final_Customer_Data_2013_2015_6.csv
-rw-rw-r--. 1 omnidevint omnidevint 27721536 Aug 25 15:38 Final_Customer_Data_2013_2015_7.csv
The contents of each files like below where the field 38 contains customer_id.
Code:
head -3 Final_Customer_Data_2013_2015_0.csv
first_name|last_name|email_address|brand|store_no|sales_associate_no|create_date|create_source|address_type_code|address_1|address_2|address_3|address_4|address_5|post_code|telephone_no|country_code|gvisa_flag|marital_status|gender|title|mobile_no|phone_opt_in_flag|mail_opt_in_flag|email_type|opt_in_flag|transaction_id|bm_customer_id|address_Active_Flag|mail_opt_in_date|phone_opt_in_date|date_last_modified|membership_date|head_of_household_flag|email_opt_in_date|points_current_balance|rewards_id|customer_id
Paula|Frediani|plfrediani@comcast.net|GYM|8888|9999|07/22/1999|STORE|BILLING|1906 Miller Ave||Belmont|CA||94002 1765|6506545255|US||U|U||6506545255|0|1|Rewards Signup||||1|02/17/2007||11/12/2016|06/18/2010|1||133|144RG2JS7F31735|58
Valerie|Burkart|valzb@sbcglobal.net|GYM|8888|9999|08/05/1999|STORE|BILLING|UNKNOWN||PLEASANTON|CA||94566|9254850902|US||U|U||9254850902|0|1|Rewards Signup|||2253998879592428|1|02/17/2007||03/18/2012|03/17/2012|1||239|1AS4V98V8G31735|255
And we are working in linux operating system and in bash shell.
Code:
Linux sftp311 3.10.0-327.el7.x86_64 #1 SMP Thu Oct 29 17:29:29 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
And we need the distinct customer_id count from all the csv files in the directory.Based on these info ,if you provide me any script ,that will be helpful.Thanks in advance.
Hi,
I have a challenging task,in which i have to find the duplicate files by its name and size,then i need to take anyone of the file.Then i need to open the file and find for more than one pattern and count of that pattern.
Note:These are the samples of two files,but i can have more... (2 Replies)
Ok, another fun hiccup in my UNIX learning curve. I am trying to count the number of occurrences of an IP address across multiple files named example.hits. I can extract the number of occurrences from the files individually but when you use grep -c with multiple files you get the output similar to... (5 Replies)
Seems like can use awk and perl command. But I don't have the idea to write the command line. Thanks for all of your advise.
For example, if I have the file whose content are:
Sample 1. ATAGCAGAGGGAGTGAAGAGGTGGTGGGAGGGAGCT
Sample 2. ACTTTTATTTGAATGTAATATTTGGGACAATTATTC
Sample 3.... (1 Reply)
how to count the total number of lines of all the files under a directory using perl script..
I mean if I have 10 files under a directory then I want to count the total number of lines of all the 10 files contain. Please help me in writing a perl script on this. (5 Replies)
Please advice how can we search for a string say (abc) in multiple files and to get total occurrence of that searched string. (Need number of records that exits in period of time).
File look like this (read as filename.yyyymmdd)
a.20100101
b.20100108
c.20100115
d.20100122
e.20100129... (2 Replies)
i want to find the no:of occurrences of a word in a file
cat 1.txt
unix script unix script
unix script unix script unix script unix script
unix script unix script unix
unix
script
unix script unix script now i want to find , how many times 'unix' was occurred
please help me
thanks... (6 Replies)
Hi Guys,
I need to write a script to compare the count of two csv files each having 5 columns.
Everyday a csv file is recived.
Now we need to compare the count of todays csv file with yesterday's csv file and if the total count of records is same in todays csv file and yesterday csv file out... (3 Replies)
Hi,
Very good wishes to all!
Please help to provide the shell script for generating the record counts in filed wise from the .csv file
My question:
Source file:
Field1 Field2 Field3
abc 12f sLm
1234 hjd 12d
Hyd 34
Chn
My target file should generate the .csv file with the... (14 Replies)
Hi,
I have a .dat file with contents like the below:
Input file
============SEQ NO-1: COLUMN1==========
9835619
7152815
============SEQ NO-2: COLUMN2 ==========
7615348
7015548
9373086
============SEQ NO-3: COLUMN3===========
9373086
Expected Output: (I just... (1 Reply)
Hello All,
just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table.
Can you please suggest why? or any better idea?
export FILE="/abc/autom/file/geo_JOB.csv"
Export= `sqlplus -s dev01/password@dEV3... (16 Replies)
Discussion started by: Hope
16 Replies
LEARN ABOUT DEBIAN
csv2mipe
CSV2MIPE(1) User Contributed Perl Documentation CSV2MIPE(1)NAME
csv2mipe.pl - Generates MIPE file based on 3 tab-delimited files
based on MIPE version v1.1
arguments: * tab-delimited file with PCR-level data
* tab-delimited file with SNP-level data
* tab-delimited file with assay-level data
Columns in file with PCR-level data:
pcr_id
pcr_modified (might be multiple, divided by semi-colon ";")
pcr_project (might be multiple, divided by semi-colon ";")
pcr_researcher (might be multiple, divided by semi-colon ";")
pcr_species
source_type
source_id
design_seq
primer1_oligo
primer1_seq
primer1_tm
primer2_oligo
primer2_seq
primer2_tm
design_remark (might be multiple, divided by semi-colon ";")
use_seq
use_revcomp
use_remark (might be multiple, divided by semi-colon ";")
pcr_remark (might be multiple, divided by semi-colon ";")
Columns in file with SNP-level data:
pcr_id
snp_id
snp_pos
snp_amb
snp_remark (might be multiple, divided by semi-colon ";")
Columns in file with assay-level data:
pcr_id
snp_id
assay_id
assay_type
assay_enzyme
assay_oligo
assay_specific
assay_tail
assay_strand
assay_remark (might be multiple, divided by semi-colon ";")
SYNOPSIS
csv2mipe.pl <pcr_file.csv> <snp_file.csv> <assay_file.csv>
ADDITIONAL INFO
See http://mipe.sourceforge.net
AUTHOR
Jan Aerts (jan.aerts@bbsrc.ac.uk)
perl v5.14.2 2005-07-20 CSV2MIPE(1)