Sponsored Content
Top Forums Shell Programming and Scripting Creating a control file for a group of files Post 302441231 by Maya_Pillai on Friday 30th of July 2010 12:44:59 AM
Old 07-30-2010
Creating a control file for a group of files

Hi,

We have almost 45,000 data files created by a script daily. The file names are of format-ODS.POS.<pharmacyid>.<table name>.<timestamp>.dat. There will be one data file like this for each pharmacy and each table.(Totally around 45,000)

The requirement is to create a control file for each pharmacy id with file name, No.of rows , table name, timestamp and MD5 value.


The data files look like this -

Code:
ODS.POS.ABC89.ADT_LOG.07272010_033303.dat
ODS.POS.AEC12.ADT_LOG.07272010_033303.dat
ODS.POS.ABC78.ADT_LOG.07272010_033303.dat 
 
ODS.POS.AEC12.TR_ITM_CPN_TND.07272010_033303.dat
ODS.POS.AEC13.TR_ITM_CPN_TND.07272010_033303.dat
ODS.POS.ABC89.TR_ITM_CPN_TND.07272010_033303.dat
ODS.POS.ABC78.TR_ITM_CPN_TND.07272010_033303.dat


The requirement is to create a control file for each pharmacyid

Controlfile 1 -> for pharmacy id - ABC89

Control file 1 (ODS.POS.<Pharmacyid>.timestamp.ctl )should contain the filename, no.of rows, table name, time stamp and MD5 value for the below files.

Code:
ODS.POS.ABC89.ADT_LOG.07272010_033303.dat
ODS.POS.ABC89.TR_ITM_CPN_TND.07272010_033303.dat


Control file 2->for pharmacy id - AEC12

Control file 2 ((ODS.POS.<Pharmacyid>.<timestamp>.ctl )should contain the filename, no.of rows, table name, time stamp and MD5 value for the below files.

Code:
ODS.POS.AEC12.ADT_LOG.07272010_033303.dat
ODS.POS.AEC12.TR_ITM_CPN_TND.07272010_033303.dat


can somebody help on this?

Thanks
Maya

Last edited by Scott; 08-02-2010 at 06:16 AM.. Reason: Code tags, please...
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

listing files excluding files from control file

I have a directory named Project.I have a control file which contains valid list of files.I would like list the files from directory Project which contains files other than listed in the control file. Sample control file: TEST SEND SFFFILE CONTL The directory contains followign... (15 Replies)
Discussion started by: ukatru
15 Replies

2. UNIX for Dummies Questions & Answers

user control- supplementary group

What is a supplementary group and how do you make a supplementary group for members of a team? Thanks in advance. ---------- Post updated 05-10-11 at 12:09 PM ---------- Previous update was 05-09-11 at 10:08 PM ---------- anybody? (1 Reply)
Discussion started by: nlassiter
1 Replies

3. UNIX for Advanced & Expert Users

creating a secondary read only group with setfacl

We have created ACL's to allow two differnet groups to access some directories. You can see output from getfacl below. group::rwx group:rbauser:r-- The original group has full access, the secondary group has read only. However users in the secondary group can't see the directories. Think this... (1 Reply)
Discussion started by: dw82199
1 Replies

4. UNIX for Dummies Questions & Answers

Creating a file where the owner and group is not root

Hi, I'm the root user on my computer, but I'm writing a script that does a lot of file handling. Every time I create a file or directory it automatically requires root privileges. Is there a way I can just create a file that the user can access without a password? For example in my script I... (20 Replies)
Discussion started by: jdilts
20 Replies

5. Linux

Default user:group permissions while creating files and directories

Hi, I am working on setup a environment where only a specific user can upload the builds on htdocs of apache. Now i want that a specific user can copy the builds on htdocs folder. I created a group "deploy" and assign user1 and user2 to this group. On Apache side i mentioned User=deploy... (3 Replies)
Discussion started by: sunnysthakur
3 Replies

6. Shell Programming and Scripting

Creating two files out of one file

Hi members, I have a file with lots of text like this: asdsfkm dfsfomnow i want to extract only the rows with HUMAN as its 1st column for this I used : ruby -ne 'print if /^Human/; exit if /^TER/' $line | awk 'print $1, "\t"$2, "\t"$3, "\t"$4, "\t"$5}' >$line"new" This is creating a file... (1 Reply)
Discussion started by: CAch
1 Replies

7. Solaris

Big /etc/group creating issue

I have Solaris-10 with kernel patch 142900-03, Update 7 Sparc server. root@ddlomps40:/# uname -a SunOS ddlomps40 5.10 Generic_142900-03 sun4u sparc SUNW,SPARC-Enterprise root@ddlomps40:/# cat /etc/release Solaris 10 5/09 s10s_u7wos_08 SPARC Copyright 2009 Sun... (5 Replies)
Discussion started by: solaris_1977
5 Replies

8. HP-UX

Creating a group of users with script

Hi, I have a file with usernames, and the comment section, e.g : Data removed by request of sanchitadutta91, 20 May 2020 I need to add these users into a server. Is it possible to use a script to create the users, together with the comment ? From the commandline to add one user, the... (2 Replies)
Discussion started by: anaigini45
2 Replies

9. UNIX for Beginners Questions & Answers

Script for creating Control file in UNIX

Delete ---- Original post, restored by mod after being deleted by abhilashnair ---- I have a requirement where, I need to create a control file which will have 3 columns in the header row as below: Filename Count Checksum This above control file has to contain metadata as above... (2 Replies)
Discussion started by: abhilashnair
2 Replies

10. UNIX for Beginners Questions & Answers

Creating script to read many files and load into database via single control file

Hi, I have many files but with only 2 names , I want to load the data of that file into database through sqlldr with single control file. how can i do that ????? Example: switch_file switch_file billing_file billing_file now these files should be loaded into same database but different... (1 Reply)
Discussion started by: niti_sharma
1 Replies
ATSADC(1)							       local								 ATSADC(1)

NAME
atsadc, atsa1, atsaftp, atsahttp -- counter-collection SYNOPSIS
atsadc [ t n ] [ ofile ] atsa1 [ t n ] atsaftp atsahttp DESCRIPTION
System activity-data can be gathered on special request of a user [see atsar(1) ] or automatically, on a routine basis, as described here. Usually the kernel maintains statistical counters that are incremented as various system actions occur. These include counters for CPU uti- lization, disk utilization, memory utilization and various network statistics. The program atsadc and the shell-script atsa1 are used to collect, save, and process these counters. The program atsadc (the data collector) samples system data n times with an interval of t seconds between samples, and writes in binary format to ofile or (default) to standard output. The sampling interval t should be greater than 1 second. If t and n are omitted, a special reset-record is written. This facility is used when booting to a multi-user state, to mark the time at which the counters restart from zero. For example, the reset-mark can be added to the daily data by the command: /usr/local/bin/atsadc /var/log/atsar/atsa`date +%d` Note that this entry is written to the /etc/rc.d/init.d/atsar file. The shell-script atsa1 is used to collect and store data in the binary file /var/log/atsar/atsadd where dd is the current day of the month. The arguments t and n cause records to be written n times at an interval of t seconds, or once if omitted. Furthermore this script takes care that log-files older than a week are removed once a day. A file containing following entries should be added to the /etc/cron.d directory to produce records every 20 minutes during working hours and hourly otherwise: 0 * * * 0-6 root /usr/local/bin/atsa1 20,40 8-17 * * 1-5 root /usr/local/bin/atsa1 See crontab(1) for details. The shell-script atsaftp counts the new transfers registered in the FTP-logfile(s) since the previous time this script was activated; the new counters are stored in the /var/log/atsar/ftpstat file in ASCII-format. The names of the FTP-logfiles to be watched are specified in the /etc/atsar.conf configuration-file. The shell-script atsahttp counts the new transfers registered in the HTTP-logfile(s) since the previous time this script was activated; the new counters are stored in the /var/log/atsar/httpstat file in ASCII-format. The names of the HTTP-logfiles to be watched are specified in the /etc/atsar.conf configuration-file. Both scripts must be activated just before the program atsadc is started, which also collects these counters. FILES
/var/log/atsar/atsadd Daily data file, where dd are digits representing the day of the month. SEE ALSO
atsar(1), crontab(1) AUTHOR
Gerlof Langeveld, AT Computing (gerlof@ATComputing.nl) AT Computing July 2004 ATSADC(1)
All times are GMT -4. The time now is 05:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy