Sponsored Content
Top Forums Shell Programming and Scripting File name and format validation Post 302390847 by vsmeruga on Friday 29th of January 2010 07:54:14 AM
Old 01-29-2010
File name and format validation

Hi Gurus,

I used unix long time back. I need help for writing a unix script which can be automated and execute every day on specific time.

1.) This is the actual functional requirement.
Informatica should reject incoming files that have invalid filenames or file formats

2.) My File name will be of below format:
Code:
<interface_name>_<source_name>_<template-type>_<sequence_number>_<datetime-stamp>.csv

e.g. VJA_AP_BRSIN_00001_2601201012316.csv

3.) For eg: if I have the below fields in my file
Code:
SiteID
Num(3) 
ICOMS Action
TBD
SKU
Char(50) 
Serial
Char(16)

I should validate the name of the file and also its format. For this, a shell script has to be written. Every day, i see 50 files in my incoming directory.

I feel, The level of field validation should be
A) Count of fields in a file should be matched
B) Datatypes and field value size should match
C) if any NOT NULL fields coming as NULL - check

I appreciate if some one could help me ASAP.

---------- Post updated at 07:54 AM ---------- Previous update was at 07:49 AM ----------


Hi All,

I also need to check whether the file has a header , detail records and footer in my validation

Last edited by Scott; 01-30-2010 at 06:40 AM.. Reason: Removed formatting, added code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Unix File Validation! Help

Hi All, I got a file with 3 fields delimited by hyphen "-". I have to validate & cleanse the data before i begine the processing Requirements 1. No record should contain more than 2 delimiters 2. No record should even contain less than 2 delimiters 3. Any records that matches rule 1 &... (8 Replies)
Discussion started by: minnuverma
8 Replies

2. UNIX for Dummies Questions & Answers

File Validation

Hi All, I am new to UNIX scripting. I need to validate a file. I need to check whether the file has a header , detail records and footer. If all the file is good I need to create a status file with the status 'Y' else 'N'. I have pasted the an example of the file below: ... (5 Replies)
Discussion started by: kumar66
5 Replies

3. UNIX for Dummies Questions & Answers

file name with timestamp validation

Hi , I am trying to write a shell script which will validate all .csv file names in a directory , my file name format is as below. CDR_SCAN_Report_YYYYMMDDHHMI.csv 1. I need to validate the name should start with CDR_SCAN_Report. 2. And the time stamp part is a valid time stamp,if it is... (1 Reply)
Discussion started by: shruthidwh
1 Replies

4. Shell Programming and Scripting

File Name Validation

Hi All, I am trying to validate the file name against the naming convention mentioned in configuration file. In the configuration file, the file name convention is mentioned as: Myfile_SQ<NN>_<NN>_YYYYMMDD_HHMMSS.xml The actual file received is Myfile_SQ10_30_20110423_073002.xml How do... (1 Reply)
Discussion started by: angshuman
1 Replies

5. Shell Programming and Scripting

File validation

Hello, File contains 1,3 and 5 are required columns. it's working use this command. awk -F\| '$1 && $3 && $5' test1.txt > test2.txt How can use this unix programming.while using runnign this scirpt,it's raising the error. awk: ^ syntax error #!/usr/bin/ksh `awk -F\| '$1 &&... (3 Replies)
Discussion started by: bammidi
3 Replies

6. Shell Programming and Scripting

XML file validation

Hi, i am new to unix script. i have two xml files. one is orderinfo.xml and another one is xsd file(EmailMessage.xml). we need to compare the both file for poper nodes exists are not. for example: <xsd:element name="EmailMessage"> tag in the EmailMessage.xml file(xsd sheet),this tag... (2 Replies)
Discussion started by: bmk
2 Replies

7. Shell Programming and Scripting

File validation

Hi there, As a part of file validation, I needed to check for delimiter count in the file. My aim is to find, how many records have failed to have predefined numbers of delimiters in the file. My code looks like below i=`awk -F '|' 'NF != 2 {print NR, $0} ' ${pinb_fldr}/${pfile}DAT |... (3 Replies)
Discussion started by: anandapani
3 Replies

8. Shell Programming and Scripting

Validation of date from file name

I'm writing a shell script for cleanup of older files from various sub-directories inside a main directory The structure of directories is as below: Logs daily online archive weekly online archive... (1 Reply)
Discussion started by: asyed
1 Replies

9. UNIX for Dummies Questions & Answers

Validation before moving the File

Hi, Before moving the files from source directory to target directory I need to check if the files have these strings 'Error', 'Not Valid' ,' YTF-' or if the file is a zero byte file, if the file contains these strings or if it is a zero byte file i should log a entry in the log file and fail... (5 Replies)
Discussion started by: gaur.deepti
5 Replies

10. UNIX for Beginners Questions & Answers

Flat File Validation

Hi Team there is a requirement to do validate on Flat files using Shell Scripting. Suppose we have two flat files ( F1, F2). Validation 1. Row count between F1 and F2 a) Write it in a file with results of count of each file and differences if any 2. Apply checksum or any... (1 Reply)
Discussion started by: shlearner
1 Replies
kstat(1M)						  System Administration Commands						 kstat(1M)

NAME
kstat - display kernel statistics SYNOPSIS
kstat [-lpq] [-T u | d ] [-c class] [-m module] [-i instance] [-n name] [-s statistic] [interval [count]] kstat [-lpq] [-T u | d ] [-c class] [module:instance:name:statistic...] [interval [count]] DESCRIPTION
The kstat utility examines the available kernel statistics, or kstats, on the system and reports those statistics which match the criteria specified on the command line. Each matching statistic is printed with its module, instance, and name fields, as well as its actual value. Kernel statistics may be published by various kernel subsystems, such as drivers or loadable modules; each kstat has a module field that denotes its publisher. Since each module may have countable entities (such as multiple disks associated with the sd(7D) driver) for which it wishes to report statistics, the kstat also has an instance field to index the statistics for each entity; kstat instances are numbered starting from zero. Finally, the kstat is given a name unique within its module. Each kstat may be a special kstat type, an array of name-value pairs, or raw data. In the name-value case, each reported value is given a label, which we refer to as the statistic. Known raw and special kstats are given statistic labels for each of their values by kstat; thus, all published values can be referenced as module:instance:name:statistic. When invoked without any module operands or options, kstat will match all defined statistics on the system. Example invocations are pro- vided below. All times are displayed as fractional seconds since system boot. OPTIONS
The tests specified by the following options are logically ANDed, and all matching kstats will be selected. A regular expression containing shell metacharacters must be protected from the shell by enclosing it with the appropriate quotes. The argument for the -c, -i, -m, -n, and -s options may be specified as a shell glob pattern, or a Perl regular expression enclosed in '/' characters. -c class Displays only kstats that match the specified class. class is a kernel-defined string which classifies the "type" of the kstat. -i instance Displays only kstats that match the specified instance. -l Lists matching kstat names without displaying values. -m module Displays only kstats that match the specified module. -n name Displays only kstats that match the specified name. -p Displays output in parseable format. All example output in this document is given in this format. If this option is not specified, kstat produces output in a human-readable, table format. -q Displays no output, but return appropriate exit status for matches against given criteria. -s statistic Displays only kstats that match the specified statistic. -T d | u Displays a time stamp before each statistics block, either in ctime(3C) format ('d') or as an alphanumeric representation of the value returned by time(2) ('u'). OPERANDS
The following operands are supported: module:instance:name:statistic Alternate method of specifying module, instance, name, and statistic as described above. Each of the mod- ule, instance, name, or statistic specifiers may be a shell glob pattern or a Perl regular expression enclosed by '/' characters. It is possible to use both specifier types within a single operand. Leaving a specifier empty is equivalent to using the '*' glob pattern for that specifier. interval The number of seconds between reports. count The number of reports to be printed. EXAMPLES
In the following examples, all the command lines in a block produce the same output, as shown immediately below. The exact statistics and values will of course vary from machine to machine. Example 1: Using the kstat Command example$ kstat -p -m unix -i 0 -n system_misc -s 'avenrun*' example$ kstat -p -s 'avenrun*' example$ kstat -p 'unix:0:system_misc:avenrun*' example$ kstat -p ':::avenrun*' example$ kstat -p ':::/^avenrun_d+min$/' unix:0:system_misc:avenrun_15min 3 unix:0:system_misc:avenrun_1min 4 unix:0:system_misc:avenrun_5min 2 Example 2: Using the kstat Command example$ kstat -p -m cpu_stat -s 'intr*' example$ kstat -p cpu_stat:::/^intr/ cpu_stat:0:cpu_stat0:intr 29682330 cpu_stat:0:cpu_stat0:intrblk 87 cpu_stat:0:cpu_stat0:intrthread 15054222 cpu_stat:1:cpu_stat1:intr 426073 cpu_stat:1:cpu_stat1:intrblk 51 cpu_stat:1:cpu_stat1:intrthread 289668 cpu_stat:2:cpu_stat2:intr 134160 cpu_stat:2:cpu_stat2:intrblk 0 cpu_stat:2:cpu_stat2:intrthread 131 cpu_stat:3:cpu_stat3:intr 196566 cpu_stat:3:cpu_stat3:intrblk 30 cpu_stat:3:cpu_stat3:intrthread 59626 Example 3: Using the kstat Command example$ kstat -p :::state ':::avenrun*' example$ kstat -p :::state :::/^avenrun/ cpu_info:0:cpu_info0:state on-line cpu_info:1:cpu_info1:state on-line cpu_info:2:cpu_info2:state on-line cpu_info:3:cpu_info3:state on-line unix:0:system_misc:avenrun_15min 4 unix:0:system_misc:avenrun_1min 10 unix:0:system_misc:avenrun_5min 3 Example 4: Using the kstat Command example$ kstat -p 'unix:0:system_misc:avenrun*' 1 3 unix:0:system_misc:avenrun_15min 15 unix:0:system_misc:avenrun_1min 11 unix:0:system_misc:avenrun_5min 21 unix:0:system_misc:avenrun_15min 15 unix:0:system_misc:avenrun_1min 11 unix:0:system_misc:avenrun_5min 21 unix:0:system_misc:avenrun_15min 15 unix:0:system_misc:avenrun_1min 11 unix:0:system_misc:avenrun_5min 21 Example 5: Using the kstat Command example$ kstat -p -T d 'unix:0:system_misc:avenrun*' 5 2 Thu Jul 22 19:39:50 1999 unix:0:system_misc:avenrun_15min 12 unix:0:system_misc:avenrun_1min 0 unix:0:system_misc:avenrun_5min 11 Thu Jul 22 19:39:55 1999 unix:0:system_misc:avenrun_15min 12 unix:0:system_misc:avenrun_1min 0 unix:0:system_misc:avenrun_5min 11 Example 6: Using the kstat Command example$ kstat -p -T u 'unix:0:system_misc:avenrun*' 932668656 unix:0:system_misc:avenrun_15min 14 unix:0:system_misc:avenrun_1min 5 unix:0:system_misc:avenrun_5min 18 EXIT STATUS
The following exit values are returned: 0 One or more statistics were matched. 1 No statistics were matched. 2 Invalid command line options were specified. 3 A fatal error occurred. FILES
/dev/kstat kernel statistics driver ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ |ATTRIBUTE TYPE |ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
sh(1), time(2), ctime(3C)gmatch(3GEN), kstat(3KSTAT), attributes(5), kstat(7D), sd(7D), kstat(9S) NOTES
If the pattern argument contains glob or Perl RE metacharacters which are also shell metacharacters, it will be necessary to enclose the pattern with appropriate shell quotes. SunOS 5.10 14 Oct 2004 kstat(1M)
All times are GMT -4. The time now is 11:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy