Sponsored Content
Top Forums Shell Programming and Scripting Need a ready Shell script to validate a high volume data file Post 302859187 by Guruprasad on Wednesday 2nd of October 2013 02:19:50 PM
Old 10-02-2013
Need a ready Shell script to validate a high volume data file

Hi,

I am looking for a ready shell script that can help in loading and validating a high volume (around 4 GB) .Dat file . The data in the file has to be validated at each of its column, like the data constraint on each of the data type on each of its 60 columns and also a few other constraints on each of the column.

After validation, the rows wiith error records need to be written to an exception report and the correct data has to be loaded separately.

If anyone has an idea or any ready to use shell script which can customised for my requirement it will be greatly appreciated.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Want to execute rest of the script after the file is ready ...

Hi All I have a requirement like, where a file gets generated in a particular dir and once the file is ready and available then I want to execute rest of the script, because untill and unless the file exists and is available there is no use of running rest of the commands in that script. ... (5 Replies)
Discussion started by: csaha
5 Replies

2. Shell Programming and Scripting

Better way to Validate column data in file.

I am trying to validate the third column in a pipe delimited file. The column must be 10 char long and all digits 0-9. I am writing out two new files from the existing file, if it would be quicker, I could leave the bad rows in the file and ignore them in the next process. What I have is... (12 Replies)
Discussion started by: barry1
12 Replies

3. Shell Programming and Scripting

The scope of the shell/perl script is to read the input text file. Validate the expiry date of each

The scope of the shell/perl script is to read the input text file. Validate the expiry date of each certificate and send the mail to the user. The user takes action to add the new certificate to the storage file and user owns the responsibility to update the input text file with the new certificate... (5 Replies)
Discussion started by: casmo
5 Replies

4. Shell Programming and Scripting

how to validate data of excel using perl or shell

I have a software which generates excel report with some specific data. The excel file format is .xls (old 2003 format) The data are in the forms like differenct cells contains numeric, string and alphanumeric data. The data per cell for specific input data is fixed. I need to retrive specific... (5 Replies)
Discussion started by: PratLinux
5 Replies

5. Shell Programming and Scripting

Validate file count in korn shell script

Hi, I have files in the directory like below which I need to validate if all the required files are present. A_B_001 of 002_time1.txt A_B_002 of 002_time1.txt A_B_001 of 001_time2.txt Scenarios- a)If file with 001 of 002_time1 or 002 of 002_time1 is missing in the folder,script should... (6 Replies)
Discussion started by: aneeta13
6 Replies

6. UNIX for Dummies Questions & Answers

Shell script to read lines in a text file and filter user data Shell Programming and Scripting

sxsaaas (3 Replies)
Discussion started by: VikrantD
3 Replies

7. Shell Programming and Scripting

Script to ingest a csv, validate data and insert into Oracle

Hi all i would appreciate your help... I am looking for a set of unix commands which i can use to 1) ingest a csv file with a known format 2) validate the filename 3) validate the data/datatypes 4) Insert into an oracle db Can you help get me started? yogz888 (1 Reply)
Discussion started by: yogz888
1 Replies

8. Shell Programming and Scripting

Output large volume of data to CSV file

I have a program that output the ownership and permission on each directory and file on the server to a csv file. I am getting error message when I run the program. The program is not outputting to the csv file. Error: the file access permissions do not allow the specified action cannot... (2 Replies)
Discussion started by: dellanicholson
2 Replies

9. Shell Programming and Scripting

Bash to verify and validate file header and data type

The below bash is a file validation check executed that will verify the correct header count of 10 and the correct data type in each field of the tab-delimited file. The key has the data type of each field in it. My real data has 58 headers in it but only the header and next row need to be... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

Shell Script to validate the access for Servers

Hi Experts, Please advise to Write a Shell script, To check , whether the user has access to UNIX PRODUCTION server or not? If no, then need to provide the access to UNIX PRODUCTION server and then need to provide access to print only few FILES. After connecting to ther server, again... (5 Replies)
Discussion started by: vasuvv
5 Replies
vgexport(1M)															      vgexport(1M)

NAME
vgexport - export an LVM volume group and its associated logical volumes SYNOPSIS
mapfile] outfile] vg_name mapfile vg_name DESCRIPTION
Using the format of the first command line of the above, the command can be used to remove a volume group from the system. The volume group will be removed without modifying the logical volume information found on the physical volumes. The volume group identified by vg_name is removed from the or file, and the associated device files including the vg_name directory and file are removed from the system. The volume group information and data is untouched on the physical volume. These disks can be imported to another system with the command (see vgimport(1M)). Scan Option Using the format of the second command line of the above, the command generates a mapfile that can be copied to other systems that are part of a high availability cluster (use the option if you do not want to remove the volume group from the system the command is being run from) and the command (see vgimport(1M)) can be used to recreate the volume group. See also vgchange(1M). The mapfile contains a description of the volume group and its associated logical volume(s) (if any). The logical volume information found on the physical volumes is not modi- fied. Options and Arguments recognizes the following options and arguments: vg_name The path name of the volume group. By default, a file named is created in the current directory. This file contains a description of the volume group and its associated logical volume(s) (if any). Use this option to specify a different name for the file, mapfile. This file can be used as input to (see vgimport(1M)). When used with the option, the volume group specified in the mapfile can be shared with other systems in the high availability cluster. Preview the actions to be taken but do not update the or file or remove the devices file. This option is best used in conjunction with the option. Print verbose messages including the names of the physical volumes associated with this volume group. Scan option. When the option is specified, then the options must also be specified. A mapfile is created that can be used to cre- ate volume group entries on other systems in the high availability cluster (with the command). Write the current set of pv_paths for the volume group to the outfile. The outfile may then be used as the infile for the option. If used together with the option the volume group is not exported but the list of pv_paths is still written to the outfile. This may be useful to derive a list of pv_paths for the volume group or to use on another system which is sharing the volume group and which has an identical configuration. EXTERNAL INFLUENCES
Environment Variables determines the language in which messages are displayed. If is not specified or is null, it defaults to "C" (see lang(5)). If any internationalization variable contains an invalid setting, all internationalization variables default to "C" (see environ(5)). EXAMPLES
Export the volume group into mapfile The volume group will be removed from the exporting system. Export the volume group and write the disk names into the file Create a mapfile to be copied to other systems in a high availability cluster to build the volume group information for the volume group, Note that the volume group is not removed from the exporting system. The importing systems will create the volume group with the command using the and options. SEE ALSO
vgimport(1M), vgscan(1M). vgexport(1M)
All times are GMT -4. The time now is 03:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy