Sponsored Content
Top Forums Shell Programming and Scripting Script to validate header in a csv file Post 303018873 by panyam on Monday 18th of June 2018 04:05:18 AM
Old 06-18-2018
Something like below:

Code:
head -1 fileA | grep "$(cat fileB)" 
if [[ $? -ne 0 ]] ;
then
echo "fileA header does not match with fileB"
exit 1;
fi

Note: even if there is a difference with "spaces" match wont happen.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

validate csv file load

Hi All, I am using sqlldr to load my csv files into the database. The code in the sh script is as follows. sqlldr ${DBUSER}/${DBPASS}@${ORACLE_SID} \ data=myCSV.data \ bad=myCSV.bad \ control=myCSV.ctl \ ... (0 Replies)
Discussion started by: rahulrathod
0 Replies

2. Shell Programming and Scripting

How to validate a CSV file?

Hi. I think some people have already asked this, but the answers/questions seem to be about validating the contents inside a CSV file. I am simply after a simple variable solution (ie 0 = false, 1 = true) that I can use in my script to say that file so-and-so is actually a CSV file, or in some... (4 Replies)
Discussion started by: ElCaito
4 Replies

3. Shell Programming and Scripting

How can I add a header to a csv file

I have a csv file which has no header. the file has 15 fields and needs to go out with a header of 8 fields. The header content needs to have some variables and some fixed that i have set up: variable header fields OUTFILE_YEAR=`date '+%y'` DATE=`date '+%d%m%y'` TIME=`date '+%H:%M:%S'`... (6 Replies)
Discussion started by: Pablo_beezo
6 Replies

4. Shell Programming and Scripting

How to: Validate a CSV file using an XSD?

Hi All, I was wondering if there is a utility/perl library/way of validating the contents of a CSV file using an XSD. i.e. Customer csv (including Header & Trailer) Cust_num, Cust_nme, Cust_typ, Cust_act_dte, Cust_loc, 101,Joe's Pizza,Retail,10121979,Detroit, 102,Sony... (6 Replies)
Discussion started by: luinzi
6 Replies

5. UNIX for Dummies Questions & Answers

Merge all csv files in one folder considering only 1 header row and ignoring header of all others

Friends, I need help with the following in UNIX. Merge all csv files in one folder considering only 1 header row and ignoring header of all other files. FYI - All files are in same format and contains same headers. Thank you (4 Replies)
Discussion started by: Shiny_Roy
4 Replies

6. Shell Programming and Scripting

Script to validate file header and trailer

Hi, I need a script that validates a file header/detail/trailer. File layout is: Header - Rec_Type|File_name|File_Date Detail - Rec_Type|field1|field2|field3... Trailder - Rec_Type|File_name|File_Date|Record_count Sample Data: HDR|customer_data.dat|20120709... (7 Replies)
Discussion started by: ash_sh
7 Replies

7. Shell Programming and Scripting

Exclude the header row in the file to validate

Hi All, File contains header row.. we need to exclude the header row...no need to validate the first row in the file. Data in the file should take valid data(two columns)..we need to exclude the more than two columns in the file except the first line. email|firstname a|123|100 b|345... (4 Replies)
Discussion started by: bmk
4 Replies

8. Shell Programming and Scripting

Script to ingest a csv, validate data and insert into Oracle

Hi all i would appreciate your help... I am looking for a set of unix commands which i can use to 1) ingest a csv file with a known format 2) validate the filename 3) validate the data/datatypes 4) Insert into an oracle db Can you help get me started? yogz888 (1 Reply)
Discussion started by: yogz888
1 Replies

9. UNIX for Beginners Questions & Answers

Validate csv file

Hi guys, i want to validate the no.of colums in a csv file ,but if there is a comma(,) in any of the data values it should skip and count only valid (,) commas. e.g 1.abc,pqrs,1234,567,hhh result :4 2.abc,pqrs,1234,567,hhh,"in,valid",end12 result:6 here script should skip the comma inside... (10 Replies)
Discussion started by: harry123
10 Replies

10. Shell Programming and Scripting

Bash to verify and validate file header and data type

The below bash is a file validation check executed that will verify the correct header count of 10 and the correct data type in each field of the tab-delimited file. The key has the data type of each field in it. My real data has 58 headers in it but only the header and next row need to be... (6 Replies)
Discussion started by: cmccabe
6 Replies
fmerge(1)						      General Commands Manual							 fmerge(1)

NAME
fmerge - merge files SYNOPSIS
fmerge [ option... ] basefile fileA fileB fmerge -Help fmerge -VERSion DESCRIPTION
The fmerge program is used to compare the changes between two different descendants of a base file, and creates an output file which con- tains both sets of changes. This is useful when two users both take the same version of a file and make independent edits to it, and then later want to create a file which contains both sets of edits. In such a use, the original file that both sets of edits is derived from is called the base file. The two files containing the edits are called file A and file B. The command: fmerge basefile fileA fileB -o outputfile produces the output file which contains the edits contained in file A and file B, based on the base file . If the -Output option was not used, or if no outputfile is specified, then the merged lines are typed to the standard output. The order of specifying file A and file B is usually unimportant. The fmerge program can also be used to remove earlier edits made to a module. To do this, make the version containing the edits you want to delete be the basefile. Make the version previous to the edit you want deleted be file A. Finally, make the most recent version of the file which contains the other edits (including the one you want deleted) be file B. Then the result of merging will be the newest version of the module minus the changes made by the edit you wanted removed. For example, if three successive versions of some module have the names edit10, edit11 and edit12, and you want the changes done by edit11 to be undone, but still want the changes done by edit12, then you use the command: fmerge edit11 edit10 edit12 -o outputfile While merging the two sets of edits, fmerge may discover conflicts. A conflict occurs when the same line of the base file is changed by both of the two sets of edits. The change can be due to new lines being inserted, lines being deleted, or both. When conflicts occur, the output file contains conflict identification lines, which are lines containing the string '/-/-/-/'. These lines indicate the region where the two sets of edits are incompatible. You must then edit the output file and remove these lines, and in addition correct the conflicts manually in order to produce the correct result. OPTIONS
The following options are understood: -Conflicts [ conflictfile ] Since conflicts due to deletions are invisible in the output file, and inserts do not specify which of the two edits inserted the lines, there is an alternative output format from the fmerge program. This output format describes what happens to each line of the base file, so that conflicts are easier to detect and fix. The command: fmerge basefile fileA fileB -c conflictfile produces the file describing the results of the merge in detail. If the -Conflicts option is specified without any conflictfile name, then the conflicts are send to the standard output. If there are conflicts, and the -Conflicts options is not specified, the fmerge program will exit with a status of 1. The conflict file contains lines which contain three characters and then some text. The first three characters describe what is happening to the base file at that point. These characters are the following: IA This line was inserted by file A. DA This line was deleted by file A. IB This line was inserted by file B. DB This line was deleted by file B. <blanks> This line is unchanged. X This is a conflict identification line. U There are unspecified unchanged lines here. Each set of conflicts is flagged by three identification lines. The first line indicates the beginning of the conflict, and speci- fies the line numbers for the base file and two divergent files. The second conflict identification line separates lines changed by file A from lines changed by file B. The third conflict identification indicates the end of the conflict. You can edit this conflict file to remove the conflicts. This involves deleting the conflict identification lines, and changing the conflicting lines as necessary to fix the conflict. While doing this, remember to leave three blank characters at the front of any new lines you insert while correcting the conflicts. When you are done, there should be no lines which begin with an 'X' in the file. All other lines can remain. Then you can use the command: fmerge conflictfile -o outputfile to create the new output file which has the desired data. Once again, if no -Output option or outputfile is used, the output is send to the standard output. -Unchanged number Besides physical conflicts, there can be logical conflicts. These are changes made to different lines in the base file such that the program is no longer correct. Such conflicts cannot be detected by a program, and so these must be checked manually. In order to make this process easier, the -Unchanged option can be used to reduce the size of the conflict file to only include regions near changed lines. This file can then be examined in order to detect possible logical conflicts. As an example, the command: fmerge basefile fileA fileB -c -u 3 will send to the standard output all changes made by either sets of edits, with only three unchanged lines surrounding each edit. When using the -Unchanged option, the conflict file will contain lines starting with 'U'. These represent unchanged lines, and the number following the letter is the number of unchanged lines. The resulting conflict file cannot be read to produce an output file because of the missing lines. If this is attempted, an error will be generated. It is possible to use both -Output and -Conflicts in the same command. Thus you can produce the output file which you hope is cor- rect, and also produce the conflict file which you can use to check for logical conflicts. -Verbose [ number ] This option can be specified with any other action, and outputs status information about the progress of the action. This is use- ful for debugging of problems, or just for amusement when the system is slow or a large file is being processed. It accepts a numeric argument to indicate the verbosity for output. The levels are as follows: 0 No output at all (except for errors). 1 Single-line output describing action (default). 2 Detailed status as action proceeds. -Failures number This option restricts the number of physical conflicts that are allowed before failing. This is used if you are not interested in the results if there are too many conflicts. -Help Give some help on how to use the fmerge program. -Ignore Ignore all conflicts. -Ignore_Identical_Conflicts The option may be used to suppress conflicts which make identical deletes, or identical inserts, or identical changes. This is often desirable when merging two source code branches. -VERSion Show what version of fmerge is running. All options may be abbreviated; the abbreviation is documented as the upper case letters, all lower case letters and underscores (_) are optional. You must use consecutive sequences of optional letters. All options are case insensitive, you may type them in upper case or lower case or a combination of both, case is not important. For example: the arguments "-help, "-HELP" and "-h" are all interpreted to mean the -Help option. The argument "-hlp" will not be under- stood, because consecutive optional characters were not supplied. Options and other command line arguments may be mixed arbitrarily on the command line. The GNU long option names are understood. Since all option names for fmerge are long, this means ignoring the extra leading '-'. The "-option=value" convention is also understood. FILE NAME EXPANSION
As a convenience, if a pathname begins with a period and a environment variable exists with that name, then the value of the environment variable will be used as the actual pathname. For example, if a environment variable of .FOO has the value this.is.a.long.name, then the command fmerge -o .FOO is actually equivilant to the command fmerge -o this.is.a.long.name If you want to prevent the expansion of a pathname which begins with a period, then you can use an alternate form for the pathname, as in: fmerge -o ./.FOO BINARY FILES
In general, fmerge can handle all text files you throw at it, even international text with unusual encodings. However, fmerge is unable to cope elegantly with files which contain the NUL character. The fcomp(1) program simply prints a warning, and continues, you need to know that it converts NUL characters into an 0x80 value before performing the comparison. The fmerge(1) program also converts the NUL character to an 0x80 value before merging, after a warning, and any output file will contain this value, rather than the original NUL character. The fhist(1) program, however, generates a fatal error if any input file contains NUL characters. This is intended to protect your source files for unintentional corruption. Use -BINary for files which absolutely must contain NUL characters. EXIT STATUS
The fmerge program will exit with a status of 1 on any error. The fmerge program will only exit with a status of 0 if there are no errors. REFERENCES
This program is based on the algorithm in An O(ND) Difference Algorithm and Its Variations, Eugene W. Myers, TR 85-6, 10-April-1985, Department of Computer Science, The Uni- versity of Arizona, Tuscon, Arizona 85721. See also: A File Comparison Program, Webb Miller and Eugene W. Myers, Software Practice and Experience, Volume 15, No. 11, November 1985. COPYRIGHT
fmerge version 1.18.D001 Copyright (C) 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2008, 2009 Peter Miller; This program is derived from a work Copyright (C) 1990 David I. Bell. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. AUTHORS
Peter Miller Web: http://miller.emu.id.au/pmiller/ //* E-Mail: pmiller@opensource.org.au David I. Bell Web: http://www.canb.auug.org.au/~dbell E-Mail: dbell@canb.auug.org.au Reference Manual FHist fmerge(1)
All times are GMT -4. The time now is 11:25 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy