Hi,
Thanks all for your effort. Let me just put what exactly i am looking for;
I have attached an input file and here what i do from it;
Code :
cat -vt diffStreamVersions.txt |sed 's/SQL>//g' | sed 's/ //g' | tr ':' '\n' |tr '-' '\t' | tr '|' '\t' | sort -n |
/usr/local/bin/gawk -F";" '{OFS="\t";if(x!=$1){print "\n"$1;x=$1};print "\t"$3}'
Output is as follows;
Code :
CH_PPV_AUDIT_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!BehaviourOnError Abort
NEW!OutputFileNameExtractionMask
NEW!Subdirectory
NEW!TargetDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/${Affiliate}_PPV
NEW!TempDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/${Affiliate}_PPV/temp
OLD!BehaviourOnError Restart
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/CH_PPV
OLD!TempDirectory ${EventLinkUserHome}/delivery/UPC_PE_Rejected_Loader_stream/INPUT/CH_PPV/temp
CH_PPV_AUDIT_ENCODER
NEW#UPC_CFG_AUDIT_REJECTED_ENCODER 1.1.0.
OLD#UPC_CFG_CH_PPV_AUDIT_ENCODER 1.0.2.
NEW!OutputDescriptionFileName vod.xml
OLD!OutputDescriptionFileName audit_file_description.xml
NEW?AUDIT_REJECTED 0
NEW?RDC_AUDIT_OUT 0
OLD?AUDIT_REJECTED 0
OLD?RDC_AUDIT_OUT 0
OLD?VALIDATION_AUDIT_OUT 0
NEW@UPC_CFG_AUDIT_REJECTED_ENCODER REJ
OLD@UPC_CFG_CH_PPV_AUDIT_ENCODER REJ
CH_PPV_BACKUP
NEW#UPC_CFG_BACKUP 1.2.1.
NEWNODE
NEW#EL_APP_FTP_DISTRIBUTOR 2.1.4.upc1
NEW#EL_LIB_SFTP 1.1.0.
NEW!BehaviourOnError Abort
NEW!BlockSize
NEW!Cipher
NEW!Compression
NEW!CompressionMethod gzip
NEW!ConnectionEstablishmentTimeout
NEW!ConnectionRetryCount
NEW!ConnectionRetryInterval
NEW!DailyDirectories no
NEW!DestinationHost
NEW!ExternalCommand
NEW!FlagFileSuffix
NEW!InputLinkDirectories Yes
NEW!InputLinksToSubDirectories CAS_COLLECTOR=RAW/&SOURCEID,COLLECTED=RAW/&SOURCEID
NEW!Kex
NEW!MaxDays 0
NEW!MaxFiles 0
NEW!MaximumReconnectionInterval
NEW!MaximumSustainedNetworkDowntime
NEW!OriginalFilenameMask
NEW!Password
NEW!Port
NEW!PrivateKey
NEW!PublicKey
NEW!RemoteOS
NEW!RemotePrefix
NEW!RemoteSuffix
NEW!Ruleset backup.pm
NEW!SourceIdDirectories no
NEW!TargetDirectory /backup/BACKUP_EL603/${StreamName}
NEW!TempDirectory
NEW!TempSuffix
NEW!TransferMode
NEW!TransferProtocol Local
NEW!TransferRetryCount
NEW!TransferRetryInterval
NEW!TransferType
NEW!Username
NEW?CAS_COLLECTOR 1
NEW?COLLECTED 1
NEW?ENCODED_AUDIT 1
NEW?ENCODED_DWH 1
NEW?ENCODED_KENAN 1
CH_PPV_CAS_COLLECTOR
NEW#UPC_CFG_CAS_COLLECTOR 2.1.0.
OLD#UPC_CFG_CAS_COLLECTOR 2.0.0.
NEW#UPC_APP_CAS_COLLECTOR 2.3.0.
OLD#UPC_APP_CAS_COLLECTOR 2.2.4.
NEW!BindTimeout 10s
NEW!CasHeartbeatInterval 5m
NEW!CloseFilesAtSchedule 0
NEW!CommitInterval 10s
NEW!MaxAgeOfFile 30m
NEW!NoDataTimeout 12h
NEW!OutputFileNameCreationMask ${Affiliate}CAS_&CREATIONTIME(%Y%m%d%H%M%S)
NEW!RecordLimit 10000
NEW!RestartInterval 1d
NEW!TransactionIdFile ${EventLinkUserHome}/delivery/${StreamName}/WORKING/id_file.txt
OLD!BindTimeout 10
OLD!CasHeartbeatInterval 300
OLD!CommitInterval 10
OLD!OutputFileNameCreationMask CHCAS_&CREATIONTIME(%Y%m%d%H%M%S)
OLD!TransactionIdFile ${EventLinkUserHome}/delivery/UPCCH_PPV/WORKING/id_file.txt
CH_PPV_DECODER
NEW#UPC_CFG_CAS_DECODER 1.0.0.
OLD#UPC_CFG_CH_PPV_CAS_DECODER 1.0.0.
NEW#EL_APP_ASCII_DECODER 3.0.3.upc5
OLD#EL_APP_MDS_ARM_FR_DECODER 1.1.5.
NEW!AuditFiltered 1
NEW!CSVFieldsContainDelimiters 0
NEW!DiscardEmptyFiles 0
NEW!DiscardOnRejection 0
NEW!EscapeWithBackslash 0
NEW!HeaderIdentifier
NEW!InputDescription cas.desc
NEW!InvalidLink INVALID
NEW!OutputLink DECODED
NEW!PercentOfRejectedRecordsAllowed
NEW!RecordIdentifier
NEW!RejectInvalid 0
NEW!SendInvalid 1
NEW!TrailerIdentifier
OLD!DiscardFailureThreshold 15
OLD!NetworkElementID CAS
OLD!SendDiscardedFilesToLink
OLD!UseRecordSequenceNumbers
NEW?DECODED 0
NEW?INVALID 0
OLD?DECODED_CAS 0
CH_PPV_DWH_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!CompressionMethod compress
NEW!OutputFileNameExtractionMask CDR\.(.+)
NEW!Password
NEW!PrivateKey ${EventLinkUserHome}/.ssh/id_rsa
NEW!PublicKey ${EventLinkUserHome}/.ssh/id_rsa.pub
NEW!Subdirectory UPC${Affiliate}_$1
NEW!TargetDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/DWH
NEW!Username ${EventLinkUser}
OLD!CompressionMethod nocompression
OLD!Password u1elkpe
OLD!PrivateKey
OLD!PublicKey
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/DWH
OLD!Username u1elkpe
CH_PPV_DWH_ENCODER
NEW#UPC_CFG_CAS_DWH_ENCODER 1.1.0.
OLD#UPC_CFG_CH_PPV_BRIO_ENCODER 1.1.0.
NEW!CreateEmptyOutputFileOnFlush 0
NEW!CreateEmptyOutputFileOnSeparation 0
NEW!CreateEmptyOutputFiles 0
NEW!MaxAgeOfFile
NEW!OutputDescriptionFileName output_description.xml
OLD!CreateEmptyOutputFileOnFlush
OLD!CreateEmptyOutputFileOnSeparation
OLD!CreateEmptyOutputFiles
OLD!MaxAgeOfFile 1M
OLD!OutputDescriptionFileName brio_description.xml
NEW@UPC_CFG_CAS_DWH_ENCODER REJ
OLD@UPC_CFG_CH_PPV_BRIO_ENCODER REJ
CH_PPV_FRAUD_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!OutputFileNameExtractionMask
NEW!Password
NEW!PrivateKey ${EventLinkUserHome}/.ssh/id_rsa
NEW!PublicKey ${EventLinkUserHome}/.ssh/id_rsa.pub
NEW!Subdirectory
NEW!TargetDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/FRAUD
NEW!TempDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/FRAUD/tmp
NEW!Username ${EventLinkUser}
OLD!Password u1elkpe
OLD!PrivateKey
OLD!PublicKey
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/FRAUD
OLD!TempDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/FRAUD/tmp
OLD!Username u1elkpe
CH_PPV_HORIZON_BLN
NEW#UPC_CFG_HORIZON_BLN 1.5.2.
OLD#UPC_CFG_HORIZON_BLN 1.5.1.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 3.1.0.
NEW!KenanCounter K_0_1_2
NEW!OracleDatabaseName ${EventLinkDatabase}
OLD!AuditOutType ${AuditOutType}
OLD!KenanCounter K_0_2_1
OLD!OracleDatabaseName U1ELKPE
OLD!StreamName ${StreamName}
CH_PPV_KENAN_DISTRIBUTOR
NEW#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc2
OLD#EL_CFG_FTP_DISTRIBUTOR 2.1.0.upc1
NEW!OutputFileNameExtractionMask
NEW!Subdirectory
NEW!TargetDirectory ${EventLinkUserHome}/delivery/${StreamName}/OUTPUT/KENAN
NEW!Username ${EventLinkUser}
OLD!TargetDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/OUTPUT/KENAN
OLD!Username u1elkpe
CH_PPV_KENAN_DWH_BLN
NEW#UPC_CFG_CH_PPV_KENAN_BRIO_BLN 2.0.0.
OLD#UPC_CFG_CH_PPV_KENAN_BRIO_BLN 1.0.5.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 1.11.2.
NEW!ReprocessLimit 100d
OLD!202_DEFAULT_TIME 000000
OLD!AuditOutType ${AuditOutType}
OLD!LsAdmin ${EventLinkUserHome}/EventLink/lookup_server3/bin/ls_admin
OLD!ReprocessLimit 100
OLD!StreamName ${StreamName}
NEW?RDC_OUT1 0
OLD?RDC_OUT 0
OLD@UPC_CFG_CH_PPV_KENAN_BRIO_BLN INVALID_TVOD
OLD@UPC_CFG_CH_PPV_KENAN_BRIO_BLN REJECTED
CH_PPV_KENAN_ENCODER
NEW#UPC_CFG_CAS_KENAN_ENCODER 1.1.0.
OLD#UPC_CFG_CH_PPV_KENAN_ENCODER 1.0.0.
NEW#EL_LIB_CML 1.5.5.
OLD#EL_LIB_CML 1.5.4.
NEW!CreateEmptyOutputFileOnFlush 0
NEW!CreateEmptyOutputFileOnSeparation 0
NEW!CreateEmptyOutputFiles 0
NEW!DataRecordType PV1
NEW!MaxAgeOfFile
NEW!OutputDescriptionFileName dpv_old.xml
NEW!OutputFileNameCreationMask PV1.&SEQ(7).&CREATIONTIME(%Y%m%d%H%M%S).&SOURCEID.00.&MIN(EL_PURCHASE_DATETIME) &MAX(EL_PURCHASE_DATETIME)
NEW!RecordLimit
NEW!SeparateSequenceNumbersInOutput 0
OLD!CreateEmptyOutputFileOnFlush
OLD!CreateEmptyOutputFileOnSeparation
OLD!CreateEmptyOutputFiles
OLD!MaxAgeOfFile 86400
OLD!OutputDescriptionFileName kenan_description.xml
OLD!OutputFileNameCreationMask PV1.&SEQ(7).&CREATIONTIME(%Y%m%d%H%M%S).&SOURCEID.00.&MIN(EL_PURCHASE_LOCALTIME) &MAX(EL_PURCHASE_LOCALTIME2)
OLD!RecordLimit 100
OLD!SeparateSequenceNumbersInOutput 1
NEW@UPC_CFG_CAS_KENAN_ENCODER REJ
OLD@UPC_CFG_CH_PPV_KENAN_ENCODER REJ
CH_PPV_LOCAL_COLLECTOR
NEW#EL_APP_FTP_COLLECTOR 2.1.4.upc3
OLD#EL_APP_FTP_COLLECTOR 2.1.4.upc1
NEW#EL_LIB_NODE_EXTENSIONS 1.1.1.
NEW#UPC_LIB_PERL_UTILS 1.11.2.
NEW!BehaviourOnError Abort
NEW!OnlyCollectFilesOlderThan
NEW!SourceDirectory ${EventLinkUserHome}/delivery/${StreamName}/INPUT/
OLD!BehaviourOnError Restart
OLD!SourceDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/INPUT/
CH_PPV_OUTPUT_BACKUP
DONOTEXISTANYMORE
OLD#UPC_CFG_BACKUP 1.1.1.
OLD#EL_APP_FTP_DISTRIBUTOR 2.1.4.upc1
OLD#EL_LIB_SFTP 1.1.0.
OLD!BehaviourOnError Restart
OLD!BlockSize
OLD!Cipher
OLD!Compression
OLD!CompressionMethod gzip
OLD!ConnectionEstablishmentTimeout
OLD!ConnectionRetryCount
OLD!ConnectionRetryInterval
OLD!DailyDirectories no
OLD!DestinationHost
OLD!ExternalCommand
OLD!FlagFileSuffix
OLD!InputLinkDirectories Yes
OLD!Kex
OLD!MaxDays 30
OLD!MaxFiles 0
OLD!MaximumReconnectionInterval
OLD!MaximumSustainedNetworkDowntime
OLD!OriginalFilenameMask
OLD!Password
OLD!Port
OLD!PrivateKey
OLD!PublicKey
OLD!RemoteOS
OLD!RemotePrefix
OLD!RemoteSuffix
OLD!Ruleset backup.pm
OLD!SourceIdDirectories no
OLD!TargetDirectory /backup/BACKUP_EL603/UPCCH_PPV
OLD!TempDirectory
OLD!TempSuffix
OLD!TransferMode
OLD!TransferProtocol Local
OLD!TransferRetryCount
OLD!TransferRetryInterval
OLD!TransferType
OLD!Username
OLD?ENCODED_AUDIT 1
OLD?ENCODED_DWH 1
OLD?ENCODED_KENAN 1
CH_PPV_PRODIS_COLLECTOR
NEW!DeleteSource No
NEW!RemotePrefix old_
NEW!SourceDirectory ${EventLinkUserHome}/delivery/${StreamName}/INPUT/PRODIS
NEW!Username ${EventLinkUser}
OLD!DeleteSource Yes
OLD!RemotePrefix
OLD!SourceDirectory ${EventLinkUserHome}/delivery/UPCCH_PPV/INPUT/PRODIS
OLD!Username u1elkpe
CH_PPV_PRODIS_COMMON_BLN
NEW#UPC_CFG_PRODIS_COMMON_BLN 1.2.1.
OLD#UPC_CFG_PRODIS_COMMON_BLN 1.2.0.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 2.0.0.
OLD!AuditOutType ${AuditOutType}
OLD!StreamName ${StreamName}
CH_PPV_PRODIS_DECODER
NEW#EL_APP_ASCII_DECODER 3.0.3.upc5
OLD#EL_APP_ASCII_DECODER 3.0.3.upc4
NEW!PercentOfRejectedRecordsAllowed
CH_PPV_PRODIS_FDC
NEW!SequenceChecking 0
NEW!TRS.DataStorage ${StreamName}
OLD!SequenceChecking 1
OLD!TRS.DataStorage UPCCH_PPV
CH_PPV_PRODIS_RDC
NEW#UPC_CFG_RECORD_CHECKER 8.2.0.
OLD#UPC_CFG_RECORD_CHECKER 8.1.0.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
OLD#UPC_LIB_PERL_UTILS 2.0.0.
NEW!DuplicateCheckEnabled No
NEW!OutputCounter B_0_1_2
NEW!RestartInterval 1d
NEW!TRS.DataStorage ${StreamName}
OLD!DuplicateCheckEnabled Yes
OLD!NodeName CH_PPV_PRODIS_RDC
OLD!OutputCounter B_0_2_1
OLD!StreamName ${StreamName}
OLD!TRS.DataStorage UPCCH_PPV
NEW@UPC_CFG_RECORD_CHECKER INVALID_DATA
CH_PPV_RAW_BACKUP
DONOTEXISTANYMORE
OLD#UPC_CFG_BACKUP 1.1.1.
OLD#EL_APP_FTP_DISTRIBUTOR 2.1.4.upc1
OLD#EL_LIB_SFTP 1.1.0.
OLD!BehaviourOnError Restart
OLD!BlockSize
OLD!Cipher
OLD!Compression
OLD!CompressionMethod gzip
OLD!ConnectionEstablishmentTimeout
OLD!ConnectionRetryCount
OLD!ConnectionRetryInterval
OLD!DailyDirectories no
OLD!DestinationHost
OLD!ExternalCommand
OLD!FlagFileSuffix
OLD!InputLinkDirectories No
OLD!Kex
OLD!MaxDays 30
OLD!MaxFiles 0
OLD!MaximumReconnectionInterval
OLD!MaximumSustainedNetworkDowntime
OLD!OriginalFilenameMask
OLD!Password
OLD!Port
OLD!PrivateKey
OLD!PublicKey
OLD!RemoteOS
OLD!RemotePrefix
OLD!RemoteSuffix .&CREATIONTIME(%Y%m%d%H%M%S)
OLD!Ruleset backup.pm
OLD!SourceIdDirectories yes
OLD!TargetDirectory /backup/BACKUP_EL603/UPCCH_PPV/RAW
OLD!TempDirectory
OLD!TempSuffix
OLD!TransferMode
OLD!TransferProtocol Local
OLD!TransferRetryCount
OLD!TransferRetryInterval
OLD!TransferType
OLD!Username
OLD?CAS_COLLECTOR 1
OLD?COLLECTED 1
CH_PPV_RDC
NEW#UPC_CFG_RECORD_CHECKER 8.2.0.
OLD#UPC_CFG_RECORD_CHECKER 7.1.5.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
NEW!AuditFields AUD_UA=I_UA,AUD_STU_NUMBER=I_STU_NUMBER,AUD_IMS_PRODUCT_ID=I_IMS_PRODUCT_ID
NEW!DuplicateCheckEnabled No
NEW!DuplicateCounter P_0_1_1
NEW!DuplicateDurationCounter
NEW!DurationField
NEW!InputCounter
NEW!InputDurationCounter
NEW!Keys I_UA,I_IMS_PRODUCT_ID
NEW!OutputCounter
NEW!OutputDurationCounter
NEW!RestartInterval 1d
NEW!SeparationCondition
NEW!SeparationValue1
NEW!SeparationValue2
NEW!SeparationValue3
NEW!SeparationValue4
NEW!SeparationValue5
NEW!TRS.DataStorage ${StreamName}
OLD!DuplicateCheckEnabled Yes
OLD!Keys I_UA,I_PURCHASE_DATE,I_PURCHASE_TIME,I_IMS_PRODUCT_ID
OLD!StreamName ${StreamName}
OLD!TRS.DataStorage UPCCH_PPV
NEW?RDC_OUT1 0
NEW?RDC_OUT2 0
NEW?RDC_OUT3 0
NEW?RDC_OUT4 0
NEW?RDC_OUT5 0
NEW?VALID 0
OLD?RDC_OUT 0
OLD?VALIDATION_OUT 0
NEW@UPC_CFG_RECORD_CHECKER DUPLICATE_RECORDS
NEW@UPC_CFG_RECORD_CHECKER INVALID_DATA
OLD@UPC_CFG_RECORD_CHECKER PV_INVALID_DATA
CH_PPV_VALIDATOR
NEW#UPC_CFG_CAS_VALIDATOR 1.1.1.
OLD#UPC_CFG_VOD_VALIDATOR 1.1.1.
NEW#UPC_LIB_PERL_UTILS 3.1.2.
NEW!InputCounter I_0_2_1
NEW!RejectedCounter P_0_1_1
NEW!RestartInterval 1d
NEW!Timezone ${Timezone}
OLD!AuditOutType ${AuditOutType}
OLD!StreamName ${StreamName}
NEW?AUDIT_REJECTED 0
NEW?DECODED 0
NEW?INVALID 0
NEW?VALID 0
OLD?DECODED_CAS 0
OLD?VALIDATION_AUDIT_OUT 0
OLD?VALIDATION_OUT 0
NEW@UPC_CFG_CAS_VALIDATOR INVALID_DATA
OLD@UPC_CFG_VOD_VALIDATOR INVALID_DATETIME
The script I want to show output as comparisons between the two same types of records;
NOTE: NEW#,OLD#,NEW!,OLD!,NEW?,OLD?,NEW@,OLD@ are my own defined ways to differentiate between the different types of outputs.
<Name>
<TechnicalName><TAB><OldValue><TAB><NewValue>
Also there are cases in which only NEW is present and OLD is not there leave compared value of OLD as NULL and vicaversa.
I hope i made the complete problem understood. I real;ly need your help as i have been stuck with this for long.
Thanks Again for your patience.