Shell Programming and Scripting

@MadeINGermany

Thanks. I changed the locale settings and it's working.

But the only problem I saw is when there is no values in some of the columns then the file looks like ;;;; with just separators.

The code is removing one of the last separator ";". When it contains no values then it will show just multiple separators like ;;;; .

It is removing the last ";" at the end of few lines (rows). I have provided the example below. Left section is input and the right one is output where you can see one of the ";" is removed by the program.





And if the column is added in the last it is not getting reported in the mismatch.txt. It is reporting only when the columns are added in the middle. Can you please let me know how to resolve this issue of one missing ";" at the end of few rows? and how to report if the column or columns are added at the end or middle or beginning?


Merlin Joseph

The printed columns are determined by the matches between the line in declare.txt and the 1st line in feed.txt
So I guess you must add one (or three) more matching field to declare.txt

@MadeINGermany

But the number of columns I declared are found in the input file feed.txt and they are matching.

It's about a scenario when no new columns are added in the feed.txt :

The program has to just return the feed.txt file with a different name feed2.txt with all it's data and columns intact, when the program does not find any new columns added in the feed.txt

Please advice.

Merlin Jospeh

@Rudic,

I have even tested your script. It works well for the first time and then when it is executed for the second time it is removing the values from most of the columns. It is even creating a "mismatch.txt" even when there are no columns added in the "feed.txt"

I found these values in the mismatch.txt when I did not add any new columns and values (for testing) in the feed.txt (input file).

Column 2: image
Column 3: id
Column 4: showcase_id
Column 5: showcase_name
Column 6: showcase_zip_code
Column 7: vehicle_id
Column 8: carmodel_id
Column 9: make_id
Column 10: external_mpn
Column 11: carmodel_name
Column 12: make_name
Column 13: colour
Column 14: colour_alias
Column 15: colour_secondary_name
Column 16: colour_secondary_alias
Column 17: trim
Column 18: catalog_price
Column 19: customer_bonus
Column 20: reprise_bonus
Column 21: legal_notice
Column 22: rebate_legal_notice
Column 23: eco_bonus
Column 24: price
Column 25: customer_benefit_percent
Column 26: options
Column 27: code
Column 28: name
Column 29: matriculation_on
Column 30: status
Column 31: updated_at
Column 32: created_at
Column 33: mileage
Column 34: vehicle_acceleration_0_100kph
Column 35: vehicle_air_conditioning
Column 36: vehicle_body_type
Column 37: vehicle_co2_emission_level
Column 38: vehicle_combined_fuel_economy
Column 39: vehicle_cylinders_count
Column 40: vehicle_doors_count
Column 41: vehicle_driven_wheels
Column 42: vehicle_emission_standard
Column 43: vehicle_engine_capacity
Column 44: vehicle_extra_urban_fuel_economy
Column 45: vehicle_fiscal_horse_power
Column 46: vehicle_height
Column 47: vehicle_kerbweight
Column 48: vehicle_length
Column 49: vehicle_max_load_capacity
Column 50: vehicle_name
Column 51: vehicle_primary_fuel_tank_capacity
Column 52: vehicle_primary_fuel_type
Column 53: vehicle_published_hp_metric
Column 54: vehicle_seats_count
Column 55: vehicle_secondary_fuel_type
Column 56: vehicle_type
Column 57: vehicle_luggage_capacity
Column 58: vehicle_tvs_tax
Column 59: vehicle_transmission_shiftable
Column 60: vehicle_trim_name
Column 61: vehicle_trim_code
Column 62: vehicle_urban_fuel_economy
Column 63: vehicle_vehicle_warranty_kms
Column 64: vehicle_vehicle_warranty_months
Column 65: vehicle_width_excluding_mirrors
Column 66: vehicle_transmission_type
Column 67: vehicle_engine_name
Column 68: vehicle_engine_code
Column 69: vehicle_rsi_id
Column 70: image_1
Column 71: image_2
Column 72: image_3
Column 73: image_4
Column 74: image_5

It is skipping the first column name "url"
and also in the output it is removing the column values when executed more than once.

Ex: offre-vehicule-neuf/make/rav4-124-d-4d-2wd-life_157127/chartres_1766?_krg=link&_krk=xyz&_krt=HREV30D003458;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ;;;;;;;;;;;;;;;;;;;;;;


Only the delimiters remain.

Thanks

Merlin Joseph

Did you check to see if the output files produced by your script contain the header lines?

@Dan Cragun,

Yes, when I checked it is just keeping one column with it's value and then removing all other columns and it's values from the output file. The header is like this in the output file

url;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Surprisingly the script provided by Mr.RudiC works for the first time and then it fails to generate the correct output.

The requirement is, report the missing columns (one or more) if added any where in the feed.txt (input file) when compared with the declare.txt (where column names are declared for comparison).

Generate an output file (feed2.txt for example) in the both the scenarios whether new columns are found or not found. If new columns are found create a file "mismatch.txt" for reporting. If new columns are not added in the input file then the output will remain same as input file (feed.txt = feed2.txt). If new columns are found then they should be removed with it's associated values for creating an output file "feed2.txt"

Please help.

Thanks
Merlin Joseph

The declare.txt and the header of feed.txt must be identical (including the field contents),
then feed.txt won't be changed.
So I have understood the original requirement.

---------- Post updated at 12:43 PM ---------- Previous update was at 11:00 AM ----------

Also, in feed.txt the number of fields must be consistent.
In your previous example, the header has not got the "updated_at" and "created_at", so the script does not work correctly (it then strips off the last two columns):

Code:

awk -F\; '{print NF}' feed.txt
72
74
74
74