Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Need help with a script to process a CSV file using SED and AWK
Top Forums Shell Programming and Scripting Need help with a script to process a CSV file using SED and AWK Post 302374415 by kkb on Tuesday 24th of November 2009 12:02:36 PM
Old 11-24-2009
Need help with a script to process a CSV file using SED and AWK
I get a CSV file every day with 2 columns and multiple rows
ex:
date1,date2 ( both the fields are varchar fields)
This data has to be updated in a table which is being done manually and i want to automate that.
1. I have to select all the data from the prod table( 2 columns { date1,date2}) into a staging table.
2. after that i have to insert the values from the CSV file into the table
3. run some validation procedures and then copy this data into prod table.
environment : sybase
OS: AIX(ksh)
I am having trouble getting data from CSV into to sybase table.
I tried to format the CSV file using SED and AWK with each column seperated by tab space and line seperated by newline.
"sed '1d' dates.txt | awk -F "," '{printf "%s\t%s\n",$1,$2}' > trial.txt"
but that gives me an when I bcp data into table. something to do with format.
any one has any idea what the right format to insert into 2 varchar columns?
or is there any other better way of doing this?

---------- Post updated at 12:02 PM ---------- Previous update was at 11:16 AM ----------

Both the fields are datetime fields.

I was thinking about storing the fields into variables and inserting these into the table using a loop.

any ideas about that ?
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sed or awk script to remove text / or perform calculations from large CSV files

I have a large CSV files (e.g. 2 million records) and am hoping to do one of two things. I have been trying to use awk and sed but am a newbie and can't figure out how to get it to work. Any help you could offer would be greatly appreciated - I'm stuck trying to remove the colon and wildcards in... (6 Replies)
Discussion started by: metronomadic
6 Replies

2. Shell Programming and Scripting

Updating a line in a large csv file, with sed/awk?

I have an extremely large csv file that I need to search the second field, and upon matches update the last field... I can pull the line with awk.. but apparently you cant use awk to directly update the file? So im curious if I can use sed to do this... The good news is the field I want to... (5 Replies)
Discussion started by: trey85stang
5 Replies

3. Shell Programming and Scripting

Using awk/sed in handling csv file.

Please study the below script and the output Script: echo "Minimum ${host} ${process} response time=${min} ms" >> ${OUTDIR}/${OUTFILE}; echo "Maximum ${host} ${process} response time=${max} ms" >> ${OUTDIR}/${OUTFILE}; echo "Average ${host} ${process} response time=${avg} ms" >>... (0 Replies)
Discussion started by: ajincoep
0 Replies

4. Shell Programming and Scripting

awk/sed/something else for csv file

Hi, I have a filename.csv in which there are 3 colums, ie: Name ; prefixnumber ; number root ; 020 ; 1234567 user1,2,3 ; 070 ; 7654321 What I want is to merge colum 2 and 3 that it becomes 0201234567 or even better +31201234567 so the country number is used and drop the leading 0.... (9 Replies)
Discussion started by: necron
9 Replies

5. Shell Programming and Scripting

HELP with AWK or SED. Need to replace the commas between double quotes in CSV file

Hello experts, I need to validate a csv file which contains data like this: Sample.csv "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 "ABCD","I",23,0,9,,"23/12/2012","OK","Street,State, 91135",0 I just need to check if all the records contain exactly the number of... (5 Replies)
Discussion started by: shell_boy23
5 Replies

6. Shell Programming and Scripting

Converting txt file into CSV using awk or sed

Hello folks I have a txt file of information about journal articles from different fields. I need to convert this information into a format that is easier for computers to manipulate for some research that I'm doing on how articles are cited. The file has some header information and then details... (8 Replies)
Discussion started by: ksk
8 Replies

7. Shell Programming and Scripting

Replacing FQDN by hostnames in a CSV file with sed & awk

Hello, Beginning with shell scipting, I'm trying to find in a csv file, the lines where the field related to hostname is displayed as an FQDN intead the hostname. (some lines are correct) and the to correct that inside the file: Novell,11.0,UNIX Server,bscpsiws02,TxffnX1tX1HiDoyBerrzWA==... (2 Replies)
Discussion started by: Wonto
2 Replies

8. Shell Programming and Scripting

Combined sed+awk for lookup csv file

have written a combined sed+awk to perform a lookup operation which works but looking to enhance it. looking to match a record using any of the comma separated values + return selected fields from the record - including the field header. so: cat foo make,model,engine,trim,value... (6 Replies)
Discussion started by: jack.bauer
6 Replies

9. Shell Programming and Scripting

Datestamp format 2nd change in csv file (awk or sed)

I have a csv file formatted like this: 2014-08-21 18:06:26,A,B,12345,123,C,1232,26/08/14 18:07and I'm trying to change it to MM/DD/YYYY HH:MM for both occurances. I have got this: awk -F, 'NR <=1 {print;next}{"date +%d/%m/%Y\" \"%H:%m -d\""$1 "\""| getline dte;$1=dte}1' OFS="," test.csvThis... (6 Replies)
Discussion started by: say170
6 Replies

10. UNIX for Beginners Questions & Answers

awk and sed script to create one output CSV file

Hi All , I would require your help to generate one output file after post processing of one CSV file as stated below This file is just a small cut from a big file . Big file is having 20000 lines PATTERN,pat0,pat1,pat2,pat3,pat4,pat5,pat6,pat7,pat8,pat9... (2 Replies)
Discussion started by: kshitij
2 Replies

LEARN ABOUT ULTRIX

dtif

DTIF(5) 							File Formats Manual							   DTIF(5)

Name
       DTIF - Digital Table Interchange Format

Description
       Digital	Table Interchange Format (DTIF) is the standard format for the storage and interchange of documents that contain data tables, for-
       mulas, and spreadsheets. You can use DTIF to store and retrieve database information, interchange spreadsheets, and reference table data in
       compound documents.

       DTIF defines the logical structure and physical layout of a data table, the values within the table (absolute data and/or expressions), and
       presentation attributes (formatting) to be used when displaying or printing the table. DTIF works with Digital Document Interchange  Format
       (DDIF) so that you can store or reference DTIF tables in DDIF-encoded compound documents.

       A  DTIF	document  can  contain	a sequence of one or more tables and is uniquely identified by a product name, a version number, and other
       descriptive information such as the document's title and creation date.	Each DTIF table is a 2-dimensional display of  data  values  orga-
       nized in columns and rows that has its own structure and table data stored in cells.

       In  DTIF  documents,  attributes  specify  the  type  and format of information pertaining to the data stored in a table. Column attributes
       describe information for all the cells in a particular column, whereas generic column attributes can be applied to any column in any  table
       that  references  them. Format attributes define the printed and displayed presentation of data stored in the table.  Format attributes can
       also be redefined at the window, column, or cell level.

See Also
       CDA(5), DDIF(5), DTIF(5)
       Compound Document Architecture Manual

																	   DTIF(5)
All times are GMT -4. The time now is 03:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy