08-18-2009
jp2542a: Thanks, I will give that a try this morning and see what results I get. I'm not familiar with awk, but plan to learn it so this may be an opportunity.
edidataguy: Thanks for pointing that out! The only constant is that all will begin with UR and all will end with XD and then an empty (after removal of ^M) line. This is obviously generated on a Win machine, buy I am working entirely on Linux desktop and a Linux server.
That said, it may help if I state that I will then use this to insert and delete records in a MySQL database and run as a cron job. Thanks for the help!
Last edited by LoveSquid; 08-18-2009 at 11:04 AM..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi Fellows,
I have been struggling to fix an issue in csv records to compose sql statements and have been really losing sleep over it. Here is the problem:
I have csv files in the following pipe-delimited format:
Column1|Column2|Column3|Column4|NEWLINE
Address Type|some descriptive... (4 Replies)
Discussion started by: khayal
4 Replies
2. Shell Programming and Scripting
sample file (comma as field separators)
MessageFlow,1,BusIntBatchMgr,a
OOBEvent,1,BusIntBatchMgr,a
TaskEvents,1,,a
MTTrace,1,,a
MTWarning,,1,a
MessageFlow,1,Batch,a
OOBEvent,1,Batch,a
TaskEvents,1,,a
EAISAPIdocWizard,1,BusIntMgr,a
EAISAPBAPIWizard,1,BusIntMgr,a... (3 Replies)
Discussion started by: vrclm
3 Replies
3. Shell Programming and Scripting
Hi Friends,
I have come across some files where some of the columns don not have data.
Key, Data1,Data2,Data3,Data4,Data5
A,5,6,,10,,
A,3,4,,3,,
B,1,,4,5,,
B,2,,3,4,,
If we see the above data on Data5 column do not have any row got filled. So remove only that column(Here Data5) and... (4 Replies)
Discussion started by: ks_reddy
4 Replies
4. UNIX for Dummies Questions & Answers
Hi all,
I have two (2) sets of files that are based on some snapshots of database that I want to merge and insert any missing sequential number.
Below are example representation of these files:
file1:
DATE TIME COL1 COL2 COL3 COL4 ID
01/10/2013 0800 100 ... (3 Replies)
Discussion started by: newbie_01
3 Replies
5. Shell Programming and Scripting
Hi,
I have a csv file with content like:
1,0,100
1,1,150
2,0,200
2,1,250
3,0,300
3,1,350
I want an output such that all numbers in 3rd col where 2nd col is "0" come in the same col in the output. The same goes for numbers where 2nd col is "1".
1 100 150
2 200 250
3 300 350
Tnx... (2 Replies)
Discussion started by: jamaje
2 Replies
6. Shell Programming and Scripting
Hi, please help with this, I need to insert missing values into a matrix for a regression analysis.
I have made up an example. The first three columns are variables with levels and the next 3 are values, the 4th column missing values should be replaced by 0s, and 5th and 6th column missing... (3 Replies)
Discussion started by: ritakadm
3 Replies
7. Shell Programming and Scripting
Hello experts!
I have 2 files.
file1 is a list file containing uniquely names. e.g.:
name1 number number
name2 number number
name5 number number
name10 number number
...
file2 is a data file arbitrary containing the names of file1 in paragraphs separated by "10" e.g.
name4 ... (3 Replies)
Discussion started by: phaethon
3 Replies
8. Shell Programming and Scripting
I have 2 for loop in my program , first one will list files based on timestamp and second one list the files based on type(RPT / SUB_RPT).Here is my code:
#!/bin/ksh
STG_DIR=/home/stg
for pattern in `find $STG_DIR -type f -name 'IBC*csv' | awk -F'' '{print $(NF-1)}' | sort -u`
do
echo... (2 Replies)
Discussion started by: ann15
2 Replies
9. Shell Programming and Scripting
Using the file below, which will always have the first indicated by the digit after the -
and last id in it, indicated by the digit after the -, I am trying to use awk
to print the missing line or lines in file following the pattern of the previous line.
For example, in the file below the next... (4 Replies)
Discussion started by: cmccabe
4 Replies
10. UNIX for Beginners Questions & Answers
can anyone help me!!!! How to I parse the CSV file
file name : abc.csv (csv file) The above file containing data like
abv,sfs,,hju,',',jkk wff,fst,,rgr,',',rgr ere,edf,erg,',',rgr,rgr I have a requirement like i have to extract different field and assign them into different... (4 Replies)
Discussion started by: J.Jena
4 Replies
LEARN ABOUT DEBIAN
dbf2mysql
DBF2MYSQL(1) General Commands Manual DBF2MYSQL(1)
NAME
dbf2mysql, mysql2dbf - convert between xBase and MySQL databases
SYNOPSIS
dbf2mysql [-v[v]] [-f] [-u|-l] [-n] [-o field[,field]] [-e conversion-file] [-s old-name=new-name[,old-name=new-name]] [-i field[,field]]
[-d database] [-t table] [-c[c]] [-p primary] [-h host] [-F] [-q] [-r] [-x] [-P password] [-U user] dbf-file
mysql2dbf [-v[v]] [-u|-l] -d database -t table [-h host] [-q query] [-P password] [-U user] dbf-file
DESCRIPTION
dbf2mysql takes an xBase file and sends queries to an MySQL server to insert it into an MySQL table.
mysql2dbf dumps a MySQL table to a dbf file.
OPTIONS
-v Produce some status output.
-vv More verbose.
-vvv Even more verbose: produce a progress report.
-f Translate all field names in the xBase file to lowercase.
-u Translate all text in the xBase file to uppercase.
-l Translate all text in the xBase file to lowercase.
-n Allow NULL fields: 'NOT NULL' will be not added in table creation statement.
-o field[,field[,...]]
List fields to insert into MySQL database. Primary use is to ease import of complex dbf files where we want only few fields. NOTE:
-o is processed before substitution (-s), so you have to use dbf field names here.
-e conversion-file
Specify file for CHAR fields conversion. File format is: 1st line: number of characters to convert (number of lines). Further
lines: <char_to_convert> <char_after_conversion>.
-s old-name=new-name[,old-name=new-name[,...]]
Takes a list of field name/new field name pairs. Primary use is to avoid conflicts between field names and MySQL reserved keywords.
When the new field name is empty, the field is skipped in both the CREATE clause and the INSERT clauses, i.e. it will not be present
in the MySQL table. For example:
-s ORDER=HORDER,REMARKS=,STAT1=STATUS1
-i field[,field[,...]]
List fields to be indexed. MySQL field names should be used here.
-d database
Select the database to insert into. Default is 'test'.
-t table
Select the table to insert into. Default is 'test'.
-c[c] Create table if one doesn't exist already. If the table already exists, drop it and build a new one. The default is to insert all
data into the named table. If -cc is specified, no records will be inserted.
-p primary
Select the primary key. You have to give the exact field name.
-h host
Select the host to insert into. Untested.
-F Fixed length records. (By default CHAR is saved as VARCHAR.)
-q dbf2mysql: "Quick" mode. Inserts data via temporary file using 'LOAD DATA INFILE' MySQL statement. This increased insertion speed
on my PC 2-2.5 times. Also note that during whole 'LOAD DATA' affected table is locked.
mysql2dbf: Specify custom query to use.
-r Trim trailing and leading whitespace from CHAR type fields data.
-x Start each table with _rec and _timestamp fields.
-P password
Specify password on the MySQL server.
-U user
Specify user on the MySQL server.
BUGS
mysql2dbf can't write MEMO files at this time.
Possibly incorrect field lengths for REAL numbers.
AUTHORS
Originally written by Maarten Boekhold <boekhold@cindy.et.tudelft.nl>, Oct 1995.
Patched for MySQL by Michael Widenius <monty@analytikerna.se>, 3 Nov 1996.
Manual page written for Debian GNU/Linux from README by Heiko Schlittermann <heiko@lotte.sax.de>, Aug 1997; updated by Matej Vela
<vela@debian.org>, Nov 2001.
COPYRIGHT
Use this piece of software as you want, modify it to suit your needs, but please leave my name in place ok? :)
DISCLAIMER
The authors do not accept any responsibility for possible damage you get as result of using this program.
Debian Project 2001-11-18 DBF2MYSQL(1)