Fixing this data at source is not currently an option although I agree it would make more sense, I currently have to work with the data I am sent.
A typical example of the data would be
Note that even though each row contains 18 fields it is only the first ten fields which make up the unique key.
I am currently playing with the idea of loading the total sent rows into an oracle table and then loading each file recieved into a temp table allowing me to do some set arithmetic on it to get what I need (SELECT TABLE2 MINUS TABLE1), but if possible I would prefer to do this just using files on the server.
Hi Gurus,
I had a question regarding avoiding duplicates.i have a file abc.txt
abc.txt
-------
READER_1_1_1> HIER_28056 XML Reader: Error occurred while parsing:; line number ; column number
READER_1_3_1> Sun Mar 23 23:52:48 2008
READER_1_3_1> HIER_28056 XML Reader: Error occurred while... (7 Replies)
Hi,
I need to remove duplicates from a file. The file will be like this
0003 10101 20100120 abcdefghi
0003 10101 20100121 abcdefghi
0003 10101 20100122 abcdefghi
0003 10102 20100120 abcdefghi
0003 10103 20100120 abcdefghi
0003 10103 20100121 abcdefghi
Here if the first colum and... (6 Replies)
Hi Unix gurus,
Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me.
File format: CSV file
File has four columns with no header... (8 Replies)
Hi Experts,
Please check the following new requirement. I got data like the following in a file.
FILE_HEADER
01cbbfde7898410| 3477945| home| 1
01cbc275d2c122| 3478234| WORK| 1
01cbbe4362743da| 3496386| Rich Spare| 1
01cbc275d2c122| 3478234| WORK| 1
This is pipe separated file with... (3 Replies)
Hi All,
I have an xml file that contains information like this
<ID>574922<COMMENT>TEXT
TEXT
TEXT</COMMENT></ID>
<ID>574922<COMMENT>TEXT
TEXT
TEXT</COMMENT></ID>
<ID>412659<COMMENT>TEXT
TEXT
TEXT
TEXT
TEXT</COMMENT></ID>
<ID>873520<COMMENT>TEXT</COMMENT></ID>... (5 Replies)
Hi Folks -
I'm quite new to awk and didn't come across such issues before. The problem statement is that, I've a file with duplicate records in 3rd and 4th fields. The sample is as below:
aaaaaa|a12|45|56
abbbbaaa|a12|45|56
bbaabb|b1|51|45
bbbbbabbb|b2|51|45
aaabbbaaaa|a11|45|56
... (3 Replies)
Dear All
I have 200 data files and each files has many duplicates.
I am looking for the automated awk script such that it checks and removes the duplicates from the each file and saving them as new files for all 200 files in the respective folder.
For example my data looks like this..
... (12 Replies)
Hi All,
I am merging files coming from 2 different systems ,while doing that I am getting duplicates entries in the merged file
I,01,000131,764,2,4.00
I,01,000131,765,2,4.00
I,01,000131,772,2,4.00
I,01,000131,773,2,4.00
I,01,000168,762,2,2.00
I,01,000168,763,2,2.00... (5 Replies)
i hav two files like
i want to remove/delete all the duplicate lines in file2 which are viz unix,unix2,unix3.I have tried previous post also,but in that complete line must be similar.In this case i have to verify first column only regardless what is the content in succeeding columns. (3 Replies)
Discussion started by: sagar_1986
3 Replies
LEARN ABOUT NETBSD
weekly
WEEKLY(5) BSD File Formats Manual WEEKLY(5)NAME
weekly, weekly.conf -- weekly maintenance
DESCRIPTION
The /etc/weekly script is run, by default, every Saturday morning on a NetBSD system. The /etc/weekly.conf file specifies which of the stan-
dard weekly services are performed.
The variables described below can be set to ``YES'' or ``NO'' in the /etc/weekly.conf file. The default settings are in the
/etc/defaults/weekly.conf file. (Note that you should never edit /etc/defaults/weekly.conf directly, as it is often replaced during system
upgrades.)
rebuild_locatedb This rebuilds the locate(1) database, /var/db/locate.database, which must also exist, in order to be rebuilt.
rebuild_mandb This rebuilds the apropos(1) database /var/db/man.db, using makemandb(8) with the -f option.
rebuild_whatisdb This rebuilds the whatis(1) database(s). Note that NetBSD provides a default whatis.db for the system manual pages and
this may not be needed. (Adjust your /etc/man.conf as necessary; see man.conf(5) for details.)
FILES
/etc/weekly weekly maintenance script
/etc/weekly.conf weekly maintenance configuration
/etc/weekly.local local site additions to /etc/weekly
SEE ALSO daily.conf(5), monthly.conf(5)HISTORY
The /etc/weekly script first appeared in 4.3BSD. The /etc/weekly.conf configuration file appeared in NetBSD 1.3.
BSD March 6, 2012 BSD