Sponsored Content
Full Discussion: Naive coding...
The Lounge What is on Your Mind? Naive coding... Post 302993028 by bakunin on Sunday 5th of March 2017 08:16:02 PM
Old 03-05-2017
A few observations about programming habits and programming in general:

I think the real difference between naive and non-naive (clever) programming is not so much to make (or make not) use of a certain language construct but the employment of certain algorithms. The example of Corona688 shows that very clearly. The difference in simplicity (and perhaps speed) comes not from using some "clever language feature" a non-expert might not know about, but from the ability to look at the problem from a different angle and draw the right conclusion.

Here is a story from my own practice: I once worked for a development team who created some database application. At one time they had trouble with a daily import routine because it ran approximately 32 hours. (For the non-experts: standard days come with only 24 of them. ;-) ) The import script worked like that: first a huge SQL-statement to create some table. This table was fed into a loop where something was changed, then the whole was fed back into the DB. The script looked like this (only the structure):

Code:
#! /bin/ksh

db2sql "...some 300 lines of SQL here...." |\
while read record ; do
     newrecord=$(echo $record | sed '....some changing of the record here....')
     db2sql "... import record from $newrecord here..."
done

exit 0

There is nothing "logically wrong" here and when you test it with 5 records it is probably as fast as any other solution. Doing it with some ten-million-records table, though, will reveal that calling sed one time for every record is slightly slower (by several orders of magnitude) than to call it once in a pipeline:

Code:
db2sql "...some 300 lines of SQL here...." |\
sed '....some changing of the records here....' |\
while read newrecord ; do
     db2sql "... import record from $newrecord here..."
done

So, this was my first instinctive comment and (we tested it later) it would have brought down the processing time by ~3.5 hours. But i worked on this together with the DBA and he observed inside the 300-line-SQL-monster (which i had ignored, being an SQL-ignorant) that most of it was effectively creating a left outer join of the whole database on itself and then dropping 99% of it, So, after selecting what really was to be selected - and finding out that the changes could be done inside instead of using the shell and an external program to do it - - we arrived at:

Code:
db2sql "select for update ...some 20 lines of SQL here...."

which took - about 5 seconds. All the while management was calling for bigger hardware to "meet increasing needs" instead of just firing this incompetent click-boy who confused some graphic representation of the DB within a tool (Informatica) with reality.

So, what is the differrence between clever and naive programming? It is about the same difference as 32 hours and 5 seconds.

My two cents.

bakunin
These 5 Users Gave Thanks to bakunin For This Post:
 

7 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

can I use this coding

I apologise because I had pasted this question in the newbies forum first (because i am a bit of a newbie) but thought it might be better suited in here if i have to sepearate parameters can I use this syntax especially the or part (||) and is this correct if (6 Replies)
Discussion started by: w33man
6 Replies

2. Shell Programming and Scripting

Coding on my Mac

I would like to start coding on my mac, but I'm getting an error when I attempt to execute my script -bash : testscript: command not found I have verified that the #! line points to the correct directory. If you have some insight it would be greatly appreciated! - D (1 Reply)
Discussion started by: DKNUCKLES
1 Replies

3. UNIX for Dummies Questions & Answers

pro*c coding

Hi All, I am new to pro*C. I have a select statement as select a.ename,a.sal,a.empno from emp where &n=(select count(distinct(b.sal)) from emp b where a.sal<=b.sal for this query I have to write a pro*C program. So can you please send me the complete code. Then I will foloow the same... (1 Reply)
Discussion started by: user71408
1 Replies

4. Homework & Coursework Questions

Naive Bayes

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I have to write a program in Unix to do the following.Given a phrase like george hates john, by... (0 Replies)
Discussion started by: gizmo87
0 Replies

5. Shell Programming and Scripting

Need help with coding

HI, Can some one guide me how to make changes to the script below so that it can load the history of a program to IT server ? Format of data: YYYYMMDD065959.dsk.log YYYYMMDD235959.dsk.log currently both are loaded together. Need to separate them as above format. Thanks in advance. ... (1 Reply)
Discussion started by: crazydude80
1 Replies

6. Windows & DOS: Issues & Discussions

Need help with coding

HI, Can some one guide me how to make changes to the script below so that it can load the history of a program to IT server ? Format of data: YYYYMMDD065959.dsk.log YYYYMMDD235959.dsk.log currently both are loaded together. Need to separate them as above format. Thanks in advance. ... (2 Replies)
Discussion started by: crazydude80
2 Replies

7. Shell Programming and Scripting

HTTP coding

My company has an in house instant messaging system (like WhatsApp) where users can communicate with each other. I currently have code to email me certain items from my Sparc machine running SunOS 5.10. I want what I am emailing myself to now instant message me. The team that created the messenger... (5 Replies)
Discussion started by: shorty
5 Replies
All times are GMT -4. The time now is 12:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy