How to explain at your boss that code/resources optimization is important?

Ahhhw, every time is so frustrating..

We have a dedicated server in our hosting company, and everytime i have to write down a new app (or an add to an pre-existing app), i use to 'lose' some time to optimize the code for many behaviors (reducing the db query, optimizing the db structure, reducing the bandwith, etc..) depending on what the app is supposed to do.

Obviously, is the point is not that i write bad code and then rebuild it, its just that after the project is complete, i allways find somethings that could be done better.

And everytime, if my boss catch me doing this, he say 'Your wasting your time! if the application need more resources, we buy more RAM, more CPU, or more bandwith!'.

What is the best (and simplyest) way to explain he that opt开发者_运维知识库imization is still important, and that is not so easy or automagically upgrating the hardware of an (production!) server?

EDIT: im not talking just about database optimization, but every aspect of an application

Chances are you are really wasting time. Most of the optimizations that you learn in college or university are sort of irrelevant in the business world. I recommend concentrating on those optimizations where you can quantify WHY your optimization is worth your company's money (your time spent is their money spent).

When trying to optimise code you need to follow the optimisation cycle. There are no short-cuts.

1. Define specific, detailed, measurable, performance requirements.
2. Measure your application's performance.
3. If it meets the requirements, stop, as continuing is a waste of time/money.
4. Determine where the bottleneck is (which machine? CPU, memory, disk, etc.?).
5. Fix the bottleneck (either hardware or software fix, whichever is faster/cheaper).
6. Go to 2.

From your post it doesn't sound like you have done 1 or 2, so you've skipped over 3 and 4 because you don't know enough to do them, and are now stumbling blindly around in 5 attempting to optimise whatever you think needs optimising.

So from what you've said I'd have to agree with your boss. You are wasting your time. Or at least you haven't followed enough process to prove otherwise, which is essentially the same thing.

Keep detailed notes and time sheets the next time you make a server move/upgrade, and protocol every minute of work that arose from it (including things in the weeks after). Then you'll have hard evidence that upgrading is expensive, and optimizations help delay the investment.

In the current situation, you might be able to do something with benchmarks and statistics along the line of "without my optimization, the server will be at 90% capacity with X number of users. With my optimization, we can cater for Y users more, before we have to get a new machine."

On the other hand, the line between optimization and over-optimization is thin. While writing code that doesn't waste resources is good craftsmanship and should be a human right, RAM and disk space are awfully cheap these days. Optimizations also tend to make code more complicated, and harder to maintain for others. When you control your own hardware, code optimization may not always be the top goal.

Do you have performance objectives for the application(s)? Free capacity, response time, bandwidth usage etc. Without them you will find it difficult to justify why an improvement needs to be made. You need to be able to quantify the problem before people will spend money on it.

If the application is already meeting the requirements then your boss is probably right.

To ensure reserve capacity for future customers you (and your boss) could set a threashold such that once performance comes within a percentage of the requirements you need to investigate the problem. This is the time for you to request/push/negociate for performing optimisations as he will be aware of the problem and the risk of doing nothing. If you can recommend some changes to correct the problem that will cost less than the hardware then hopefully he should agree to let you do it. Of course the reverse still applies. If the total cost of the hardware upgrade is cheaper then that is probably the better option.

Wait until buying RAM and CPU becomes more expensive than paying you to optimize the code. Then you have a case.

There's a break even point between improving software vs. improving hardware. For desktop systems, a good threshold is by purchase time:

If that machine fails, can you go to the next mall and buy a replacement within an hour?

As long as you can answer yes, optimization is futile - an investment into the future maybe, but is that really your decision alone?

A similar rule for web sites might be "how long would it take to get a new hosting contract on that?"

This isn't perfect, it covers just a single aspect of scalability. I like it because it's an easy to understand aspect outside of technical concerns.

Of course, you should have more material if that aspect suddenly gets brushed away. It helps to just ask what are the companies/bosses plans for a few key scenarios. - e.g. what contingency plans for DDOS attacks, for "rush events" etc. Having some data to back that up, some key stories ("company foo was mentioned on popular night show, web server broke down under requests, business lost, potential customers angered") helps a lot.

The key to realize is that (assuming from your description alone) it is your responsibility to forsee such events, and provide the technical solutions, but it's not your responsibility to mke these decisions.

Also, getting a written statement from your boss that "five hours downtime isn't a problem, get bck to work on the important things!" is a good CYA.

about the same way one would explain a developer that code/resources optimization is unimportant ...

i guess this problem arises because your way to measure importance is different than his ...

his job is to care about money and make sure he gets the "most" out of it ... and frankly, servers are cheaper than developers ... and managed servers are cheaper than admins ... and the ugly truth is, that many people think that way ... personally, I think PHP isn't really the right choice for performance-critical parts of an app ... neither are RMDBSs ... but nearly every managed server runs PHP and MySQL and business clients prefer to have an app they can run everywhere, based on a technology widely known, which turns out to be PHP and MySQL on the server side ... the world is a cruel place and the truth is ugly ... now when it comes to you, you have only 2 choices:

• live with it
• work with people who are more like you, and who share the same ambitions

your boss does the job right, from his perspective ... what you do is a waste of company money, because the company's goal is not to write beautiful and highly optimized code ... whatever your company does, in the end you only provide a tool ... and the value of a tool is measured by the use it provides in relation to the money it costs ...

if your optimizations were economically reasonable, then the easiest way would be not to optimize anymore and wait for the moment until your boss comes to you, asking for speedup ...

i suggest, you either look for a new job, or focus on other important things:

• good design and use of best practices such as KISS, DRY, SOLID and GRASP
• scalable architecture (doesn't mean you have to actually implement it, just have an idea how the whole thing could scale well)
• maintainability, including readability and good documentation
• flexibility & extensibility
• robustness & correctness
• do not reinvent the wheel and use the right tools for a job - use adequate technologies, libraries and frameworks
• fast developement

these goals are important for the present and the future of a software project (also when it comes to money (especially the bold ones)) ... and they provide the possibility of doing optimization when actually needed ...

this probably isn't the answer you wanted, but I hope it helps ...

Optimizing is often really just a waste of time, what is more important is optimizing for maintainability:

- keep the code simple (see: KISS)
- move duplicate code into functions (see: DRY)
- optimize comments (as few comments as possible, as many as necessary)
- remove unused code

Those changes do not necessarily affect the speed of the program but they reduce the time needed to implement changes and fix bugs so they have real business value as they save expensive developer time.

If you're having to explain why there is a need for optimisation, then you're already on a low-win-rate path. Unfortunately, the best argument is when there is a situation where performance is becoming a noticeable problem (e.g. customers complaining) - this makes people sit up and listen if they're getting constant ear-ache about it. Often, the solution can be that throwing hardware at it is the best/cheapest solution.

However for me, performance is important, when working on highly scalable systems, you need to spend time on optimisations. It should at the very least be on your mind while you're developing and a lot of things can be swallowed up in the dev process (as opposed to after you've finished, then going back to optimise).

e.g.
- What happens if my database has 100,000,000 records instead of 100,000? (you can easily fool SQL Server into thinking there are more records in a table than there actually are to see what the effect on the execution plan would be).

If you're being pro-active about optimisation (which I personally think is good), then you need to try and come with a real-world meaning to your boss. e.g. "if we optimise our site by cutting down the page size, we could save x amount of bandwidth a month, saving us xx much money".

You do have to be careful about micro-optimisations though as you can micro-optimise until the cows come home, for much lower benefit.

You could just wait until your boss says "Why is it taking so long?" Then you could say "Hey, boss, we could buy 10 times as much hardware, or I could spend a couple days and make it 10 times faster". Here's an example.

Added: If your boss says "Well, why did you make it slow in the first place?" I would have no trouble explaining that he could hire God's Right Hand Programmer, and the code as first written would still have room for optimizing.

If you are on a relational database (and your not working with trivial applications).... you cannot afford to write bad database queries. Since you are you using a red-gate tag I assume you are on SQL server. If you design a bad schema (including queries) on SQL Server a database server when you start getting moderate traffic you will be forced to scale in horrendous ways. Once you get a high level of traffic you will be screwed -- and will need to hire a very expensive consultant/DBA to undo the mess -- use this argument with your boss :D.

What I am trying to get at is even though SQL is declarative and it doesn't appear that you have to pay attention to performance -- it doesn't really work that way. You have to design your data model by keeping in mind the way it is going to be used.

If you are writing business logic in application code ... well that doesn't matter much -- you can fix that latter with less trouble if performance becomes an issue. Most business logic these days is stateless (especially if its web-based or web-service based) meaning you can just chuck more machines at the problem -- if it's not than its just a bit harder.

Every time something takes long than it would have if the refactoring had been done ahead of time make sure your boss hears about it.

IMO, It's less a case of convincing that optimizations need to be done (and based on your Boss's responses, you're not going to win this fight), and more a case of always building in refactoring time into your estimates.

Do a cost analysis...

This will take me X hours and will mean every project needs 50% less data storage. Based on us doing 2 projects a year using on average 100Gb space, that will save $Y after a year, $Z after 3 years.

Crude example but it suffices.

Ultimately you and your boss are both wrong... you think code has to be fast (primarily) to satisfy your idealism, he doesn't care at all as he wants to deliver products.