Saturday, January 31, 2015

To a junior DBA

At UKOUG Tech conference in Liverpool last year I had an interesting talk with a DBA that had two years of experience. Being 23 years the DBA naturally stood out from the crowd. I talked about my thoughts on what is important for a DBA, and since I spend way too much on user group activities and conferences I argued that going to such events are useful for a DBA, and continued with other ramblings about what I think is important for a DBA to do. I hope I stopped soon enough, but stop talking and start listening is an art. Anyway, I decided to write down a list of recommendations for a new DBA. This is what I think is important.

Backup and recovery stands out. The database is your responsibility, that is why you are hired as a DBA. In the database, which in earlier times also was called a databank, the data is the money.  If you can't get the database back after a disaster they will blame you and no other. In fact if you can imagine one reason why you in some situation will not be able to recover the database you should mail it to your manager now. The concept of backup and recovery should be easy to grasp, but realities are that many different scenarios exist and how to deal with them depends on the situation. There are plenty of stuff written on this from masters of Recovery Manager (RMAN) to people more dedicated to risk management. This subject is more than just getting the database back running, it is also about keeping the system available, guard against data loss and testing systems for faults.

  • Have a fire drill once in a while to test your routines and skills.
  • Get some clear policies in place like, how much data are you allowed to lose, and how much time (downtime) do you have to recover your database. 
The costs and implementation will depend on  this. If you cannot lose any data and management starts talking about 99.999% uptime a simple backup is not enough. "How much data can you lose?" Is a phrase that can freak people out, but if you can't lose data, is your system  and routines really prepared for that?

Tuning and Optimisation is by many thought of as your problem and your problem only. Reality is that when a system is not performing it is usually not the DBA's fault. Obviously, a badly configured database will not work well, but the Oracle database performs remarkably well with default configurations.  Your database depends on the server it is running on, the network between the database and connected clients, and so on. But most frequently you will discover this: the reason for bad performance is bad design.

Design as in "invented by a developer who is a single user on his own PC with lots of CPU and memory, but little data and load to reveal bad algorithms". Tuning is a word looked down on by some in the community since it came from an era where quick fixes and esoteric queries would reveal problems in the database; with the right secret revealed to you, you could do some magic and have everything run perfectly. Still some people are tempted by black magic, so be critical to what Google brings you.

Optimisation on the other hand is taking what you have, see where the time is wasted, and take away as much as you can (or want) by replacing one or a few parts of the job by something that runs better. Often a redesign is necessary, or if you are lucky, fiddling with an index may do the job.

My best advice is to find a way to identify where time is wasted and zooming in on that. This means profiling in some way and work from there. By having some methods ready to detect waste or a time sink, you can stop spending time improving on what is already working reasonably well. In other words, stop the parameter tweaking (nobody changes the db_block_size these days), take one step back and ask, what is going on here. What you learned in computer science at the university may come very useful here. To put your mind on the right track I cannot think of a better place to start than to read through the papers at Method R, especially Thinking Clearly About Performance.

Get involved in a user group and go to conferences, meetups or whatever is easy for you. Many companies have many developers or IT-workers, but only one or two DBAs. In the community there are people who love to talk about their work, they will share experience with you, and tell you what is not in the manuals. If your manager won't send you to a conference, here are some arguments you may try:

  • One course covers one subject, but a conference covers many, during one intensive conference you can get inputs and learning in many fields.
  • Problems you may have at work can be discussed by others at the conference. Chances are that what you are going through is interesting and valuable input to others and you are likely not the first one to go through this.
  • Make friends and discover that you are not alone. Getting friends and contacts outside your pool is possibly the best part of it. Meeting people IRL and being in touch on many channels is priceless to me.
  • Many conferences are run by the community, meaning that a demand for a profit margin is lower at conferences than at a course delivered by a company. It translates to more learning for the money than at most other places.

Find a social network that works for you and become visible. Twitter works well for me, I meet likeminded people there and discover other things we have in common besides Oracle tech. Most of my friends I have now come from the Twitter community.

Take notes and start a blog Starting as a DBA today must be different from how it was 10-20 years ago. As you go, take notes and post it on your own blog. Even mistakes; DBAs are busy, if other can learn from your mistakes it is a valuable contribution. The problems you have now may be more relevant for another junior DBA than a beginner's post written many years ago. And, you will be surprised how useful your procedures will be later when you have to go back and perform a task you haven't done for a while.

If you are a senior DBA reading this, maybe you should blog about what you learned and want to pass on? I think many have had an experience that stood out, something that put their mind on the right track. Blog about it!