February 2009
Mon Tue Wed Thu Fri Sat Sun
« Jan   Mar »
 1
2345678
9101112131415
16171819202122
232425262728  

Month February 2009

Links for the Week of 2009-02-25

SQL Server

Index-Index-Index-Index-GOOSE! (Part One) The always short, and occasionally humorous, Tim Ford dives into the statistics DMVs and returns with a good explanation of what they are and how to use them.

Suggested DBA Work Plan If you’re like me, you’re not a production DBA, yet. I’m an accidental DBA who became a development DBA who is happily on his way to becoming a production DBA. Rod Colledge put together a great list of daily, weekly, and monthly tasks to help keep your database running smoothly.

SQL Server 2008 Proximity Search With The Geography Data Type This is just plain cool, thanks Denis!

Development

Visual Studio 2010 Videos Get started learning visual studio now! Denis Gobo was kind enough to round up some links to VS2010 training videos from the Microsoft folks. Looks like some great features are coming for people out there in development land.

Stuff & Things

Three Panel Soul – On Rabbits Three panels of a comic. This one is about sneaky, sneaky wabbits.

AMP Font Viewer Do you like fonts? I like fonts. We should hang out. Bad references to Idiocracy aside, AMP Font viewer is a free font manager for windows. If you, like me, have multiple gigabytes of TrueType and OpenType fonts, then this will be a great managing that mess of fonts.

This was just too cool to not share

Production Problems, Community, and You

How many of you out there have had production issues? You, the liar in the back, put your hand up. Well, we experienced a major performance issue in production last week.

For those that don’t know, I’m not a production DBA. I’m what they call a “development DBA.” I write stored procedures, tune the SQL, write indexes, and work with developers to ensure speedy data access. As someone who comes from the software development world, it’s a logical place to be. But, history aside, I don’t possess the knowledge of the SQL Server internals like many of my production DBA brethren.

Back to the problem at hand. On Wednesday, the users started noticing exceptional slow downs in the application. These weren’t the usual slow downs where things took a little bit longer than expected, queries that normally took 2 seconds were taking minutes to come back. Since our ASP.NET application is, largely, a passthrough that calls stored procedures and loads the results into a datagrid, we quickly ruled out application performance. Plus, I tested some stored procedures on the production database and they ran slowly. (I say our application is largely a passthrough, but I don’t mean to do a disservice to the developers. It is quite a sophisticated piece of software, it just doesn’t do a lot of heavy lifting.)

Luckily, I was able to leverage a few resources to resolve this issue.

1) Blogs

I subscribe to a lot of blogs. A LOT of blogs. From reading Brent Ozar’s blog, I remembered that he has a section at the top of his website on using perfmon and another on performance tuning. It also helps that he recently presented at the CBusPASS user group on using perfmon. I fired up my browser, went to Brent’s blogs, and started working with the monitoring set up that he outlined. This got me going pretty quickly and gave me a great deal of information.

In addition to checking up on Brent’s step by step instructions, I remembered that another SQL Server blogger had posted a SQL script to find the top 10 slowest running queries and their execution plans. I couldn’t (and still can’t) remember who wrote it. Which leads to:

2) Social Networks

In this case, Twitter. I opened up TweetDeck and sent out a tweet with a question asking if anyone had seen that blog post. Someone remembered what I was talking about and sent me a link. Since I’m not much of a production DBA, yet, I kept raising questions on twitter which were rapidly answered by many different people. I was able to work with them to figure out the best solution to my specific questions and move forward quickly.

Within 15 minutes, I was armed with a methodology to track down root cause of the problem as well as a query that would help me pinpoint any specific query problems. Without the collective knowledge of the SQL Server DBA community, it would have taken me a lot longer to understand and solve the problem.

It turns out that the problem was a missing column in an index that caused a full table scan on one of the largest tables in the database which, in turn, caused locking issues.

Links for the Week of 2009-02-20

SQL Server

Missing Index Information and Query Stats Grant Fritchey put together a nice little query to pull XML query plans out of the DMVs. Thankfully, these plans can also tell use which indexes are missing

Scalar Functions vs. Table Valued Functions vs. Inline Code Aaron Alton does some testing on these three to determine which is the most performant.

Development

Reducing the cost of getting a stack trace Exceptions are expensive. Getting a full stack trace is expensive. Ayende Rahien shows you how to make this a little bit cheaper.

GUI design and prototyping tools We’ve all had to create a UI at some point. Fabrice Marguerie has a list of some great UI prototyping tools. (Also check out the UI design pattern library post)

Stuff & Things

20 tips for better conference speaking Cameron Moll is a designer whose skills I’ve greatly admired. But, in addition to being a phenomenal designer, he’s also a great speaker. Here he shares some tips to help the rest of us become better speakers too.

Public Speaking: A Primer Paul Randal provides a nice introduction to public speaking. Like Cameron’s article, there are some great items in there. Yes, I do have speaking and presenting on the brain. I’m going to be presenting three times in the next two months.

What was your first computer and what were some of your favorite games?

Denis Gobo started up a new meme about your first computer and favorite games.

My first computer was a Tandy 1000SX with 640KB of RAM that my family got when I was 9 or 10, I think. My dad quickly overclocked it to an astounding 7.6 MHz of pure Intel 8088 power and added a 20 MB hard drive. For the time, it was an amazing computer.

My two favorite games were Elite and Starflight. Proving that my love of sci-fi and space opera began a very long time ago.

That Tandy 1000, with its noisy dot matrix printer and 16 color graphics, served as my primary computer until 1999 when I received an upgrade to a 60 MHz Gateway machine.

I’m going to go ahead and tag Rick Kierner since most of the data folks seem to have been tagged at this point.

Automating T-SQL Testing

I recently became even more lazy than usual and was looking for a way to test all of my stored procedures (there are over 180 of them in my production database). Since we have a limited hardware and software budget, buying extra software isn’t a part of the equation. So, I took it upon myself to put together a solution.

My first task was to figure out how to accomplish this. Since my solution needed to take into account the dynamic nature of the environment, I realized that I needed a way to catalog the stored procedures and any possible combination of parameters. I started by creating a table to hold the names of stored procedures and the parameters:

  CREATE TABLE [test].[Procedures](
  	[ProceduresId] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY,
  	[ProcedureName] [nvarchar](128) NOT NULL,
  	[Run] [int] NOT NULL,
  	[ParameterName] [nvarchar](255) NULL,
  	[ParameterType] [nvarchar](255) NULL,
  	[ParameterOrder] [int] NULL,
  	[ParameterValue] [nvarchar](255) NULL
  );

A quick explanation of the columns in the table:

  • ProcedureName – clearly this stores the name of the procedure.
  • Run – if a procedure has optional parameters, the Run column can be used to distinguish between different combinations of parameters and group them together.
  • ParameterName – the name of the parameter.
  • ParameterType – this isn’t used in the dynamic code coming up later, but it does help to sort things out in your brain.
  • ParameterOrder – again, like ParameterType, this isn’t used in the dynamic code that I wrote, but it does help to keep things in mind. You could easily modify the T-SQL to write the parameters out in order instead of using named parameters.
  • ParameterValue – the value of the parameter being passed in during the test run. You could increase this to VARCHAR(4000) or something like that, depending on your needs.

We have a table, but no procedure to test. Here’s a quick one that I wrote to use the Northwind database:

  CREATE PROCEDURE [dbo].[FindCustomer] (
    @CompanyName NVARCHAR(40) = NULL,
    @ContactName NVARCHAR(30) = NULL
  )
  AS
  SET NOCOUNT ON;

  IF @CompanyName IS NULL AND @ContactName IS NULL
  BEGIN
    RAISERROR('Either Company Name, Contact Name, or both must be supplied', 16, 0);
    RETURN
  END

  DECLARE @sql AS NVARCHAR(MAX);
  DECLARE @newline AS NVARCHAR(2);

  SET @newline = NCHAR(13) + NCHAR(10);

  SET @CompanyName = '%' + @CompanyName + '%';
  SET @ContactName = '%' + @ContactName + '%';

  SET @sql = '
  SELECT CustomerID,
         CompanyName,
         ContactName
    FROM dbo.Customers
   WHERE 1 = 1 ' + @newline;

  IF @CompanyName IS NOT NULL
    SET @sql += '   AND CompanyName LIKE @company_name ' + @newline;

  IF @ContactName IS NOT NULL
    SET @sql += '   AND ContactName LIKE @contact_name ' + @newline;

  EXEC sp_executesql @sql,
       N'@company_name AS NVARCHAR(40), @contact_name AS NVARCHAR(30)',
       @CompanyName, @ContactName;

  GO

Pretty cool, eh? It just selects customers that match either the Company Name, Contact Name, or both. But not neither. If neither parameter has a value, the procedure will throw an error.

Now, let’s load that table we created:

  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 1,	NULL,	NULL,	NULL,	NULL);
  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 2, '@CompanyName',	'NVARCHAR(40)',	1, 'A');
  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 3, '@CompanyName', 'NVARCHAR(40)', 1, 'A');
  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 3, '@ContactName', 'NVARCHAR(30)', 2, 'Maria');

Fantastic, now there’s data and a procedure to test. How do we go about running these tests? Well, with a little bit of dynamic SQL trickery we can accomplish this pretty easily.

Note: I’ve updated this to not have syntax highlighting because of a problem with the GeSHi syntax highlighting code that was turning [text()] into [TEXT()]. This, in turn, was causing a syntax error in the resulting SQL, which I tracked down through the SQL Server Central forums.

  SET NOCOUNT ON;

  IF OBJECT_ID(N'tempdb..temp') IS NOT NULL
    DROP TABLE #temp;

  IF OBJECT_ID(N'tempdb..procs') IS NOT NULL
    DROP TABLE #procs;

  DECLARE @s AS NVARCHAR(1);
  DECLARE @sql AS NVARCHAR(MAX);
  DECLARE @newline AS NVARCHAR(2);
  DECLARE @proc_sql AS NVARCHAR(MAX);

  SET @s = N'''';
  SET @sql = N'';
  SET @newline = NCHAR(13) + NCHAR(10);

  SET @sql = N'
  DECLARE @error AS BIT;
  DECLARE @errors TABLE (
    ErrorNumber INT,
    ErrorSeverity INT,
    ErrorState INT,
    ErrorProcedure NVARCHAR(126),
    ErrorLine INT,
    ErrorMessage NVARCHAR(2048)
  );

  SET @error = 0;

  BEGIN TRANSACTION;' + @newline + @newline;

  SET @proc_sql = N'BEGIN TRY
    --myproc--
  END TRY
  BEGIN CATCH
    SET @error = 1;

    INSERT INTO @errors VALUES
    (
  	  ERROR_NUMBER(),
      ERROR_SEVERITY(),
      ERROR_STATE(),
      ERROR_PROCEDURE(),
      ERROR_LINE(),
      ERROR_MESSAGE()
    );
  END CATCH' + @newline + @newline;

  SELECT ProcedureName,
         Run,
         ParameterOrder,
         ParameterName
         + N' = '
         + QUOTENAME(ParameterValue, @s) AS x
    INTO #temp
    FROM test.[Procedures]
   ORDER BY Run, ParameterOrder;

  SELECT N'EXEC '
         + ProcedureName
         + N' '
         + COALESCE([params], '') AS command
    INTO #procs
    FROM (SELECT t.ProcedureName,
                 t.Run,
                 STUFF((SELECT N', '
                               + x AS [text()]
                          FROM #temp AS t2
                         WHERE t2.ProcedureName = t.ProcedureName
                           AND t2.Run = t.Run
                         ORDER BY t2.ParameterOrder
                           FOR XML PATH('')), 1, 1, '') AS [params]
            FROM #temp AS t
    ) AS x
  GROUP BY ProcedureName, [params], Run
  ORDER BY Run;

  SELECT @sql += REPLACE(@proc_sql, N'--myproc--', command)
    FROM #procs;

  SET @sql += @newline + @newline
            + 'IF @error = 1
    SELECT * FROM @errors;

    ROLLBACK TRANSACTION;'

  EXEC sp_executesql @sql;

  DROP TABLE #temp;
  DROP TABLE #procs;

This procedure creates dynamic SQL that will call all of the stored procedures that we’ve put in the test.Procedures table. You could even integrate this with something like TSQLUnit to create a more robust, dynamic, unit testing framework.

With some additional trickery and modifications to the test.Procedures table (or a second table), this could be modified to call every function you’ve created as well.

Column Order Does Matter… Sometimes

I was debugging a query that was giving me some really strange results. Without pasting the query here, basically it does some interesting math to compute the estimated amount of greenhouse gases produced in tons of CO2, N2O, and CH4. So far this is pretty simple, right?

Well, depending on the type of location we have different possible queries and the results are merged with a union and then summed. This is still pretty standard.

Here’s where things got tricky.

The query returned results, we double checked them, and then moved it all to production. Over time there was a need to dynamically select which emissions the users wanted to see, so I changed the query to be nice and dynamic. Unfortunately, as a side effect, I introduced a bug that went completely uncaught because it didn’t actually throw an error.

Basically, my select ended up looking like this (but with real code):

  SELECT CO2,
         CO2Factor,
         N2O,
         N2OFactor,
         CH4,
         CH4Factor
    FROM MagicTable
  UNION ALL
  SELECT CO2,
         N2O,
         CH4,
         CO2Factor,
         N2OFactor,
         CH4Factor
    FROM OtherMagicTable

Can you spot the error?

Basically, since the columns weren’t in the same order, there was something goofy going on in the background and sums weren’t happening correctly and N2O was coming back with a number suspiciously similar to the CO2 number. (If you know anything about greenhouse gas emissions, this should strike you as completley wrong unless I was running a nitrogen factory or something.)

Moral of the story is: Column order can matter.

Unfortunately, I don’t actually understand the why of this and I feel rather bad since one of my primary goals is to understand why errors are occurring before I push fixes into production. In this case, customers had noticed and were complaining. Any ideas in the comments are more than welcome and might just win you a prize. (The prize is my undying thanks.)

Links for 2009.02.12

SQL Server

Update Statistics Before or After an Index Rebuild? Colin Stasiuk talks about when you should update stats in relation to rebuilding/reorganizing indexes. I’m not just linking to this because I was the catalyst for his blog post, but because there are some great things in here and I learned a lot from it.

Best Practices for installing SQL Server service packs, hotfixes, cumulative updates Beatrice Nicolini put together a great list of best practices for keeping your SQL Server installations up to date. Some of this just came intuitively, some of it I didn’t know until I read this.

Index Fragmentation Findings: Part 2, Size Matters Part two of Brent Ozar’s enlightening series on Index Fragmentation. Brent does a great job of explaining index fragmentation, what it means, and why it matters in ways that even the thickest developer turned DBA can understand.

Development

Build a Silverlight game, win $5,000 I don’t think this needs any more description – make a game, ???, win cash. Thanks to Brian H. Prince for bringing this to everyone’s attention!

Things you now know I tagged Rick Kierner in our latest goofy meme and he reciprocated by providing some great advice for developers/DBAs/whatevers on how to enhance their career. Working on a project with Rick really changed the way I looked at a lot of the things I do on a daily basis and I owe him more than he realizes.

General Stuff

How To Drag Your Butt Through That (Fill In The Blank) Book Aaron Alton gives some motivational hints on how to make it through that tedious [subject goes here] book that you’re currently struggling to get through. Now you can get fit AND get nerdy at the same time!

Found Emoticons of the First Two Decades of the 21st Century William Gibson came across this list of electrical plugs somewhere and flipped it around to make some crazy emoticons. My favorite quote from the text is “Sixth row, sixth square from right, is a very bad romantic feeling that nobody will experience until 2012.”

Classy Games (Part 2 of 2) For those of you not familiar with it, Something Awful is normally a humor web site that can best be described as “mildly not safe for work”. Normally I wouldn’t link to them (even though I’m pretty sure I have before). Every Friday they run a Photoshop contest. This week it’s a re-imagining of classic video game covers. There are some artistic gems in here.

CBusPASS Meeting 2009.02.12

We had our second user group meeting tonight, and I would like to send a special thanks to Brent Ozar for being willing to present. We had around twenty people in attendance physically and another dozen or so in attendance virtually via Live Meeting.

The resources from Brent’s presentation on Perfmon and Profiler are available online at the CBusPASS Meeting Archive.

Brent’s posts about Perfmon and analytics are available online:
http://www.BrentOzar.com/perfmon
http://www.BrentOzar.com/perfmoncloud

In addition, Brent mentioned Excel Table Analysis Tools for the Cloud:
http://www.SQLServerDataMining.com/cloud

In addition, Brent is the Editor-in-Chief for SQL Server Pedia (where this blog is also syndicated).

Thanks again, Brent, for taking your time to present. I’d also be an awful user group leader if I didn’t thank my steering committee for their invaluable help, Battelle For Kids for providing the meeting space, and Quest Software for providing the Live Meeting.

For those who missed out, Brent will be giving this presentation again on February 19 with the West Michigan SQL Server User Group. You can find more info on Brent’s upcoming events page.

Things You Now Know…

Things You Know Now…

Colin Stasiuk (@BenchmarkIT) has tagged me in his recent blog post Things You Know Now…

Here are some of mine, in no particular order of importance or relevance:

Accept Failure

Failing is a part of live. Well, I prefer to think of it as not succeeded, but that’s a story for a different blog post that I’m never going to write. Anyway, the key is to accept the fact that you’re going to make mistakes. They happen to the best of us. Hell, I get called out on spelling errors in my blog posts sometimes and I have a degree in English.

Early on in my career, I thought that since I had a job in IT, I was supposed to act like I knew what I was doing and that I wasn’t supposed to make mistakes. As a result, I heaped a ton of pressure on myself to succeed and when I did, eventually, fail, it took a lot out of me and I didn’t like to own up to my failures because I felt like I shouldn’t have been making them.

These days, I have a different attitude about failing. Instead of trying to hide it, or at least not own up to my failures, I openly admit to them. Heck, I’ve openly admitted to some pretty stupid things on twitter and in this blog. Instead of hanging my head in shame and hiding under my desk, I go out on the internet and find the answer to the problem, fix whatever I broke, and then openly and publicly admit to it. I’ve often said that the key to learning is time, failure, and semi-public humiliation. I chose to circumvent the semi-public humiliation by openly sharing my failures and the solution in the hope that someone will learn from what I’ve done and won’t have to repeat my mistake.

Don’t be a Know-It-All

By this I mean you don’t have to know everything. Another rookie mistake on my part was to assume that I had to know everything. Every time a new technology hit the streets, or even was released in alpha, I tried to learn it. I would run out and find a book or read articles. I spent so much time chasing the new hotness that I didn’t take the time to focus on a lot of core skills and build depth of knowledge. Admittedly, as a result I can talk conversantly about a lot of technical topics that I would have no knowledge of otherwise (Smalltalk, anyone?).

Others have hit on this in the past, but I cannot express the value of being a T-shaped professional. There’s nothing wrong with having a broad, shallow knowledge base to draw from, but it’s important to pick something you love and drill down into it. Master a technology or platform. Don’t take this to mean that I think everyone needs to be a one dimensional professional. Keep learning. Even when you’re mastering analysis services or WPF or whatever, take the time to learn about Flash Remoting or Processing or Ruby on Rails or whatever strikes your fancy. You never know where you’re going to find an idea that helps you solve your current work problem.

Get Involved

Involving yourself in the developer community is such a wonderful feeling. You get to share knowledge and converse with other smart, talented, passionate people. Whether you’re going to a local user group, answering forum questions, or having an informal meet up with local developers you’re giving back your knowledge and learning from others.

I was terrified the first time that I went to a developer group. I thought that I would get noticed as some kind of fraud or n00b who didn’t know anything. Instead I met some great developers and reconnected with old friends and former co-workers. It’s been a blast and, as a result of my tentative involvement, I’ve restarted the local PASS chapter.

Whatever you do (blog, newsgroup, StackOverflow, MSDN, user group), involve yourself and do it with passion. There’s nothing like giving back to the community that you’ve learned so much from.

Tag, suckers!

I’m calling out these folks to play along with our fun game:

Links for the Week of 2009-01-06

SQL Server

Tomorrow’s Microsoft BI Platform Derek Comingore (SQL Server MVP) gives a great overview of the Kilimanjaro release of SQL Server. This is a solid overview of the pieces and parts that will make up the 2010 feature pack release before the release of SQL 11 (in 2011).

Top 10 SQL Server 2008 Features for the Database Administrator Mike Weiner and Burzin Patel put together a great list of features for DBAs in SQL Server 2008. A lot of enhancements were made in the BI space and in T-SQL itself and it’s always good to get a refresher on what’s out there to help us all with maintaining and administering our databases.

Back To Basics: Clustered vs NonClustered Indexes; what’s the difference? Denny Cherry put together a great refresher on the differences between the two types of index available in SQL Server. This is a solid overview that you should keep around for anyone who asks you that question.

Development

Arguments against using an ORM layer – an ammunition stockpile Corey Trager tackles a very touchy argument: to ORM or not ORM. I’ve been on both sides of the fence during my career, and it’s a very difficult argument to make from either side. Luckily, Corey’s own argument is both bolstered and refuted by the comments on his blog. It’s a great read no matter what your opinion is.

General

Technical Presentations: Be Prepared for Absolute Chaos When you’re giving a presentation, you need to be prepared for whatever can go wrong. Scott Hanselman relates a recent experience of his presenting at TeachReady8 (an internal Microsoft conference), and talks about the problems he ran into giving a presentation.

Learning from failure is overrated Jason Fried at 37signals goes off a little bit about the value placed on failure these days. His thoughts are some great things to keep in mind when you’re reviewing past failures and successes.

Online Credibility, How to build it and how to lose it in an instant We all value our online presence. Well, I hope you do. Johnathan Kehayias provides some solid advice on how to monitor your own behavior in forums, both for the sake of your own credibility, but also so that we’re all able to contribute and help each other while providing solid, valuable, advice.

This site is protected with Urban Giraffe's plugin 'HTML Purified' and Edward Z. Yang's Powered by HTML Purifier. 401 items have been purified.