Tag dynamic sql

Upcoming PASS AppDev SIG Presentation

On March 31st, at 1 PM EST, I will be giving a presentation via Windows LiveMeeting for the PASS AppDev SIG. I know the next question on your mind, it’s probably “When are you going to shut up?” “Jeremiah, pray tell, what is the topic of this delightful and amazing presentation?” Well, folks, when you tune in on March 31st you be treated to my presentation, which is titled: Dynamic SQL: What’s all the fuss about, anyway?

As soon as I have LiveMeeting info, I will post it here and make sure an announcement gets out to the public.

If you’re interested in becoming a part of the AppDev SIG get in touch with Todd Robinson, his contact info is on the SIG’s web page, and ask for more information.

Crafting Presentations

First off, why did I title this “Crafting Presentations” instead of something along the lines of “How to Write a Kick Ass Presentation”? Two reasons. One, I didn’t want to. Two, I think that there’s more involved in a presentation than writing.

Yes, there is a lot of writing involved in a presentation, but it’s a different kind of writing than a targeted blog post. A presentation has to quickly convey information to an audience that might only be marginally interested in the topic or who might just be there for the free pizza, although I hope people come to my user group for more than free pizza and soda. Presentations also engage the audience on multiple levels. There are the words and images on the slides, the speaker, and any demonstrations. All of this needs to flow together well. It’s not just writing for the screen, it’s a mixed multimedia extravaganza! (Maybe I should work on a presentation involving a fog machine and a laser light show…)

Just the slide deck and notes alone occupy a considerable amount of time and effort, but the code samples require significant attention. While putting together the code samples for tomorrow’s presentation on dynamic SQL, I think I wrote about 4 times as much SQL as I ended up using. Taking into account that my sample code tops out around 800 lines, that’s a lot of sample code to write.

Why did I write so much sample code only to throw it away? Was it all crap? No. The sample code that I threw away was actually pretty decent code. It just didn’t work clearly enough. There were more than a few examples that I wrote that did more to show just how clever I could be than they did to illustrate the points in my presentation. And, at the end of the day, I really do want people to know how clever I am. And, at the end of the day, I really want the audience to come away with a clear understanding of the topic.

A few people have asked me how much time I spend on presentations. I never really had a good answer until I paid attention while I was putting this presentation together. It turns out that I spend between 20 and 40 hours putting each presentation together. A lot of this time is spent coming up with good code examples and making sure that each topic clearly flows to the next topic and that the code samples follow a logical progression starting from the simple and building toward a complete finale – like a really good movie about a cat and dog that lose their family only to journey across several states and finally be reunited in with the missing family in the last five minutes in a heart rending scene that leaves the audience in tears.

There’s a lot of craft involved in a great presentation. Everything needs to flow from beginning to end, educate the listeners and, ultimately, leave them in tears.

Automating T-SQL Testing

I recently became even more lazy than usual and was looking for a way to test all of my stored procedures (there are over 180 of them in my production database). Since we have a limited hardware and software budget, buying extra software isn’t a part of the equation. So, I took it upon myself to put together a solution.

My first task was to figure out how to accomplish this. Since my solution needed to take into account the dynamic nature of the environment, I realized that I needed a way to catalog the stored procedures and any possible combination of parameters. I started by creating a table to hold the names of stored procedures and the parameters:

  CREATE TABLE [test].[Procedures](
  	[ProceduresId] [int] IDENTITY(1,1) NOT NULL PRIMARY KEY,
  	[ProcedureName] [nvarchar](128) NOT NULL,
  	[Run] [int] NOT NULL,
  	[ParameterName] [nvarchar](255) NULL,
  	[ParameterType] [nvarchar](255) NULL,
  	[ParameterOrder] [int] NULL,
  	[ParameterValue] [nvarchar](255) NULL
  );

A quick explanation of the columns in the table:

  • ProcedureName – clearly this stores the name of the procedure.
  • Run – if a procedure has optional parameters, the Run column can be used to distinguish between different combinations of parameters and group them together.
  • ParameterName – the name of the parameter.
  • ParameterType – this isn’t used in the dynamic code coming up later, but it does help to sort things out in your brain.
  • ParameterOrder – again, like ParameterType, this isn’t used in the dynamic code that I wrote, but it does help to keep things in mind. You could easily modify the T-SQL to write the parameters out in order instead of using named parameters.
  • ParameterValue – the value of the parameter being passed in during the test run. You could increase this to VARCHAR(4000) or something like that, depending on your needs.

We have a table, but no procedure to test. Here’s a quick one that I wrote to use the Northwind database:

  CREATE PROCEDURE [dbo].[FindCustomer] (
    @CompanyName NVARCHAR(40) = NULL,
    @ContactName NVARCHAR(30) = NULL
  )
  AS
  SET NOCOUNT ON;

  IF @CompanyName IS NULL AND @ContactName IS NULL
  BEGIN
    RAISERROR('Either Company Name, Contact Name, or both must be supplied', 16, 0);
    RETURN
  END

  DECLARE @sql AS NVARCHAR(MAX);
  DECLARE @newline AS NVARCHAR(2);

  SET @newline = NCHAR(13) + NCHAR(10);

  SET @CompanyName = '%' + @CompanyName + '%';
  SET @ContactName = '%' + @ContactName + '%';

  SET @sql = '
  SELECT CustomerID,
         CompanyName,
         ContactName
    FROM dbo.Customers
   WHERE 1 = 1 ' + @newline;

  IF @CompanyName IS NOT NULL
    SET @sql += '   AND CompanyName LIKE @company_name ' + @newline;

  IF @ContactName IS NOT NULL
    SET @sql += '   AND ContactName LIKE @contact_name ' + @newline;

  EXEC sp_executesql @sql,
       N'@company_name AS NVARCHAR(40), @contact_name AS NVARCHAR(30)',
       @CompanyName, @ContactName;

  GO

Pretty cool, eh? It just selects customers that match either the Company Name, Contact Name, or both. But not neither. If neither parameter has a value, the procedure will throw an error.

Now, let’s load that table we created:

  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 1,	NULL,	NULL,	NULL,	NULL);
  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 2, '@CompanyName',	'NVARCHAR(40)',	1, 'A');
  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 3, '@CompanyName', 'NVARCHAR(40)', 1, 'A');
  INSERT INTO test.Procedures VALUES
    ('FindCustomer', 3, '@ContactName', 'NVARCHAR(30)', 2, 'Maria');

Fantastic, now there’s data and a procedure to test. How do we go about running these tests? Well, with a little bit of dynamic SQL trickery we can accomplish this pretty easily.

Note: I’ve updated this to not have syntax highlighting because of a problem with the GeSHi syntax highlighting code that was turning [text()] into [TEXT()]. This, in turn, was causing a syntax error in the resulting SQL, which I tracked down through the SQL Server Central forums.

  SET NOCOUNT ON;

  IF OBJECT_ID(N'tempdb..temp') IS NOT NULL
    DROP TABLE #temp;

  IF OBJECT_ID(N'tempdb..procs') IS NOT NULL
    DROP TABLE #procs;

  DECLARE @s AS NVARCHAR(1);
  DECLARE @sql AS NVARCHAR(MAX);
  DECLARE @newline AS NVARCHAR(2);
  DECLARE @proc_sql AS NVARCHAR(MAX);

  SET @s = N'''';
  SET @sql = N'';
  SET @newline = NCHAR(13) + NCHAR(10);

  SET @sql = N'
  DECLARE @error AS BIT;
  DECLARE @errors TABLE (
    ErrorNumber INT,
    ErrorSeverity INT,
    ErrorState INT,
    ErrorProcedure NVARCHAR(126),
    ErrorLine INT,
    ErrorMessage NVARCHAR(2048)
  );

  SET @error = 0;

  BEGIN TRANSACTION;' + @newline + @newline;

  SET @proc_sql = N'BEGIN TRY
    --myproc--
  END TRY
  BEGIN CATCH
    SET @error = 1;

    INSERT INTO @errors VALUES
    (
  	  ERROR_NUMBER(),
      ERROR_SEVERITY(),
      ERROR_STATE(),
      ERROR_PROCEDURE(),
      ERROR_LINE(),
      ERROR_MESSAGE()
    );
  END CATCH' + @newline + @newline;

  SELECT ProcedureName,
         Run,
         ParameterOrder,
         ParameterName
         + N' = '
         + QUOTENAME(ParameterValue, @s) AS x
    INTO #temp
    FROM test.[Procedures]
   ORDER BY Run, ParameterOrder;

  SELECT N'EXEC '
         + ProcedureName
         + N' '
         + COALESCE([params], '') AS command
    INTO #procs
    FROM (SELECT t.ProcedureName,
                 t.Run,
                 STUFF((SELECT N', '
                               + x AS [text()]
                          FROM #temp AS t2
                         WHERE t2.ProcedureName = t.ProcedureName
                           AND t2.Run = t.Run
                         ORDER BY t2.ParameterOrder
                           FOR XML PATH('')), 1, 1, '') AS [params]
            FROM #temp AS t
    ) AS x
  GROUP BY ProcedureName, [params], Run
  ORDER BY Run;

  SELECT @sql += REPLACE(@proc_sql, N'--myproc--', command)
    FROM #procs;

  SET @sql += @newline + @newline
            + 'IF @error = 1
    SELECT * FROM @errors;

    ROLLBACK TRANSACTION;'

  EXEC sp_executesql @sql;

  DROP TABLE #temp;
  DROP TABLE #procs;

This procedure creates dynamic SQL that will call all of the stored procedures that we’ve put in the test.Procedures table. You could even integrate this with something like TSQLUnit to create a more robust, dynamic, unit testing framework.

With some additional trickery and modifications to the test.Procedures table (or a second table), this could be modified to call every function you’ve created as well.

Column Order Does Matter… Sometimes

I was debugging a query that was giving me some really strange results. Without pasting the query here, basically it does some interesting math to compute the estimated amount of greenhouse gases produced in tons of CO2, N2O, and CH4. So far this is pretty simple, right?

Well, depending on the type of location we have different possible queries and the results are merged with a union and then summed. This is still pretty standard.

Here’s where things got tricky.

The query returned results, we double checked them, and then moved it all to production. Over time there was a need to dynamically select which emissions the users wanted to see, so I changed the query to be nice and dynamic. Unfortunately, as a side effect, I introduced a bug that went completely uncaught because it didn’t actually throw an error.

Basically, my select ended up looking like this (but with real code):

  SELECT CO2,
         CO2Factor,
         N2O,
         N2OFactor,
         CH4,
         CH4Factor
    FROM MagicTable
  UNION ALL
  SELECT CO2,
         N2O,
         CH4,
         CO2Factor,
         N2OFactor,
         CH4Factor
    FROM OtherMagicTable

Can you spot the error?

Basically, since the columns weren’t in the same order, there was something goofy going on in the background and sums weren’t happening correctly and N2O was coming back with a number suspiciously similar to the CO2 number. (If you know anything about greenhouse gas emissions, this should strike you as completley wrong unless I was running a nitrogen factory or something.)

Moral of the story is: Column order can matter.

Unfortunately, I don’t actually understand the why of this and I feel rather bad since one of my primary goals is to understand why errors are occurring before I push fixes into production. In this case, customers had noticed and were complaining. Any ideas in the comments are more than welcome and might just win you a prize. (The prize is my undying thanks.)

Grant EXECUTE Permissions on all Stored Procedures to a Single User

I have to run similar code on a regular basis, so I thought I would share it with everyone.

Basically, it grants access on every stored procedure that meets a certain naming pattern to a single user.

-- I've updated this with Aaron Bertrand's suggestions from the comments.
-- Thanks to Aaron for helping make this better!
DECLARE @sql AS NVARCHAR(MAX);
DECLARE @newline AS NVARCHAR(2);
DECLARE @user_name AS NVARCHAR(100);
DECLARE @sproc_name_pattern AS NVARCHAR(10);

SET @sql = N''
SET @newline = NCHAR(13) + NCHAR(10);
SET @user_name = N'jeremiah';
-- escaping _ prevents it from matching any single character
-- including the wildcard makes this much more portable between DBs
SET @sproc_name_pattern = N'sproc[_]%';

-- using QUOTENAME will properly escape any object names with spaces
-- or other funky characters
SELECT @sql = @sql
              + N'GRANT EXECUTE ON '
              + QUOTENAME(OBJECT_SCHEMA_NAME([object_id])) + '.'
              + QUOTENAME([name])
              + N' TO '
              + QUOTENAME(@user_name)
              + N';'
              + @newline + @newline
  FROM sys.procedures
 WHERE [name] LIKE @sproc_name_pattern;

-- this is my version of debug code, I usually run it once with the PRINT intact
-- before I actually use sp_executesql
--PRINT @sql;

EXEC sp_executesql @sql;

Retrieve the top X random rows from a query

Let’s suppose for a minute that you want to retrieve the top x number of rows from any given query. Keep in mind that x is a value that your users can supply.

Sure, you could do string concatenation and EXEC sp_executesql ‘SELECT TOP ‘ + @count + ‘ x FROM y;’. But, then again, a vicious hacker could get all cute and decide to change your form variable to ‘; DECLARE @bad_var AS NVARCHAR(MAX); SET @bad_var = ””; SELECT @bad_var = @bad_var”DROP TABLE ” + name + ”;” FROM sys.tables; SET @bad_var = @bad_var + ”–”; EXEC sp_executesql @bad_var;’ Or something along those lines but probably infinitely more cunning.

Well, you could also accomplish this a different way using a subquery:

DECLARE @count AS int;
SELECT @count = 2;

SELECT ContactId,
       FirstName,
       LastName,
       EmployeeId,
       AddressId,
       AddressLine1,
       AddressLine2,
       City,
       PostalCode
FROM (
  SELECT ROW_NUMBER() OVER (ORDER BY NEWID()) AS rownum,
         c.ContactId,
         c.FirstName,
         c.LastName,
         e.EmployeeId,
         a.AddressId,
         a.AddressLine1,
         a.AddressLine2,
         a.City,
         a.PostalCode
    FROM HumanResources.Employee AS e
          INNER JOIN Person.Contact AS c
            ON e.ContactId = c.ContactId
          INNER JOIN HumanResources.EmployeeAddress AS ea
            ON e.EmployeeId = ea.EmployeeId
          INNER JOIN Person.Address AS a
            ON ea.AddressId = a.AddressId
) AS x
WHERE rownum <= @count;

It’s not the prettiest thing on earth, but it works. Thanks to Rick Kierner over at rickdoes.net for asking me to help out with a similar query late last week.

This site is protected with Urban Giraffe's plugin 'HTML Purified' and Edward Z. Yang's Powered by HTML Purifier. 226 items have been purified.