January 2009
Mon Tue Wed Thu Fri Sat Sun
« Dec   Feb »
 1234
567891011
12131415161718
19202122232425
262728293031  

Month January 2009

Links for the Week of 2009-01-30

SQL Server

Set based random numbers George Mastros points out that generating a random number using RAND() in a set-based operation will always return the same value for every row in the result set. What’s a DBA to do? Luckily, George also covers a great way to get around this predicament.

Hardware for SQL Server 2008 Andrew Fryer posted a few links to SQL Server 2008 Hardware recommendations from Microsoft. Good times.

The IDENTITY Property: A Much-Maligned Construct in SQL Server Aaron Alton gives a great overview of IDENTITY and why you might want to go about using it.

Development

Sharing ASP.net Session State Between Web Applications With SQL Server – Part I Back when I was a web developer I tried to figure out how to do this on multiple occasions (back in the .NET 1.1 days). Unfortunately, I never came up with any good way to accomplish this without re-implementing session state storage and project deadlines always took hold so we developed quick, hacky, brittle solutions. Looks like Alex Cuse has put together something a lot more robust than anything I’ve ever come up with. Thanks Alex! (Part 2 available here)

Software + Services in Plain English Brian H Prince has provided a link to a video featuring some sweet paper dolls to explain Software + Services (not Software as a Service). Plus, there’s some implied dating between the paper dolls when they “meet for coffee” and end up talking about their crappy IT infrastructure (total nerd date).

Löve In Two Dimensions Why the Lucky Stiff provides information on a game/graphics programming framework called Löve that’s something like a combination of Lua and SDL.

General

Hold the Coprophagia William Gibson is, hands down, my favorite author. As he works on a new novel he publishes fragments on his blog. Enjoy.

22 Most Used Free Fonts By Professional Designers The kind folks over at instantshift.com have put together a list of freely available fonts from the pros. Very good additions to anyone’s collection.

Calendar Table? Yes please!

I finally got sick of not having a calendar table, so I went ahead and made one today. It’s nothing special and it really didn’t take much time, but I thought I would go ahead and share my script in the hopes that google will index it and somebody won’t have to think about it:

CREATE TABLE [dbo].[Calendar](
	[Date] [datetime] NOT NULL,
	[FirstDayOfMonth] [datetime] NOT NULL,
	[LastDayOfMonth] [datetime] NOT NULL,
PRIMARY KEY CLUSTERED
(
	[Date] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON, FILLFACTOR = 100) ON [PRIMARY]
) ON [PRIMARY]
GO

DECLARE @the_date AS DATETIME;

SET @the_date = N'19990101'

-- Oooh, a loop! Yup, I was feeling very lazy and procedural and
-- really didn't feel like writing a nifty cross join to get 50 bajillion
-- rows or anything like that.
WHILE @the_date < (DATEADD(yyyy, 10, GETDATE()))
BEGIN
  INSERT INTO Calendar
  (
    Date,
    FirstDayOfMonth,
    LastDayOfMonth
  )
  VALUES
  (
    @the_date,
    DATEADD(mm, DATEDIFF(mm, 0, @the_date), 0),
    -- My data is not very granular by time, you might want to decrease this to
    -- three milliseconds if you have a lot of time-based data to store.
    DATEADD(ss,-1,DATEADD(mm,1,DATEADD(mm,DATEDIFF(mm,0,@the_date),0)))
  );

  SELECT @the_date = DATEADD(dd, 1, @the_date);
END

Or, if you have a numbers table, you could do something like this for a more set based approach:

DECLARE @the_date AS DATETIME;
SET @the_date = N'19990101'

INSERT INTO Calendar
(
  Date,
  FirstDayOfMonth,
  LastDayOfMonth
)
SELECT DATEADD(dd, n, @the_date),
       DATEADD(mm, DATEDIFF(mm, 0, DATEADD(dd, n, @the_date)), 0),
       DATEADD(ss,-1,DATEADD(mm,1,DATEADD(mm,DATEDIFF(mm,0,DATEADD(dd, n, @the_date)),0)))
  FROM nums

My numbers table has 1,000,000 rows in it, so running this will take “some time”. You’ll probably want to add a where clause to limit the number of days you create in your calendar table if that’s the case.

Grant EXECUTE Permissions on all Stored Procedures to a Single User

I have to run similar code on a regular basis, so I thought I would share it with everyone.

Basically, it grants access on every stored procedure that meets a certain naming pattern to a single user.

-- I've updated this with Aaron Bertrand's suggestions from the comments.
-- Thanks to Aaron for helping make this better!
DECLARE @sql AS NVARCHAR(MAX);
DECLARE @newline AS NVARCHAR(2);
DECLARE @user_name AS NVARCHAR(100);
DECLARE @sproc_name_pattern AS NVARCHAR(10);

SET @sql = N''
SET @newline = NCHAR(13) + NCHAR(10);
SET @user_name = N'jeremiah';
-- escaping _ prevents it from matching any single character
-- including the wildcard makes this much more portable between DBs
SET @sproc_name_pattern = N'sproc[_]%';

-- using QUOTENAME will properly escape any object names with spaces
-- or other funky characters
SELECT @sql = @sql
              + N'GRANT EXECUTE ON '
              + QUOTENAME(OBJECT_SCHEMA_NAME([object_id])) + '.'
              + QUOTENAME([name])
              + N' TO '
              + QUOTENAME(@user_name)
              + N';'
              + @newline + @newline
  FROM sys.procedures
 WHERE [name] LIKE @sproc_name_pattern;

-- this is my version of debug code, I usually run it once with the PRINT intact
-- before I actually use sp_executesql
--PRINT @sql;

EXEC sp_executesql @sql;

Retrieve the top X random rows from a query

Let’s suppose for a minute that you want to retrieve the top x number of rows from any given query. Keep in mind that x is a value that your users can supply.

Sure, you could do string concatenation and EXEC sp_executesql ‘SELECT TOP ‘ + @count + ‘ x FROM y;’. But, then again, a vicious hacker could get all cute and decide to change your form variable to ‘; DECLARE @bad_var AS NVARCHAR(MAX); SET @bad_var = ””; SELECT @bad_var = @bad_var”DROP TABLE ” + name + ”;” FROM sys.tables; SET @bad_var = @bad_var + ”–”; EXEC sp_executesql @bad_var;’ Or something along those lines but probably infinitely more cunning.

Well, you could also accomplish this a different way using a subquery:

DECLARE @count AS int;
SELECT @count = 2;

SELECT ContactId,
       FirstName,
       LastName,
       EmployeeId,
       AddressId,
       AddressLine1,
       AddressLine2,
       City,
       PostalCode
FROM (
  SELECT ROW_NUMBER() OVER (ORDER BY NEWID()) AS rownum,
         c.ContactId,
         c.FirstName,
         c.LastName,
         e.EmployeeId,
         a.AddressId,
         a.AddressLine1,
         a.AddressLine2,
         a.City,
         a.PostalCode
    FROM HumanResources.Employee AS e
          INNER JOIN Person.Contact AS c
            ON e.ContactId = c.ContactId
          INNER JOIN HumanResources.EmployeeAddress AS ea
            ON e.EmployeeId = ea.EmployeeId
          INNER JOIN Person.Address AS a
            ON ea.AddressId = a.AddressId
) AS x
WHERE rownum <= @count;

It’s not the prettiest thing on earth, but it works. Thanks to Rick Kierner over at rickdoes.net for asking me to help out with a similar query late last week.

Links for the Week of 2009-01-23

SQL Server

Top 10 Interview Questions to Ask Senior DBAs Brent Ozar poses some great questions to ask DBAs (and that you might be asked as a DBA) during an interview. These are the good kind of interview questions, not the “how would you do XYZ” kind of questions. (My default response to the XYZ questions is often “Check Books Online”.)

Breaking a String into “Words” the CLR way Jonathan Kehayias looks into breaking a string up into discrete words using the CLR instead of pure T-SQL. The CLR is a great tool that many DBAs overlook, hopefully his post will give you more ideas of how you can use the CLR on a daily basis.

Temporal Data Techniques in SQL Joe Celko writes about using temporal techniques. There are some great details in this article about building calendar tables and working with temporal data. Definitely well worth the read.

Development

Database Portability – Asset or Liability? Aaron Alton put together a great discussion about the myth of database portability and begs us to ask the all important question “How important is it, really?”

General

The Simpsons Archive: Homer’s Mmmm… Lines I’m a big fan of The Simpsons. I’m an even bigger fan of Homer’s stupid lines when he says “mmmm… [insert something here]“. Now there’s a list.

Top 10 Fonts of 2008 In addition to The Simpsons, I’m a huge fan of typography.

BLDGBLOG: Sky TV Geoff Manaugh looks at unintentional and (possibly) intentional light pollution and how it changes the urban space and how that space might be intentionally subverted for different uses in the future. Interesting ideas abound over and BLDGBLOG.

Flexible Database-Level Roles

This is following up from my previous post on Database Roles. In my previous post, I talked about fixed database roles. If you’ve forgotten, go back and re-read the article. I’ll still be here when you get back.

So, what exactly are flexible database roles? They are roles that you create in the database. It’s as simple as that.

When you start with a new database, there are no flexible roles, just the fixed roles. The upside of this is that you are free to create all of the roles that you need and grant all of the permissions that you desire to these roles.

So, where to begin? Let’s start with an empty database:

  -- Create our sample database and switch to it
  CREATE DATABASE TestFixedRoles;
  GO
  USE TestFixedRoles;
  GO

  -- Create a new user and grant them db_securityadmin
  CREATE USER user_security WITHOUT LOGIN;
  CREATE USER user_test WITHOUT LOGIN;
  GO

  EXEC sp_addrolemember N'db_securityadmin', N'user_security';
  GO

So, now we’ve created a new database, a new user, and granted db_securityadmin to our new user. You might remember from the previous article that db_securityadmin can’t grant access to fixed database roles. However, db_securityadmin has no such limitation when dealing with flexible database roles.

Now, let’s set up a test user for the purposes of this introduction to flexible database roles.

  CREATE USER test_user WITHOUT LOGIN;
  GO

This user is now a member of the PUBLIC role. What kind of access does public have?

  CREATE SCHEMA test;
  GO

  CREATE TABLE test.t1 (number INT);
  GO

  DECLARE @i INT;
  SET @i = 0;

  WHILE @i < 1000
  BEGIN
    INSERT INTO test.t1 VALUES (@i);

    SET @i = @i + 1;
  END

  EXECUTE AS USER='test_user';
  GO

  -- This will fail because we have no access to the test schema
  SELECT * FROM t1;
  GO

  REVERT
  GO

None. PUBLIC has no access, thus test_user has no access. Let’s get test_user set up with some access.

  CREATE ROLE test_role;
  GO

  -- The scope qualifier '::' is required.
  GRANT SELECT ON SCHEMA :: test TO test_role;
  GO

  EXEC sp_addrolemember N'test_role', N'test_user';
  GO

  EXECUTE AS USER='test_user';
  GO

  -- Success!
  SELECT * FROM t1;
  GO

  REVERT
  GO

There you have it! We can select from the test schema. How does this help, though, in the real world?

For starters, by creating roles and adding users to roles you can streamline managing security through the use of roles rather than having to monitor the permissions assigned to every user, login, Windows user, and Windows group.

Second, by combining roles for managing security with schemas and stored procedures, it’s possible to carefully control and define granular access to stored procedures and data.

Let’s take a look at AdventureWorks and create an example of how we might want to accomplish this.

This procedure just retrieves customers who placed an order between two dates. Pretty simple.

  CREATE PROCEDURE Sales.GetCustomersWithOrdersBetweenDates (
    @StartDate DATETIME,
    @EndDate DATETIME
  )
  WITH EXECUTE AS OWNER AS

  SELECT pc.LastName,
         pc.FirstName,
         pc.EmailAddress
    FROM Sales.SalesOrderHeader AS ssoh
         INNER JOIN Sales.SalesOrderDetail AS ssod
            ON ssoh.SalesOrderID = ssod.SalesOrderID
         INNER JOIN Sales.Customer AS sc
            ON ssoh.CustomerID = sc.CustomerID
         INNER JOIN Sales.Individual AS si
            ON sc.CustomerID = si.CustomerID
         INNER JOIN Person.Contact AS pc
            ON si.ContactID = pc.ContactID
   WHERE ssoh.OrderDate BETWEEN @StartDate AND @EndDate
   GROUP BY pc.LastName,
            pc.FirstName,
            pc.EmailAddress;
  GO

Next we’ll want to set up some roles:

  -- this is our internal sales personnel
  CREATE ROLE internal_sales_team;
  GO

  -- these are sales people in the field
  CREATE ROLE field_sales_team;
  GO

Now we’ll create users for those roles:

  CREATE USER Iris WITHOUT LOGIN;
  GO

  CREATE USER Frank WITHOUT LOGIN;
  GO

  EXEC sp_addrolemember N'internal_sales_team', N'Iris';
  EXEC sp_addrolemember N'field_sales_team', N'Frank';
  GO

  GRANT EXECUTE ON SCHEMA :: Sales TO internal_sales_team;
  GO

Now let’s test this to see how it works:

  EXECUTE AS USER = N'Iris';
  GO
  EXEC Sales.GetCustomersWithOrdersBetweenDates '20040101', '20040601';
  GO
  REVERT
  GO

  EXECUTE AS USER = N'Frank';
  GO
  EXEC Sales.GetCustomersWithOrdersBetweenDates '20040101', '20040601';
  GO
  REVERT
  GO

Iris can successfully execute the stored procedure, despite her user not having access to the Sales schema because she is a member of the internal_sales_team role which does have execute permissions on the Sales schema. However, Frank cannot execute the stored procedure since the field_sales_team role does not have access to the Sales schema. If you try to run the SQL from this stored procedure as either Frank or Iris the SQL will fail since neither user has select permissions on the Sales or Person schemas.

As I mentioned earlier, through a careful combination of users, logins, roles, Windows users, and Windows groups you can assemble a very secure, robust security infrastructure in SQL Server that can handle a variety of tasks while make your administrative life a lot easier.

Wordle

A couple of people have been playing around wordle recently. I messed around with it and generated this:

Looking carefully, you can see that I like the word ‘probably’ more than the word ‘car’.

Links for the Week of 2009-01-16

SQL Server

Re-associate SQL Users with Logins Scott Koon put up this script a while ago (back in ’07), but I had cause to use it this week. When you restore a database from a backup taken on a different machine you’ll need to re-associate users with their logins. Easy. Done.

Scripting Server Permissions And Role Assignments Kendal Van Dyke shows how it’s possible to script out all permissions and roles assigned to the users of a specific database. Good things in case you’re regularly in the habit of nuking your database and rebuilding it.

Development

Forcing a UI redraw from JavaScript Thomas Fuchs provides a script to force a page redraw (different from a refresh) using JavaScript. Given the different rendering engines in different browsers (I’m looking at you Safari), problems can rise up when you’re dynamically building page content. The only way to fix it is to force a repaint of the page. This is a good one to keep in your library, if you do web development.

Starting out with Objective-C Todd Ritter put together a list of great resources for people interested in learning Objective-C. It’s an expressive language and is on my list of languages to learn and play around with. Hopefully ’09 will be the year for it.

General

Amazon Web Services Explained Brent Ozar manages to explain Amazon Web Services in a way that makes sense to me. After reading this post I finally understand why I would use AWS apart from storing massive quantities of data that I never need to view again.

iBusiness Intelligence – Business Intelligence 2.0 with an i for Design Rob Paller talks about the Mint.com iPhone application and how their UI ideas should be applied to other forms of BI.

Apocalyptic Machine Scultpures are Wonderful in a Morbid, Sinister Sort of Way Like the people at gizmodo said, “Kris Kuksi probably had a disturbing childhood (or dropped acid in graveyards). But it doesn’t make his pieces any less awesome.” I am fascinated by art that’s built up from a combination of different imagery, there’s something compelling about Kris’s work. Enjoy.

How I Get By Without sysadmin

The people who follow me on twitter have probably noticed that when I’m not tweeting absolute nonsense, I’m tweeting about performance tuning SQL. What most people don’t know is that I don’t have sysadmin access on our servers, nor do I have any of the other permissions that would let me collect trace information or access the dynamic management views. I’ll be honest, at first I thought this would be an insurmountable task – how could I possibly tune queries without access to the tools that I have typically used to tune queries? Eventually, I realized that I always had the knowledge that the tools provided, so I got to work putting this knowledge to work for me.

Write Queries Organically

What the hell? Queries aren’t fruit salad! You’re absolutely right, but it’s still possible to do a bit of genetic programming when working on a query. Here’s how this process works for me:

  1. A query request comes in with a query attached from the previous system.
  2. I take a look at the old report and read the SQL so I can understand what’s going on.
  3. I slowly build the core query in SSMS before I create any dynamic logic.
  4. I add of the dynamic logic, test it a few times, and then throw it over the wall for the development team to play with.

Building the core query is where genetic programming comes into play. As I write a query, I’ll start with a core table and slowly add in additional tables and predicates one at a time until I’ve built my query. The whole time I’m doing this, I keep telling SSMS to return the actual execution plan. Normally, if everything runs quickly, I give the execution plan a quick once over with each step to look for glaring trouble spots that need attention before I move on. As I move through the query, I don’t add any aggregation, I just keep adding tables, predicates, and output columns. This lets see where I need to group, verify the data I’m getting, and actually see the raw data before I start aggregating values. This last part is extremely helpful when you’re dealing with CTEs and aggregates.

How does this help me tune the query? Whenever there is a slow down because of a bad join or missing index it’s immediately visible and I can take action. Remember how I said I always return the actual execution plan? There’s a reason for that. I can open up the execution plan and quickly determine what is causing the slow down. If a query goes from taking about a second to 30 seconds, I know that I’ve either done something incredibly wrong or else I’ve missed something in a join or where clause.

Start Small

Whenever I start writing a query, I start with the table and conditions that will filter down results the fastest. This comes from knowing how SQL Server builds query results – I suggest taking a look at Itzik Ben-Gan’s fantastic book Inside Microsoft SQL Server 2005: T-SQL Querying for an introduction to this topic. Basically, I find the conditions that will be the most restrictive and build my query out from there. I work with a range of data that covers 7 years and the detail tables contain 40,000,000+ rows. This isn’t a lot in the grand scheme of things, but when seconds matter, it’s important to have the smallest possible number of records to join on before you get to the monster table.

Cheat and Use Hints

When all else fails, or when the execution plan is just plain bad, I look at the indexes being used and I compare them to the possible indexes on the table. There are covering indexes in place for most common queries on any single table. When it makes sense, I force the engine to use specific indexes through judicious application of index hints. What I don’t do is feed my queries into the Database Tuning Advisor and apply the indexes it suggests. I do use the DTA tools to look for indexes that I might have missed, but I create indexes myself and occasionally, carefully, combine its suggestions with my own to create better covering indexes.

When working almost exclusively with dynamic SQL, it’s important to provide index hints since the number of possible execution plans is staggering and SQL Server will not be able to hold them all in memory. It’s better (and cheaper) to front load the brainpower during query development than to find out that you have poorly performing queries and have to rewrite them on the fly while customers are calling up and complaining about poor performance.

Avoid Parameter Sniffing

Parameter sniffing is when SQL Server caches the values of variables passed into the query for later re-use. The problem is that these might be infrequently used values or the values might always be different. When you have multiple clients accessing the database and @ClientId is a parameter for every query, you don’t want that value to be cached. Instead, I immediately copy stored procedure parameters to a second set of internal parameters. Unfortunately, I lose the advantage of caching if there are common queries that will always use the same or similar parameters, so I’m careful about how I do this. When it makes sense, I don’t copy parameter values.

Careful Conditions

One of the easiest places to lose performance is by writing poorly performing where clauses. Essentially, make sure the columns in your where conditions are always using an indexed column and that you are using the columns in a way that will make use of the index – WHERE DATEPART(yyyy, my_col) > 2006 isn’t going to cut it. Denis Gobo has a great write up on this topic over at Less Than Dot.

Conclusion

There are a lot of great tools out there, both within SQL Server and from third party vendors, that make this process easier. It’s also important to realize that these are skills that we all have and can put to use before resorting to tools to get the last 10 or 20% of performance tuning.

Related Topics

Here are two related articles about security within SQL Server.

SQL Server Security – Database Roles
Flexible Database-Level Roles

Links for the week of 2009-01-09

SQL Server

The first pillar – A Coherent Design Louis Davidson has started to go in depth into his pillars of database design. Louis clearly explains what a coherent database design is and why it’s important.

sp_indexinfo – enhanced index information procedure Tibor Karaszi has put together a custom stored procedure to provide additional information about tables and indexes, including missing indexes on a table.

Start / Stop SQL Server, SSIS, SSAS, SSRS and SQL Server Agent at once with a batch file Jorg Klein has posted a batch file to stop/start SQL Server and associated services on command. When I develop at home, I using a black MacBook and run Windows in a VM. While memory isn’t at a premium, CPU cycles sometimes are. I’m sure this will come in very handy for speeding up the boot time of my virtual machine.

Seek or Scan? Gail provides some insight as to why you might be seeing an index scan instead of an index seek and why you should be using covering indexes.

Free eBook from RedGate : Dissecting SQL Server Execution Plans (Grant Fritchey) Jonathan Kehayais provides a link to an amazing book by Grant Fritchey. Grant dives into how to read execution plans. I’ve read one chapter and I’m looking forward to reading the rest.

Development

NH Prof New Feature: The Query Cache NHibernate fans, it is time to rejoice! NHProf has a new feature (still in dev) that lets you view the queries being sent to the database.

10 jQuery scripts to improve your site’s interface Designer Daily has published links to 10 sweet jQuery plugins for improving your UI. I’m a huge fan of jQuery – I get access to a massive library of cross-browser plugins and functionality that I would otherwise have to develop myself and I get access to an amazing library of base functions that let me rapidly build a functioning website.

General

Networking – Part 2 Andy Warren talks a bit about how he feels regarding networking. When I first started thinking about networking more, I felt a bit dirty – like I would have to go push myself like some kind of used car salesman (no offense to any used car salespeople reading this blog, I’m sure you’re great people, you just have a reputation). After talking to co-workers and friends, I realized it’s not about selling your brand but expanding your horizons. Glad to see that someone else agrees with me.

Setting Up a Killer Demo Buck Woody gives 12 great techniques to make your presentations and demos even better. I have three presentations coming up in the next three months, so I’ll be referring to this post a great deal as I prepare.

This site is protected with Urban Giraffe's plugin 'HTML Purified' and Edward Z. Yang's Powered by HTML Purifier. 214 items have been purified.