October 2009
Mon Tue Wed Thu Fri Sat Sun
« Sep   Nov »
 1234
567891011
12131415161718
19202122232425
262728293031  

Month October 2009

Links for the Week of 2009.10.30

These are a day late. Sorry about that. It’s been one helluva week.

Also, November 1st marks the beginning of National Novel Writing Month. So, if you’ve had a story inside you waiting to get out, wait no further!

SQL Server

Development

Stuff & Things

Dogfood

I’m a bit late on this one, but I’m very pleased to announce that I will be speaking at the local Dogfood Conference. I’ll be speaking about SQL Server 2008 R2 and will be conducting an Ask the Experts session in the afternoon, but I’m not the only person who will be at the conference. There are going to be a lot of great sessions at the conference covering the range of topics from Windows 7 to SharePoint to F#.

Check out the Thursday and Friday agendas and sign up.

Card Games

If you’re going to the PASS Summit this year, and I suggest that you do, you should be playing twitter bingo. How do you play? You go out of your wait to meet people at the Summit. If you meet someone who is on your card, you get their ‘code word’ and write it on your bingo card. The rules are explained over at SQL Server Pedia, but, basically, it’s just like playing bingo.

You can print out your cards at http://www.sqlseverpedia.com/bingo/ or pick one up at PASS at the SQLServerPedia/Quest booth.

Links for the Week of 2009.10.23

I came across a lot of great links in the last week. So many that I had to cut about 40% of them to make this list.

As always, you can check out the firehose at my delicious bookmarks page.

SQL Server

Development

Stuff & Things

SQL University – Snow Day

I was supposed to have a second blog post up this week for SQL University, however I haven’t been feeling well and have been focusing my energy on my day job. Unfortunately, I’ve had to drop the ball on a few other things. My second SQL University post for the internals week is what got dropped (as well as a few other things that don’t directly involve anyone else).

However, I did come across this little gem that I’d like to share. Logical Query Processing Poster (PDF alert)

This is the logical query processing diagram that is in Itzik Ben-Gan’s fantastic book Inside Microsoft® SQL Server® 2008: T-SQL Querying. This explains how SQL Server builds a query from the ground up and includes all phases of query processing.

PASS Board of Directors

The PASS Board of Director elections are closed and the results are in.

I am incredibly honored that the PASS community elected me to represent them on the Board of Directors for the next two years.

I’m very excited to have this chance to work with the community to share our thoughts, vision, and ideas about PASS and to grow our organization to new heights. It’s going to be a lot of fun and I hope everyone will share their ideas with me so we can make PASS rock even more than it already does.

An Introduction to SQL Server System Databases

SQL Server 2008 (and 2005) contain five special databases: master, model, tempdb, msdb, and mssqlsystemresource (aka Resource). These databases are used by SQL Server for its own maintenance and management.

master

The master database contains all of the system level information for SQL Server – all of the logins, linked servers, endpoints, and other system-wide configuration settings. The master database is also where SQL Server stores information about the other databases on this instance and the location of their files. If the master database is not present, SQL Server cannot start.

Always take regular backups of the master database.

Since the master database hold all of the information related to logins, endpoints, linked servers, and user databases, it’s important that you take a backup of the master database after configuring any of these server level changes. Otherwise, if your SQL Server suffers a catastrophic failure, those changes will be lost to the sands of time.

Books Online contains a great deal of helpful information about the master database that you can find in the appropriately named master Database article.

Speaking from personal experience, one of the most important points from that article is this: “Do not create user objects in master. Otherwise, master must be backed up more frequently.” I cannot tell you the number of times I have connected to a development server, opened up a file, and run the DDL contents of the file only to realize that I have created new tables on the master database because I did not have a USE AdventureWorks at the top of my DDL script.

model

The model database is used as a template whenever a new user database is created. You can change most database properties, create users, stored procedures, tables, views, etc – whatever you do will be applied to any new databases.

The nice thing is that you can create a guaranteed set of users, stored procedures, and options (including collation options) by making changes to the model database. Once those changes are in place, they are applied to every new database.

Outside of its role as a template, model doesn’t do anything else. You can find more information in the model Database article of Books Online.

msdb

msdb is used by the SQL Server Agent, database mail, Service Broker, and other services. If you aren’t actively working with things like jobs, alerts, log shipping, etc you can pretty safely ignore msdb… sort of.

One important item is that msdb holds backup history. Using the msdb tables (you can start by taking a look at msdb.dbo.backupset), it’s possible to determine when each database and filegroup was last backed up. This is very useful, especially when you’ve just started working at a new company or taken over the maintenance of new servers.

A word of warning: you need to make sure that you are pruning old backup history from msdb. Leaving old backup data can slow down the performance of backup and restore operations. Brent Ozar documented this during his Backup Tricks Week article Checking MSDB Cleanups. In this article, the sp_delete_backuphistory system stored procedure is mentioned. This stored procedure will delete information older than the @oldest_date parameter. It is important that you are incredibly careful when using this store procedure and don’t attempt to delete all of the backup history data in your msdb at once. Attempting to clear out a large number of database backup history records in one fell swoop can have an adverse effect on performance. I suggest removing the data one to two days at a time. It’s tedious, I know, but your users will thank you.

Resource database

The resource database is a hidden system database. This is where system objects are stored. It isn’t possible to see the resource database by normal means. However you can see the data file by navigating to C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Binn. The exact size and modification data of this file will be different from version to version, but the modified date should be the same date that you see when you run SELECT @@version.

It is best to think of the resource database as if it were another system DLL. The resource database is designed to make it easy for quick database upgrades. If new system objects are being put in place, it is only necessary to swap out the resource database MDF file.

Typically, the only way to view the contents of the resource database is using the OBJECT_DEFINITION system function.

SELECT OBJECT_DEFINITION(OBJECT_ID('sys.objects'))

-- on my test system this yields the following:
CREATE VIEW sys.objects AS
    SELECT name, object_id, principal_id, schema_id, parent_object_id,
        type, type_desc, create_date, modify_date,
        is_ms_shipped, is_published, is_schema_published
    FROM sys.objects$

tempdb

We come, at last, to tempdb. Tempdb is the workhorse of the system databases. It is the workspace that SQL Server uses to store the intermediate results of query processing and sorting. You know how you see those spools in your execution plans? When you see one of those, SQL Server is probably spooling the data to a temporary table in the tempdb. Outside of storing temporary results, tempdb is also used during snapshot isolation and for user created temporary tables (this includes table variables).

One thing that is interesting to note about tempdb is that it is re-created every time the SQL Server service is started. Any objects that you have created in tempdb will be gone once the server restarts. If you want specific tables or stored procedures to always be available in tempdb, you will need to add them to the model database or else use a stored procedure to create them in tempdb when the SQL Server is started.

A properly sized and configured tempdb is vital for effective SQL Server performance. By default tempdb starts at 8MB in size and will continue to grow by ten percent until the drive is full. If the tempdb is sized too small, system resources will be used growing the tempdb file. This overhead can be avoided by increasing the default size of tempdb.

SQL Server System Databases
System Databases
Working with tempdb in SQL Server 2005
Microsoft SQL Server 2008 Internals pp 126 – 128

Links for the Week of 2009.10.16

SQL Server

Less Than Dot – Blog – How to Monitor Database Mirroring – Monitoring database mirroring is more than just running scripts to email you ever X minutes. Paul Theriault shows how to monitor mirroring (and other aspects of performance) using alerts.
How to get the whole group of duplicate rows – CTEs are great for more than just replacing views. Mladen Prajdic shows how they can be used to find duplicate data quickly and efficiently.
Bad habits to kick: making assumptions about IDENTITY – A great discussion from Aaron Bertrand about how IDENTITY actually works as well as solutions to make it work the way people want it to work.
Introduction to SQL Server 2008 Extended Events

Stuff & Things

A Softer World: 490 – I don’t know what it is, but I like it.
SCOUTING NY – www.scoutingny.com » New York, You’ve Changed: Taxi Driver – Part III – It's interesting to note how the world changes over time and how tiny pockets of it still stay the same. This is a look at the NYC of Taxi Driver then and now.
2001: A Space Odyssey Program – During the original theatrical premier of 2001: A Space Odyssey a program was handed out. This is a beautiful example of design that conveys the mood of the film and serves to prepare the audience as well as amplify their experience.
Flickr Photo Download: 50 Years of Space Exploration
In pictures: Graffiti artists transform Scottish ghost town Polphaill – The Scotsman – Transforming the rural crapscape into a work of art.
The Pomodoro Technique – an interesting approach to solving problems and managing your time effectively.
The Gentlemen’s Guide to Dressing – Some people are born with an innate sense of style. The rest of us all need to a little bit of help.
@font-face and performance – There are some new techniques coming out for better font rendering on the web. Steve Souders goes over some of the pitfalls of these techniques to users can be aware of the downside while they’re making their choice.

SQL Server Error Logs

What is the Error Log?

SQL Server maintains its own error logs that contain messages describing informational and error events.

These are simple text files stored on disk, but it’s best to view them using the facilities provided by SQL Server to prevent any SQL operations from being blocked by opening one of these files in a text editor. Also, the error log files can become quite large – SQL Server will stream the files into the Log File Viewer whereas notepad will open the file into a single buffer and consume a great deal more memory.

A new error log file will be created when one of two things happens:

  1. The SQL Server service is started
  2. sp_cycle_errorlog is called

Once this happens, any pending writes to the current log file will complete and a new log file will be created. The actual error log files can be found at Program Files\Microsoft SQL Server\MSSQL.n\MSSQL\LOG\ERRORLOG and ERRORLOG.n.

Viewing the Error Log

The Basics

Locating the error log

Locating the error log

Viewing the Error Logs

Viewing the Error Logs

Viewing the error log is a simple operation:

  1. Connect to the SQL Server using SQL Server Management Studio
  2. Expand the Server > Management > SQL Server Logs
  3. Double click on a log file

This will open up the log file. One thing to keep in mind is that by default this log file will only cycle when the SQL Server service has started. If this is a production server and you haven’t made any configuration changes, then this file could be quite large and take a long time to open.

Refining Your View

The Error Log Filter Box

The Error Log Filter Box

Filtered Error Log Results

Filtered Error Log Results

By default, you will end up with a view of the entire contents of one or more log files. This probably isn’t very useful to you unless you want to scan the contents for something very recent. On a busy server, though, there could be hundreds of thousands of lines in a single error log file.

Clicking the filter button opens up the Filter Settings window. This gives you the opportunity to filter by User, Computer, Start and End Date, the message text, and the message source. Given that there are so many different sources for the messages in the error log, the ability to filter the error log is a huge boon. Once you’ve entered your filter criteria, check ‘Apply filter’ and then click OK. Just make sure you check ‘Apply filter’ otherwise SQL Server Management Studio will happily discard all of your filter criteria and not filter any of the data.

Care & Feeding of the Log File

Automated Log File Rotation

To prevent the log files from growing too large it’s necessary to rotate them on a regular basis. Restarting the SQL Server service every week at 3:34 AM on a Sunday isn’t advisable, so there must be some other option. Thankfully, there is. The log file can be automatically rotated using the sp_cycle_errorlog system stored procedure.

Depending on who you talk to, different people will tell you to cycle the error log on a daily, weekly, or monthly schedule. The frequency really depends on your particular system and how much data accumulates in the logs during the course of regular operations. The more writes to the error log, the more often you should cycle the log. To cycle the logs, create a new Agent Job with a single T-SQL task (or combine it with regular daily or weekly maintenance).

Number of Log Files to Maintain

Depending on the amount of storage space you have available and the amount of activity on your SQL Server, you will want to change how often you rotate your log files and also the number of log files that you want to retain. Why would you want to do this? To keep as much historical information available as you possibly can.

Configuring the Error Log

Configuring the Error Log

Changing the Number of Error Log Files

Changing the Number of Error Log Files

More importantly, how do you do this?

  1. Right click on the SQL Server Logs folder in the Object Explorer
  2. Select ‘Configure’
  3. Check the box to ‘Limit the number of error log files before they are recycled’. This seems silly, but by default SQL Server will only keep 6 error log files before trashing them.
  4. Put in a new value and click ‘OK’.

I typically keep 99 error logs before they are recycled. Why? Because I can.

Wrap Up

I hope that this has helped you understand more about the SQL Server Error Log and how you can can automate the successful maintenance of your servers’ log file information.

AppDev Virtual Meeting Today @ 1:00PM Eastern

I just wanted to get out a quick update about the AppDev Live Meeting today.

Josef Richberg will be speaking on the subject of SSIS Load Balancing. You can find more details on the Application Development Virtual Chapter home page. Or, at 1PM Eastern time (UTC -0400) you can head on over to LiveMeeting.

Abstract

How is it possible to aggregate 12,000,000 records of sales data in a short time window? In this presentation Josef Richberg will demonstrate the techniques used to randomize the input data and then use the SSIS threading framework to distribute the workload. These techniques make it possible to process a large quantity of data in a very short time frame.

About Josef Richberg

Josef is a DBA for HarperCollins Publishers working with SQL Server and SSIS. He has over 16 years experience designing, building, and tuning SQL Server. Josef is also the recipient of the ’2009 Exceptional DBA’ award and he actively blogs at http://josef-richberg.squarespace.com.

This site is protected with Urban Giraffe's plugin 'HTML Purified' and Edward Z. Yang's Powered by HTML Purifier. 230 items have been purified.