Myths, MythBusting and Facts

by daniel on October 30, 2008

Watching the TV series Mythbusting, I came to a realization that many times I was guided by myths generated without any basic facts or actual validation of the myth. This can happen due to repeating outdated information, using wrong methodologies, or simple ignorance.

Since these myths actually hinder the development of the system, one of the tasks of a system engineer is to ascertain the validity of the myth and remove unnecessary barriers.

The TV series shows a methodology for tackling myths:

  1. analyze the myth for the basic hypothesis
  2. propose a test for validating / disproving the hypothesis
  3. perform the test and analyze

When I’m annoyed by a work myth, I try to the same with several additions:

  1. look for information in the internet
  2. look for a requirement related to the myth

Here are a few samples of broken myths:

  • Windows cannot handle more than 10,000 files in a directory: This myth became apparent when the machine used for writing files began to have performance degradations after 10,000 files. Further study showed the actual problem was the file naming convention, where the suffix of the filename changes little. The limiting factor was the 8.3 filename creation as detailed here: disabling 8dot3 filename creation. Of course, changing the behavior allowed for a huge number of files per directory.

  • We use the DB machine to the maximum and the IO is the limiting factor: this myth became apparent when we started to have performance problems in our SQL 2005 database. Study of the SQL server performance showed that in a 4 core machine, only one core was used for processing the information and the rest cores where idle. Further analysis found out that the limiting factor was the number of data files and connections to the database. Increasing the number of files and modifying the clients to use 4 connections helped to fully utilize the available cores. The limiting factor still remains the CPU capacity and not the IO.

The net result is performance increase by hundredth of percents, giving enough headroom for existing projects.

This shows that even a TV program can provide insights of methodology for analysis of problems and demonstrating the value of research and testing.

No tags for this post.

Comments on this entry are closed.

Previous post:

Next post: