h4.[Energy Efficient Datacenters: The Role of Modularity in Datacenter Design !Main^download.gif!|http://mapping.sun.com/profile/offer.jsp?id=8]
*by Dean Nelson, Michael Ryan, Serena DeVito, Ramesh KV, Petr Vlasaty, Brett Rucker, and Brian Day*
{excerpt}June, 2008{excerpt}
Virtually every Information Technology (IT) organization and the clients that they serve have dramatically different requirements that impact their datacenter designs. Sun is no exception to this rule. As an engineering company, Sun has cross-functional organizations that manage the company's corporate infrastructure portfolio including engineering, services, sales, operations, and IT.
On the surface, the datacenters supporting these different organizations look as different as night and day - one looks like a computer hardware laboratory and another looks like a lights-out server farm. One has employees entering and leaving constantly, and another is accessed remotely and could be anywhere. One may be housed in a building, and another may be housed within an enhanced shipping container. Beneath the surface, however, our datacenters have similar underlying infrastructure including physical design, power, cooling, and connectivity.
* The Role of Modularity in Datacenter Design
** Choosing Modularity
** Defining Modular, Energy-Efficient Building Blocks
** Buildings Versus Containers
** Cost Savings
** About This Article
* The Range of Datacenter Requirements
** Power and Cooling Requirements
*** Using Racks, Not Square Feet, as the Key Metric
*** Temporal Power and Cooling Requirements
*** Equipment-Dictated Power and Cooling Requirements
** Connectivity Requirements
** Equipment Access Requirements
** Choosing Between Buildings and Containers
** Living Within a Space, Power, and Cooling Envelope
*** Space
*** Power
*** Cooling
** Calculating Sun's Santa Clara Datacenters
*** Efficiency in Sun's Santa Clara Software Datacenter
* Sun's Pod-Based Design
** Modular Components
** Pod Examples
*** Hot-Aisle Containment and In-Row Cooling
*** Overhead Cooling for High Spot Loads
*** A Self-Contained Pod - the Sun Modular Datacenter
* Modular Design Elements
** Physical Design Issues
*** Sun Modular Datacenter Requirements
*** Structural Requirements
*** Raised Floor or Slab
*** Racks
*** Future Proofing
** Modular Power Distribution
*** The Problem with PDUs
*** The Benefits of Modular Busway
** Modular Spot Cooling
*** Self-Contained, Closely Coupled Cooling
*** In-Row Cooling with Hot-Aisle Containment
*** Overhead Spot Cooling
*** The Future of Datacenter Cooling
** Modular Cabling Design
*** Cabling Best Practices
* The Modular Pod Design at Work
** Santa Clara Software Organization Datacenter
** Santa Clara Services Organization Datacenter
** Sun Solution Center
** Guillemont Park Campus, UK
** Prague, Czech Republic
** Bangalore, India
** Louisville, Colorado: Sun Modular Datacenter
* Summary
** Looking Toward the Future
** About the Authors
*** Dean Nelson
*** Michael Ryan
*** Serena Devito
*** Ramesh KV
*** Petr Vlasaty
*** Brett Rucker
*** Brian Day
** Acknowledgments
** References
** Ordering Sun Documents
** Accessing Sun Documentation Online
{panel:title=About the Authors|borderStyle=solid|titleBGColor=#F8D583|bgColor=white}
h6.Dean Nelson
Dean is the Senior Director of Global Lab & Datacenter Design Services (GDS) in the Workplace Resources business unit of Sun Microsystems. The GDS organization bridges the gap between Facilities and IT/Engineering and is currently managing more than USD $250 million in datacenter design and construction activity. The GDS work supports the Act portion of Sun's Eco Strategy, and was showcased at Sun's Eco Launch in August 2007.
h6.Michael Ryan
Mike Ryan is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization. Mike graduated from San Jose State University with a degree in Mechanical Engineering. He is a licensed professional mechanical engineer and has been involved in the design, construction, and operation of mission-critical facilities for the past 18 years. His experience spans numerous industries such as gas turbine co-generation, semi-conductor manufacturing, and mission-critical datacenters. For the last seven years he has focused on the design and operation of mechanical and electrical infrastructure systems supporting high-availability datacenters. mike joined Sun as a Staff Engineer for the GDS organization in March of 2006. Mike is the primary author of the GDS physical standards and was the lead design engineer for the Santa Clara, California datacenter project, one of the largest and most complex in Sun's history. Mike's design work supports the Act portion of Sun's Eco strategy, and was showcased at Sun's Eco Launch in August 2007.
h6.Serena DeVito
Serena DeVito is a Datacenter Design Engineer in Sun's Global Lab & Datacenter Design Services organization. Serena joined Sun over ten years ago and has spent the bulk of her career as a system administrator within the IT organization where she designed, deployed, and supported critical infrastructure for the company. She then moved to the Software organization and managed the beta testing project for the Sun Cluster product group. Serena acted in a customer capacity running early releases of Sun Cluster software on production home directory clusters with over one-hundred active users as well as clustered web services. Serena identified, documents, and filed bugs while supporting one of Sun's largest and most complex engineering lab environments.
h6.Ramesh KV
Ramesh has been in the technology industry for over thirteen years and has held a variety of technical & managerial roles since joining Sun in 2001. In 2007 Ramesh became a Regional Work Place Manager (RWPM) covering South Asia and India in addition to his GDS responsibilities. In this role he has been responsible for all WR activities across South Asia and India, including portfolio management, real estate services, and facilities management. Ramesh has been a critical member of the GDS technology team since 2004 and has designed multiple labs and datacenters across the Asia Pacific. Ramesh is the recipient of the PC Quest IT Implementation project of the year award in 2007 and has received twelve internal awards globally including Sun's Leadership Award in 2005.
h6.Petr Vlasaty
Petr is a Datacenter Architect and the GDS Technical Infrastructure Lead for the EMEA region. Petr has been working in the IT industry for more than 12 years. In November 2004, Petr joined the Sun Software organization as R&D Datacenter Manager and Team Lead for the Prague Datacenter Management Team. Petr designed and implemented two internal datacenter projects in Prague in partnership with the GDS team. He applied the GDS processes and standards for the new Prague datacenter expansion that was completed in June 2006. Due to Petr's outstanding performance in Prague, he was assigned as the technical lead for the Louisville, Colorado consolidation project in August of 2006. He spent four months in Colorado developing the very detailed and complex plan. Petr joined the GDS organization full time in January of 2007. He is responsible for the Europe, Middle-East and Africa (EMEA) region and continues to act as technical lead for the Colorado datacenter consolidation project, the largest and most complex in Sun's history.
h6.Brett Rucker
Brett Rucker is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization where he focuses on physical environment issues. Brett has been in the 52 Summary Sun Microsystems, Inc. Corporate Engineering, Construction and Management business for 25 Years. A Graduate of the University of Colorado, Boulder, and a licensed Professional Engineer, Brett began his career with Ball Aerospace Systems Division, a rich opportunity that gave him complete design and management responsibility for all construction projects. He supervised products including building additions, clean rooms, X-Ray rooms, datacenters, laboratories, anechoic chambers, secure areas, gallium arsenide facilities, high elevation satellite dish structural mounts, RF shielded rooms, and office, conference, and customer show areas.
h6. Brian Day
Brian Day is a Senior Program Manager in Sun's Global Lab & Datacenter Design Services organization. Brian has been working in the technology industry for over 10 years, of which more than five have been with Sun. He spent over two years in Sun's Systems group, developing automation programs for server, storage, and software QA functions. Brian left Sun in 2000 to pursue his MBA full time. He joined Adobe Systems in 2001 as a Program Manager, leading the development of several Digital Imaging plug-ins for Adobe Acrobat. Brian returned to Sun in early 2005 to program manage the GDS initiative. As Senior Program Manager & Chief of Staff for GDS, Brian is responsible for first engagement with new customers, ensuring world-wide GDS projects are resourced, deliverable expectations are set, and commitments honored.
{panel}
{panel:title=Acknowledgments|borderStyle=solid|titleBGColor=#F8D583|bgColor=white}
The authors would like to thank Steve Gaede, an independent technical writer and engineer, for his probing questions, deep insights, and ability to create a coherent story. Steve is a frequent contributor to Sun Microsystems technical documents, including many Sun BluePrints articles. He is a member of ACM and USENIX, and is active in the Boulder, Colorado professional community, having been a coordinator of the Front Page UNIX Users Group since 1984.
Thanks to the Sun MD team members who spent time contributing to this paper, including Maurice Cloutier, Liz From, Brian Kowalski, and Bob Schilmoeller.
The authors would also like to thank the following Sun contributors for the helpful and thoughtful comments they provided while reviewing this paper: Phil Morris, James Monahan, Rob Snevely, Steve Evans, Tracy Shintaku, and Peter Spence.
{panel}
{vote:Rate this blueprint}
Great
Good
Fair
Poor
{vote}
*by Dean Nelson, Michael Ryan, Serena DeVito, Ramesh KV, Petr Vlasaty, Brett Rucker, and Brian Day*
{excerpt}June, 2008{excerpt}
Virtually every Information Technology (IT) organization and the clients that they serve have dramatically different requirements that impact their datacenter designs. Sun is no exception to this rule. As an engineering company, Sun has cross-functional organizations that manage the company's corporate infrastructure portfolio including engineering, services, sales, operations, and IT.
On the surface, the datacenters supporting these different organizations look as different as night and day - one looks like a computer hardware laboratory and another looks like a lights-out server farm. One has employees entering and leaving constantly, and another is accessed remotely and could be anywhere. One may be housed in a building, and another may be housed within an enhanced shipping container. Beneath the surface, however, our datacenters have similar underlying infrastructure including physical design, power, cooling, and connectivity.
* The Role of Modularity in Datacenter Design
** Choosing Modularity
** Defining Modular, Energy-Efficient Building Blocks
** Buildings Versus Containers
** Cost Savings
** About This Article
* The Range of Datacenter Requirements
** Power and Cooling Requirements
*** Using Racks, Not Square Feet, as the Key Metric
*** Temporal Power and Cooling Requirements
*** Equipment-Dictated Power and Cooling Requirements
** Connectivity Requirements
** Equipment Access Requirements
** Choosing Between Buildings and Containers
** Living Within a Space, Power, and Cooling Envelope
*** Space
*** Power
*** Cooling
** Calculating Sun's Santa Clara Datacenters
*** Efficiency in Sun's Santa Clara Software Datacenter
* Sun's Pod-Based Design
** Modular Components
** Pod Examples
*** Hot-Aisle Containment and In-Row Cooling
*** Overhead Cooling for High Spot Loads
*** A Self-Contained Pod - the Sun Modular Datacenter
* Modular Design Elements
** Physical Design Issues
*** Sun Modular Datacenter Requirements
*** Structural Requirements
*** Raised Floor or Slab
*** Racks
*** Future Proofing
** Modular Power Distribution
*** The Problem with PDUs
*** The Benefits of Modular Busway
** Modular Spot Cooling
*** Self-Contained, Closely Coupled Cooling
*** In-Row Cooling with Hot-Aisle Containment
*** Overhead Spot Cooling
*** The Future of Datacenter Cooling
** Modular Cabling Design
*** Cabling Best Practices
* The Modular Pod Design at Work
** Santa Clara Software Organization Datacenter
** Santa Clara Services Organization Datacenter
** Sun Solution Center
** Guillemont Park Campus, UK
** Prague, Czech Republic
** Bangalore, India
** Louisville, Colorado: Sun Modular Datacenter
* Summary
** Looking Toward the Future
** About the Authors
*** Dean Nelson
*** Michael Ryan
*** Serena Devito
*** Ramesh KV
*** Petr Vlasaty
*** Brett Rucker
*** Brian Day
** Acknowledgments
** References
** Ordering Sun Documents
** Accessing Sun Documentation Online
{panel:title=About the Authors|borderStyle=solid|titleBGColor=#F8D583|bgColor=white}
h6.Dean Nelson
Dean is the Senior Director of Global Lab & Datacenter Design Services (GDS) in the Workplace Resources business unit of Sun Microsystems. The GDS organization bridges the gap between Facilities and IT/Engineering and is currently managing more than USD $250 million in datacenter design and construction activity. The GDS work supports the Act portion of Sun's Eco Strategy, and was showcased at Sun's Eco Launch in August 2007.
h6.Michael Ryan
Mike Ryan is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization. Mike graduated from San Jose State University with a degree in Mechanical Engineering. He is a licensed professional mechanical engineer and has been involved in the design, construction, and operation of mission-critical facilities for the past 18 years. His experience spans numerous industries such as gas turbine co-generation, semi-conductor manufacturing, and mission-critical datacenters. For the last seven years he has focused on the design and operation of mechanical and electrical infrastructure systems supporting high-availability datacenters. mike joined Sun as a Staff Engineer for the GDS organization in March of 2006. Mike is the primary author of the GDS physical standards and was the lead design engineer for the Santa Clara, California datacenter project, one of the largest and most complex in Sun's history. Mike's design work supports the Act portion of Sun's Eco strategy, and was showcased at Sun's Eco Launch in August 2007.
h6.Serena DeVito
Serena DeVito is a Datacenter Design Engineer in Sun's Global Lab & Datacenter Design Services organization. Serena joined Sun over ten years ago and has spent the bulk of her career as a system administrator within the IT organization where she designed, deployed, and supported critical infrastructure for the company. She then moved to the Software organization and managed the beta testing project for the Sun Cluster product group. Serena acted in a customer capacity running early releases of Sun Cluster software on production home directory clusters with over one-hundred active users as well as clustered web services. Serena identified, documents, and filed bugs while supporting one of Sun's largest and most complex engineering lab environments.
h6.Ramesh KV
Ramesh has been in the technology industry for over thirteen years and has held a variety of technical & managerial roles since joining Sun in 2001. In 2007 Ramesh became a Regional Work Place Manager (RWPM) covering South Asia and India in addition to his GDS responsibilities. In this role he has been responsible for all WR activities across South Asia and India, including portfolio management, real estate services, and facilities management. Ramesh has been a critical member of the GDS technology team since 2004 and has designed multiple labs and datacenters across the Asia Pacific. Ramesh is the recipient of the PC Quest IT Implementation project of the year award in 2007 and has received twelve internal awards globally including Sun's Leadership Award in 2005.
h6.Petr Vlasaty
Petr is a Datacenter Architect and the GDS Technical Infrastructure Lead for the EMEA region. Petr has been working in the IT industry for more than 12 years. In November 2004, Petr joined the Sun Software organization as R&D Datacenter Manager and Team Lead for the Prague Datacenter Management Team. Petr designed and implemented two internal datacenter projects in Prague in partnership with the GDS team. He applied the GDS processes and standards for the new Prague datacenter expansion that was completed in June 2006. Due to Petr's outstanding performance in Prague, he was assigned as the technical lead for the Louisville, Colorado consolidation project in August of 2006. He spent four months in Colorado developing the very detailed and complex plan. Petr joined the GDS organization full time in January of 2007. He is responsible for the Europe, Middle-East and Africa (EMEA) region and continues to act as technical lead for the Colorado datacenter consolidation project, the largest and most complex in Sun's history.
h6.Brett Rucker
Brett Rucker is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization where he focuses on physical environment issues. Brett has been in the 52 Summary Sun Microsystems, Inc. Corporate Engineering, Construction and Management business for 25 Years. A Graduate of the University of Colorado, Boulder, and a licensed Professional Engineer, Brett began his career with Ball Aerospace Systems Division, a rich opportunity that gave him complete design and management responsibility for all construction projects. He supervised products including building additions, clean rooms, X-Ray rooms, datacenters, laboratories, anechoic chambers, secure areas, gallium arsenide facilities, high elevation satellite dish structural mounts, RF shielded rooms, and office, conference, and customer show areas.
h6. Brian Day
Brian Day is a Senior Program Manager in Sun's Global Lab & Datacenter Design Services organization. Brian has been working in the technology industry for over 10 years, of which more than five have been with Sun. He spent over two years in Sun's Systems group, developing automation programs for server, storage, and software QA functions. Brian left Sun in 2000 to pursue his MBA full time. He joined Adobe Systems in 2001 as a Program Manager, leading the development of several Digital Imaging plug-ins for Adobe Acrobat. Brian returned to Sun in early 2005 to program manage the GDS initiative. As Senior Program Manager & Chief of Staff for GDS, Brian is responsible for first engagement with new customers, ensuring world-wide GDS projects are resourced, deliverable expectations are set, and commitments honored.
{panel}
{panel:title=Acknowledgments|borderStyle=solid|titleBGColor=#F8D583|bgColor=white}
The authors would like to thank Steve Gaede, an independent technical writer and engineer, for his probing questions, deep insights, and ability to create a coherent story. Steve is a frequent contributor to Sun Microsystems technical documents, including many Sun BluePrints articles. He is a member of ACM and USENIX, and is active in the Boulder, Colorado professional community, having been a coordinator of the Front Page UNIX Users Group since 1984.
Thanks to the Sun MD team members who spent time contributing to this paper, including Maurice Cloutier, Liz From, Brian Kowalski, and Bob Schilmoeller.
The authors would also like to thank the following Sun contributors for the helpful and thoughtful comments they provided while reviewing this paper: Phil Morris, James Monahan, Rob Snevely, Steve Evans, Tracy Shintaku, and Peter Spence.
{panel}
{vote:Rate this blueprint}
Great
Good
Fair
Poor
{vote}