Energy Efficient Datacenters - The Role of Modularity in Datacenter Design

Energy Efficient Datacenters: The Role of Modularity in Datacenter Design

by Dean Nelson, Michael Ryan, Serena DeVito, Ramesh KV, Petr Vlasaty, Brett Rucker, and Brian Day
June, 2008

Virtually every Information Technology (IT) organization and the clients that they serve have dramatically different requirements that impact their datacenter designs. Sun is no exception to this rule. As an engineering company, Sun has cross-functional organizations that manage the company's corporate infrastructure portfolio including engineering, services, sales, operations, and IT.

On the surface, the datacenters supporting these different organizations look as different as night and day - one looks like a computer hardware laboratory and another looks like a lights-out server farm. One has employees entering and leaving constantly, and another is accessed remotely and could be anywhere. One may be housed in a building, and another may be housed within an enhanced shipping container. Beneath the surface, however, our datacenters have similar underlying infrastructure including physical design, power, cooling, and connectivity.

  • The Role of Modularity in Datacenter Design
    • Choosing Modularity
    • Defining Modular, Energy-Efficient Building Blocks
    • Buildings Versus Containers
    • Cost Savings
    • About This Article
  • The Range of Datacenter Requirements
    • Power and Cooling Requirements
      • Using Racks, Not Square Feet, as the Key Metric
      • Temporal Power and Cooling Requirements
      • Equipment-Dictated Power and Cooling Requirements
    • Connectivity Requirements
    • Equipment Access Requirements
    • Choosing Between Buildings and Containers
    • Living Within a Space, Power, and Cooling Envelope
      • Space
      • Power
      • Cooling
    • Calculating Sun's Santa Clara Datacenters
      • Efficiency in Sun's Santa Clara Software Datacenter
  • Sun's Pod-Based Design
    • Modular Components
    • Pod Examples
      • Hot-Aisle Containment and In-Row Cooling
      • Overhead Cooling for High Spot Loads
      • A Self-Contained Pod - the Sun Modular Datacenter
  • Modular Design Elements
    • Physical Design Issues
      • Sun Modular Datacenter Requirements
      • Structural Requirements
      • Raised Floor or Slab
      • Racks
      • Future Proofing
    • Modular Power Distribution
      • The Problem with PDUs
      • The Benefits of Modular Busway
    • Modular Spot Cooling
      • Self-Contained, Closely Coupled Cooling
      • In-Row Cooling with Hot-Aisle Containment
      • Overhead Spot Cooling
      • The Future of Datacenter Cooling
    • Modular Cabling Design
      • Cabling Best Practices
  • The Modular Pod Design at Work
    • Santa Clara Software Organization Datacenter
    • Santa Clara Services Organization Datacenter
    • Sun Solution Center
    • Guillemont Park Campus, UK
    • Prague, Czech Republic
    • Bangalore, India
    • Louisville, Colorado: Sun Modular Datacenter
  • Summary
    • Looking Toward the Future
    • About the Authors
      • Dean Nelson
      • Michael Ryan
      • Serena Devito
      • Ramesh KV
      • Petr Vlasaty
      • Brett Rucker
      • Brian Day
    • Acknowledgments
    • References
    • Ordering Sun Documents
    • Accessing Sun Documentation Online
About the Authors
Dean Nelson

Dean is the Senior Director of Global Lab & Datacenter Design Services (GDS) in the Workplace Resources business unit of Sun Microsystems. The GDS organization bridges the gap between Facilities and IT/Engineering and is currently managing more than USD $250 million in datacenter design and construction activity. The GDS work supports the Act portion of Sun's Eco Strategy, and was showcased at Sun's Eco Launch in August 2007.

Michael Ryan

Mike Ryan is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization. Mike graduated from San Jose State University with a degree in Mechanical Engineering. He is a licensed professional mechanical engineer and has been involved in the design, construction, and operation of mission-critical facilities for the past 18 years. His experience spans numerous industries such as gas turbine co-generation, semi-conductor manufacturing, and mission-critical datacenters. For the last seven years he has focused on the design and operation of mechanical and electrical infrastructure systems supporting high-availability datacenters. mike joined Sun as a Staff Engineer for the GDS organization in March of 2006. Mike is the primary author of the GDS physical standards and was the lead design engineer for the Santa Clara, California datacenter project, one of the largest and most complex in Sun's history. Mike's design work supports the Act portion of Sun's Eco strategy, and was showcased at Sun's Eco Launch in August 2007.

Serena DeVito

Serena DeVito is a Datacenter Design Engineer in Sun's Global Lab & Datacenter Design Services organization. Serena joined Sun over ten years ago and has spent the bulk of her career as a system administrator within the IT organization where she designed, deployed, and supported critical infrastructure for the company. She then moved to the Software organization and managed the beta testing project for the Sun Cluster product group. Serena acted in a customer capacity running early releases of Sun Cluster software on production home directory clusters with over one-hundred active users as well as clustered web services. Serena identified, documents, and filed bugs while supporting one of Sun's largest and most complex engineering lab environments.

Ramesh KV

Ramesh has been in the technology industry for over thirteen years and has held a variety of technical & managerial roles since joining Sun in 2001. In 2007 Ramesh became a Regional Work Place Manager (RWPM) covering South Asia and India in addition to his GDS responsibilities. In this role he has been responsible for all WR activities across South Asia and India, including portfolio management, real estate services, and facilities management. Ramesh has been a critical member of the GDS technology team since 2004 and has designed multiple labs and datacenters across the Asia Pacific. Ramesh is the recipient of the PC Quest IT Implementation project of the year award in 2007 and has received twelve internal awards globally including Sun's Leadership Award in 2005.

Petr Vlasaty

Petr is a Datacenter Architect and the GDS Technical Infrastructure Lead for the EMEA region. Petr has been working in the IT industry for more than 12 years. In November 2004, Petr joined the Sun Software organization as R&D Datacenter Manager and Team Lead for the Prague Datacenter Management Team. Petr designed and implemented two internal datacenter projects in Prague in partnership with the GDS team. He applied the GDS processes and standards for the new Prague datacenter expansion that was completed in June 2006. Due to Petr's outstanding performance in Prague, he was assigned as the technical lead for the Louisville, Colorado consolidation project in August of 2006. He spent four months in Colorado developing the very detailed and complex plan. Petr joined the GDS organization full time in January of 2007. He is responsible for the Europe, Middle-East and Africa (EMEA) region and continues to act as technical lead for the Colorado datacenter consolidation project, the largest and most complex in Sun's history.

Brett Rucker

Brett Rucker is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization where he focuses on physical environment issues. Brett has been in the 52 Summary Sun Microsystems, Inc. Corporate Engineering, Construction and Management business for 25 Years. A Graduate of the University of Colorado, Boulder, and a licensed Professional Engineer, Brett began his career with Ball Aerospace Systems Division, a rich opportunity that gave him complete design and management responsibility for all construction projects. He supervised products including building additions, clean rooms, X-Ray rooms, datacenters, laboratories, anechoic chambers, secure areas, gallium arsenide facilities, high elevation satellite dish structural mounts, RF shielded rooms, and office, conference, and customer show areas.

Brian Day

Brian Day is a Senior Program Manager in Sun's Global Lab & Datacenter Design Services organization. Brian has been working in the technology industry for over 10 years, of which more than five have been with Sun. He spent over two years in Sun's Systems group, developing automation programs for server, storage, and software QA functions. Brian left Sun in 2000 to pursue his MBA full time. He joined Adobe Systems in 2001 as a Program Manager, leading the development of several Digital Imaging plug-ins for Adobe Acrobat. Brian returned to Sun in early 2005 to program manage the GDS initiative. As Senior Program Manager & Chief of Staff for GDS, Brian is responsible for first engagement with new customers, ensuring world-wide GDS projects are resourced, deliverable expectations are set, and commitments honored.

Acknowledgments

The authors would like to thank Steve Gaede, an independent technical writer and engineer, for his probing questions, deep insights, and ability to create a coherent story. Steve is a frequent contributor to Sun Microsystems technical documents, including many Sun BluePrints articles. He is a member of ACM and USENIX, and is active in the Boulder, Colorado professional community, having been a coordinator of the Front Page UNIX Users Group since 1984.

Thanks to the Sun MD team members who spent time contributing to this paper, including Maurice Cloutier, Liz From, Brian Kowalski, and Bob Schilmoeller.

The authors would also like to thank the following Sun contributors for the helpful and thoughtful comments they provided while reviewing this paper: Phil Morris, James Monahan, Rob Snevely, Steve Evans, Tracy Shintaku, and Peter Spence.

Rate this blueprint (Log In to vote.)
Choices Your Vote

Great

Good

Fair

Poor

Labels

new new Delete
datacenter datacenter Delete
eco eco Delete
power power Delete
cooling cooling Delete
blueprint blueprint Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.

Sign up or Log in to add a comment or watch this page.


The individuals who post here are part of the extended Sun Microsystems community and they might not be employed or in any way formally affiliated with Sun Microsystems. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Sun nor any other party necessarily agrees with them.

© 2010, Oracle Corporation and/or its affiliates
Powered by Atlassian Confluence
Oracle Social Media Participation Policy Privacy Policy Terms of Use Trademarks Site Map Employment Investor Relations Contact