View Source

h4. [Energy Efficient Datacenters: Electrical Design !Main^download.gif!|http://mapping.sun.com/profile/offer.jsp?id=230]

*by Michael Ryan, Brett Rucker, Dean Nelson, Petr Vlasaty, Ramesh KV, Serena DeVito, and Brian Day{*}{excerpt}
March 2009{excerpt}

This Sun BluePrints™ article discusses the electrical design principles that Sun's Global Lab & Datacenter Design Services (GDS) organization incorporated into its energy-efficient, pod-based datacenters. Sun's GDS organization is responsible for Sun's technical infrastructure (datacenters, labs, server and communication rooms and wiring closets) worldwide, and has ushered in a new generation of energy-efficient datacenters at Sun.

• "Traditional Datacenter Electrical Design" discusses how datacenter electrical systems are typically designed
• "Increasing Electrical Distribution Efficiency" highlights the opportunities that we have found to improve modularity and efficiency in datacenter electrical design.
• "Flexibility Through Modularity" shows how our design choices create flexible datacenters that contribute to business agility.
• "Extending the Datacenter Lifecycle" discusses the importance of coordinating expected datacenter lifecycles with equipment refresh cycles. It shows the many ways in which datacenters can be built from day one to support future expansion
• "Sun's Electrical Design At Work" provides a tour of our datacenter designs from the perspective of the one-line diagrams that describe their electrical systems. We focus on Sun's Santa Clara, California datacenters, including our most recent addition of Sun™ Modular Datacenters at the site.

h4. Contents

* Introduction
** About This Article
* Traditional Datacenter Electrical Design
** Utility Power and Transformer
** Switchgear
** Backup Power
** Power Distribution
** Rack Power Distribution
** Datacenter Tier Levels
** Sun's Approach to Electrical Design
* Increasing Electrical Distribution Efficiency
** Component Selection
** Equipment Power Supplies
** Power Characteristics
** Appropriate Levels of Redundancy
* Flexibility Through Modularity
** Datacenter Power Distribution
* Extending the Datacenter Lifecycle
** Electrical Yard
** Building Infrastructure
** Pod Power Distribution
* Sun's Electrical Design At Work
** Santa Clara, California
** Broomfield, Colorado
** Sun Modular Datacenter
* Summary
* About The Authors
* Acknowledgments
* References
* Ordering Sun Documents
* Accessing Sun Documentation Online

{panel:title=About the Authors|borderStyle=solid|titleBGColor=#F8D583|bgColor=white}
Mike Ryan is a Senior Staff Engineer in Sun's Global Lab & Datacenter Design Services organization. Mike graduated from San Jose State University with a degree in mechanical engineering. He is a licensed professional mechanical engineer and has been involved in the design, construction, and operation of mission-critical facilities for the past 18 years. His experience spans numerous industries such as gas turbine co- generation, semi-conductor manufacturing, and mission-critical datacenters. For the last seven years he has focused on the design and operation of mechanical and electrical infrastructure systems supporting high-availability datacenters. Mike joined Sun as a Staff Engineer for the GDS organization in March of 2006. Mike is the primary author of the GDS physical standards and was the lead design engineer for the Santa Clara, California datacenter project, one of the largest and most complex in Sun's history. He also provided supporting engineering services for the Broomfield, Colorado datacenter project. Mike's design work supports the Act portion of Sun's Eco strategy, and was showcased at Sun's Eco Launch in August 2007.

Mike is a member of ASHRAE (American Society of Heating, Refrigeration, and Air- conditioning Engineers). He also participates in the Data Center Pulse and the Critical Facilities Round Table group. Mike was the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz ad CTO, Greg Papadopoulos in July of 2008.

Dean Nelson is the Senior Director of Global Lab & Datacenter Design Services (GDS) in the Work Environments (WE) business unit of Sun Microsystems. The GDS organization bridges the gap between Facilities, IT, and Engineering, and is responsible for managing Sun's multi-billion dollar global technical infrastructure portfolio including datacenter design, standards and strategy. The GDS work resides in the "Act" portion of Sun's Eco strategy, and was showcased at Sun's Eco Launch in August, 2007, and the Colorado Data Center grand opening in January, 2009.

Dean has been in the technology industry for 19 years, of which 16 have been with Sun. He spent four years in Sun manufacturing in roles ranging from component-level debug to managing quality. This included helping the drive to achieve ISO 9002 certification. Dean joined the Sun Engineering community in 1993. He led systems and network administration support for some of Sun's largest and most complex R&D lab environments.

Dean left Sun in 2000 to join a networking startup company called Allegro Networks. At Allegro, he built a world-class QA team, state of the art global R&D lab environments and fully integrated automation system.

In 2003, Dean returned to Sun joining the newly formed N1™ Software organization. He orchestrated the integration of Terraspring and Center Run R&D labs and the merger of Sun™ Cluster software into N1 technology. In mid-2003, Dean took over management all of the N1 organization's R&D labs, build engineering, automation, and capital budget responsibilities world-wide. In 2004, he became a leading member of the Global Lab & Datacenter Design Services (GDS) team tasked with creating a strategy to standardize Sun's multi-billion dollar technical infrastructure portfolio. He was the architect of the GDS operating model and lead design engineer for lab and datacenter projects world- wide. In 2006, he became the GDS Director. Since taking over the group, Dean has delivered the GDS strategy, executing the largest technical infrastructure consolidation in Sun's history compressing the 1.4 million square foot portfolio by 48 percent, a $250 million dollar investment.

Dean holds numerous industry technical and business board positions including Founder of Data Center Pulse (http://datacenterpulse.org) - an exclusive datacenter owner community. Dean was featured in Contrarian Minds, a focus on the engineers, scientists and dreamers of Sun Microsystems. Dean was also the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz and CTO, Greg Papadopoulos in July of 2008. Dean lives in Northern California with his wife and daughter.

Serena DeVito is a Datacenter Design Engineer in Sun's Global Lab & Datacenter Design Services organization. Serena joined Sun over ten years ago and has spent the bulk of her career as a system administrator within the IT organization where she designed, deployed, and supported critical infrastructure for the company. She then moved to the Software organization and managed the beta testing project for the Sun Cluster product group. Serena acted in a customer capacity running early releases of Sun Cluster software on production home directory clusters with over one hundred active users as well as clustered web services. Serena identified, documented, and filed bugs while supporting one of Sun's largest and complex engineering lab environments.

Serena has been a core member of the GDS team since 2004. In December 2005 she joined the GDS organization to drive lab and datacenter consolidation projects world- wide. Along with building numerous labs and datacenters in the Americas, Serena was the technical lead and customer engagement engineer on the Santa Clara, California datacenter consolidation project, one of the largest and most complex datacenter consolidations in Sun's history. For this work, Serena was the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz ad CTO, Greg Papadopoulos in July of 2008.

Serena is a graduate of the University of Adelaide (Australia) with a Bachelors of Architectural Studies and a focus in Computer-Aided Design. She is a Sun Certified Solaris Administrator for the Solaris™ 7, 8 and 9 Operating Systems. She also loves traveling and riding horses.

Ramesh KV has been in the technology industry for over 13 years and has held a variety of technical & managerial roles since joining Sun in 2001.

Ramesh began his career as a customer support engineer at Kumdev computers, where his clients included companies in the manufacturing, pharmaceuticals, telecom and IT industries. He later joined SAS (now know as SASKEN) as a Systems Engineer, where he supported the NORTEL infrastructure. He then moved on to TATA Elxsi as a Design and Development Specialist, where he was involved in various key infrastructure projects.

Ramesh spent the initial years at Sun working for the RPE organization as Global Lab Manager, where hedrove the global lab model and designed, deployed, and supported critical infrastructure across USA, EMEA & India. In early 2006, Ramesh moved into the role of IT/ Lab Council head for the India Engineering Centre (IEC) reporting to the site VP as well as the Global Lab & Datacenter Services (GDS) lead for the APAC region. In this role Ramesh led the physical and technical design for the Bangalore, India Datacenter consolidation project and supported all APAC lab and datacenter projects. In 2007 Ramesh became a Regional Work Place Manager (RWPM) covering Asia South and India in addition to his GDS responsibilities. In this role he has been responsible for all workplace resources activities across Asia South & India, including portfolio management, real estate services & facilities management. Ramesh has been a critical member of the GDS technology team since 2004 and has designed multiple labs and
datacenters across Asia Pacific. Ramesh is the recipient of a research disclosure on "Web Based Multi OS Installation," PC Quest IT Implementation project of the year award in 2007 and has received 12 internal awards globally including Sun's leadership award in 2005 and 2007. Ramesh was the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz ad CTO, Greg Papadopoulos in July of 2008. Helives in Bangalore, India with his wife and daughter.

Petr Vlasaty has been working in the IT industry for more than 12 years. His experience that started as an implementation engineer and project coordinator in 1996 has grown into a variety of people, project, process and service management areas of datacenter operations and technical support services. Petr worked for ANECT to define and develop the System Support and Maintenance department responsible for systems implementations and support for key customers including the Czech Ministry of Finance,Commercial Bank, and others. Petr formed a 12-person team that defined internal and external processes, established contracts, and represented support services to customers. In November 2004, Petr joined the Sun Software organization as R&D Datacenter Manager and Team Lead for the Prague Datacenter Management Team. Petr designed and implemented two internal datacenter projects in Prague in partnership with the GDS team. He applied the GDS processes and standards for the new Prague datacenter expansion that was completed in June 2006. Due to Petr's outstanding performance in Prague, he was assigned as the technical lead for the Louisville, Colorado consolidation project in August of 2006. He spent four months in Colorado developing the very detailed and complex plan. Petr joined the GDS organization full time in January of 2007. He is responsible for the Europe, Middle-East and Africa (EMEA) region and continues to act as technical lead for the Colorado datacenter consolidation project, the largest and most complex in Sun's history.

Petr is a Staff Engineer, Datacenter Architect, and the GDS Technical Infrastructure Lead for the EMEA region. He loves sports like mountain hiking (year round, especially winter) and rock climbing. Petr lives with his wife Vera and 10 month-old son Erik in Prague, Czech Republic.

Petr is a CCSE (Check Point Certified Security Engineer), CCSA (Check Point Certified Security Administrator) and SCSA (Sun Certified Solaris Administrator). Petr was the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz ad CTO, Greg Papadopoulos in July of 2008.

Brett Rucker is a Staff Engineer in Sun's Global Lab & Datacenter Design Services organization where he focuses on physical environment issues. Brett has been in the Corporate Engineering, Construction and Management business for 25 Years. A Graduate of the University of Colorado, Boulder, and a licensed Professional Engineer, Brett began his career with Ball Aerospace Systems Division, a rich opportunity that gave him complete design and management responsibility for all construction projects. He supervised products including building additions, clean rooms, X-Ray rooms, datacenters, laboratories, anechoic chambers, secure areas, gallium arsenide facilities, high elevation satellite dish structural mounts, RF shielded rooms, and office, conference, and customer show areas. Highlights of Brett's time at Ball include design and construction of large secured program area in 7-days to support critical Corp. project, stick build design and construction of MAMA Lab Clean Room facility for 1/3 market cost (still in operation today with one of Ball's premier products), metal finish Area with a $30,000 per year energy savings in 1992 base, and creation of the Facility Engineering Program at Ball Aerospace which grew to take over Ball corporate acilities.

In 1992, Brett joined a newly forming engineering group at StorageTek to address facility backlog and project issues. Brett and his team created corporate building specifications, drawing standards, and quickly eliminated all backlog. Brett drove the high tech areas, and HVAC infrastructure as well as general projects. He represented StorageTek on refrigerant conversion and reported on this project at an ASHRAE function. Brett became manager of the New Project Management Group in 2000 directing projects at headquarters and in the field. His responsibilities soon included StorageTek's energy management program, which deployed processes and delivered high-quality, low-cost projects with consistent high metrics with limited staff support. This group lead all project designs, construction and project budgets. The energy management group delivered ISO Certification, installed monitoring systems, metrics, monthly log and issue reviews, best payback energy projects and commodity contracts that beat the market consistently. Brett joined Sun's GDS team in 2006 as a design engineer and is the physical design lead for the largest and most complex datacenter consolidation in Sun's history, the StorageTek move from Louisville, to Broomfield, Colorado.

Brett is a member of ASHRAE, and AEE, and he was the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz ad CTO, Greg Papadopoulos in July of 2008. Brett is active in developing youth programs and coaching his daughter's competitive fast-pitch team with great support from his wife and two daughters.

Brian Day is a Senior Program Manager in Sun's Global Lab & Datacenter Design Services organization. Brian has been working in the technology industry for over 10 years, of which more than five have been with Sun. He spent over two years in Sun's Systems group, developing automation programs for server, storage, and software QA functions. This work included testing, configuring, and administering some of Sun's largest systems at the time, including the Sun Enterprise™ 10000 and Sun Enterprise 6500 servers, and Sun StorEdge™ D1000 storage device. Brian then served in a similar role for iPlanet™ software, developing server side Java™ technology-based tools that automated core team processes (ticketing system, equipment tracking, on-call notification).

Brian left Sun in 2000 to pursue his MBA full time. He then joined Adobe Systems in 2001 as a Program Manager, leading the development of several Digital Imaging plug- ins for Adobe Acrobat. In 2003 he took over full time program management of the Photoshop Elements product, releasing an all new version (3.0) in the fall of 2004. This included the coordination of all key product teams (engineering, QA, marketing, localization, documentation) and ensuring a successful release for the critical holiday shopping season. Brian returned to Sun in early 2005 to program manage the GDS initiative. As Senior Program Manager & Chief of Staff for GDS, Brian is responsible for first engagement with new customers, ensuring world-wide GDS projects are resourced, deliverable expectations are set, and commitments honored. Brian's work in GDS has significantly contributed to Sun's modular design and the teams success. Brian was the recipient of Sun's prestigious Innovation Award presented by Sun's CEO, Jonathan Schwartz ad CTO, Greg Papadopoulos in July of 2008. Brian holds a Bachelor of Science in Computer Science and a Masters of Business Administration.
{panel}
{panel:title=Acknowledgments|borderStyle=solid|titleBGColor=#F8D583|bgColor=white}
The authors would like to thank Steve Gaede, an independent technical writer and engineer, for his probing questions, deep insights, and ability to create a coherent story. Steve is a frequent contributor to Sun Microsystems technical documents, including many Sun BluePrints articles. He is a member of ACM and USENIX, and is active in the Boulder Colorado professional community, having been a coordinator of the Front Range UNIX Users Group since 1984.

Thanks to Mark Monroe for providing input on power supply efficiency trends, and to James Monahan for his detailed review comments. Thanks also to the Sun MD team members who spent time contributing to this paper, including Maurice Cloutier, Liz From, Brian Kowalski, and Bob Schilmoeller.
{panel}

{vote:Rate this blueprint}
Great
Good
Fair
Poor
{vote}

The individuals who post here are part of the extended Sun Microsystems community and they might not be employed or in any way formally affiliated with Sun Microsystems. The opinions expressed here are their own, are not necessarily reviewed in advance by anyone but the individual authors, and neither Sun nor any other party necessarily agrees with them.

Copyright 1994-2009 Sun Microsystems, Inc.
Powered by Atlassian Confluence
Sun Guidelines on Public Discourse Privacy Policy Terms of Use Trademarks Site Map Employment Investor Relations Contact