Cloud Computing for Science and Engineering

A guide to cloud computing for students, scientists, and engineers, with advice and many hands-on examples. The emergence of powerful, always-on cloud utilities has transformed how consumers interact with information technology, enabling video streaming, intelligent personal assistants, and the sharing of content. Businesses, too, have benefited from the cloud, outsourcing much of their information technology to cloud services. Science, however, has not fully exploited the advantages of the cloud. Could scientific discovery be accelerated if mundane chores were automated and outsourced to the cloud? Leading computer scientists Ian Foster and Dennis Gannon argue that it can, and in this book offer a guide to cloud computing for students, scientists, and engineers, with advice and many hands-on examples. The book surveys the technology that underpins the cloud, new approaches to technical problems enabled by the cloud, and the concepts required to integrate cloud services into scientific work. It covers managing data in the cloud, and how to program these services; computing in the cloud, from deploying single virtual machines or containers to supporting basic interactive science experiments to gathering clusters of machines to do data analytics; using the cloud as a platform for automating analysis procedures, machine learning, and analyzing streaming data; building your own cloud with open source software; and cloud security. A variety of supplementary material is available at Cloud4SciEng.org, including exercises, lecture slides, and other resources helpful to readers and instructors. This book helps make the cloud computing ecosystem comprehensible for scientist and student alike. Foster and Gannon provide an introduction to concepts, an explanation of systems, clean code examples in Python, and even downloadable Jupyter notebooks. -Simson L. Garfinkel Cloud Computing Instructor; author of Architects of the Information Society. Content: A guide to cloud computing for students, scientists, and engineers, with advice and many hands-on examples. The emergence of powerful, always-on cloud utilities has transformed how consumers interact with information technology, enabling video streaming, intelligent personal assistants, and the sharing of content. Businesses, too, have benefited from the cloud, outsourcing much of their information technology to cloud services. Science, however, has not fully exploited the advantages of the cloud. Could scientific discovery be accelerated if mundane chores were automated and outsourced to the cloud? Leading computer scientists Ian Foster and Dennis Gannon argue that it can, and in this book offer a guide to cloud computing for students, scientists, and engineers, with advice and many hands-on examples. The book surveys the technology that underpins the cloud, new approaches to technical problems enabled by the cloud, and the concepts required to integrate cloud services into scientific work. It covers managing data in the cloud, and how to program these services; computing in the cloud, from deploying single virtual machines or containers to supporting basic interactive science experiments to gathering clusters of machines to do data analytics; using the cloud as a platform for automating analysis procedures, machine learning, and analyzing streaming data; building your own cloud with open source software; and cloud security. A variety of supplementary material is available at Cloud4SciEng.org, including exercises, lecture slides, and other resources helpful to readers and instructors. This book helps make the cloud computing ecosystem comprehensible for scientist and student alike. Foster and Gannon provide an introduction to concepts, an explanation of systems, clean code examples in Python, and even downloadable Jupyter notebooks. -Simson L. Garfinkel Cloud Computing Instructor; author of Architects of the Information Society. Content: Part I. Managing Data in the Cloud-2. Storage as a Service. 3. Using Cloud Storage Services. Part II. Computing in the Cloud-4. Computing as a Service. 5. Using and Managing Virtual Machines. 6. Using and Managing Containers. 7. Scaling Deployments. Part III. The Cloud as Platform-8. Data Analytics in the Cloud. 9. Streaming Data to the Cloud. 10. Machine Learning in the Cloud. 11. The Globus Research Data Management Platform. Part IV. Building Your Own Cloud-12. Building Your Own Cloud with Eucalyptus (with Rich Wolski). 13. Building Your Own Cloud with OpenStack (with Stig Telfer) the Bucket. 14. Building Your Own SaaS. Part V. Security and Other Topics-15. Security and Privacy. 16. History, Critiques, Futures. 17. Jupyter Notebooks. 18. Afterword: A Discovery Cloud. Bibliography. Index.Part I. Managing Data in the Cloud-2. Storage as a Service. 3. Using Cloud Storage Services. Part II. Computing in the Cloud-4. Computing as a Service. 5. Using and Managing Virtual Machines. 6. Using and Managing Containers. 7. Scaling Deployments.