Last May, Microsoft announced its Technical Computing Initiative, which ambitiously seeks to support researchers in their quest to solve the world’s biggest challenges. To fulfill this goal, the project aims to simplify the often arduous task of parallel programming by providing developers with the tools they need to create parallel applications in multiple environments, whether it’s on a desktop, a cluster, or in the cloud.
Today, as part of that initiative, Microsoft launched TC Labs, a portal at MSDN DevLabs that gives developers early access to Microsoft Technical Computing software as well as the opportunity to ask questions and provide feedback. The types of technologies and services available include parallel development tools in Visual Studio, distributed computing environments with Windows High Performance Computing (HPC) Server, cloud computing with Windows Azure, as well a variety of partner applications. Senior vice president of the developer division at Microsoft S. Somasegar has written a progress report of sorts about the Technical Computing Initiative on his blog.
Somasegar makes note of several important developments since the May rollout. First, Visual Studio 2010 now includes built-in support for developing, debugging, and tuning multicore and manycore applications. Second is the November debut of Service Pack 1 for HPC Server 2008 R2. The release taps into the cloud computing power of Windows Azure to give parallel applications the ability to scale from the cluster to the cloud.
Currently, TC Labs has three new pre-release technologies: TPL Dataflow; Dryad, DSC, and DryadLINQ, and Sho. Somasegar provides a rundown on each of them:
TPL Dataflow is a new member of [the .NET 4] family, layering on top of tasks, concurrent collections, and more to enable the development of powerful and efficient .NET-based concurrent systems built using dataflow concepts. The technology relies on techniques based on in-process message passing and asynchronous pipelines and is heavily inspired by the Visual C++ 2010 Asynchronous Agents Library and DevLab’s Axum language. TPL Dataflow provides solutions for buffering and processing data, building systems that need high-throughput and low-latency processing of data, and building agent/actor-based systems. TPL Dataflow was also designed to smoothly integrate with the new asynchronous language functionality in C# and Visual Basic….
Dryad, DSC, and DryadLINQ
Pioneered in Microsoft Research, Dryad, DSC, and DryadLINQ are a set of technologies that support data-intensive computing applications on Windows HPC Server 2008 R2 Service Pack 1. These technologies enable efficient processing of large volumes of data in many types of applications, including data-mining applications, image and stream processing, and various kinds of intense scientific computations. Dryad and DSC run on the cluster to support data-intensive computing and manage data that is partitioned across the cluster, while DryadLINQ allows developers to build data- and compute-intensive .NET applications using the familiar LINQ programming model.
Also begun in Microsoft Research, Sho provides those who are working on technical computing workloads an interactive environment for data analysis and scientific computing. It lets you seamlessly connect scripts written in IronPython with .NET libraries, enabling fast and flexible prototyping. The environment includes powerful and efficient libraries for linear algebra and data visualization, both of which can be used from any .NET language, as well as a feature-rich interactive shell for rapid development. Sho comes with packages for large-scale parallel computing (via Windows HPC Server and Windows Azure), statistics, and optimization, as well as an extensible package mechanism that makes it easy for you to create and share your own packages.
DevLabs welcomes community feedback and insight. These tools are prototypes, not products, but depending on user response, they could develop into fully-supported releases or become part of the shared-source community.