Is parallel processing dead, or are we just missing the boat?

Transcription

1 Is parallel processing dead, or are we just missing the boat? Ananth Grama Computer Sciences, Purdue University.

2 Is parallel processing dead, or are we just missing the boat? The problems and technologies that motivate parallel processing are very much alive (and they are getting more and more challenging), but we are completely missing the boat, both as a research community and as funding entities. And just how are we missing the boat? By focusing on easy problems that are not just irrelevant, but misleading and misdirected.

3 On Performance: Here is an illustration in a conversation I (AYG) had with a senior colleague (SC) this past week, that typifies the problem with our field: SC: I ran Dyna3D on a PC and an SGI. AYG: Oh really, and how did they compare? SC: A PC was roughly the same speed as a four node Origin (our Origin runs 250MHz R10K processors). AYG: Cool. SC: And the performance of the origin saturated and declined after 8 processors. AYG: Wow, not very scalable, is it? SC: No, not at all. For the amount of money we paid for this thing! I m just going to port it to a cluster and save a whole bunch of money.

4 On Performance: On the surface, a cluster seems an obvious choice for SC, but think of what we are saying here. Observation: "A PC was roughly the same speed as a four node Origin" Interpretation: This means that a 2 GHz PC is a whole lot faster than a 250 MHz R10K (fair enough). Fact: The Origin has a 390MB/s interconnect. Fact: The fastest network (a Myrinet or a Gb Ethernet) gets us about 500Mb/s (50% efficiency would be great!) or about 60MB/s. Conclusion: In view of these, we are really saying, the program isnt very scalable on a slow processor with a very fast interconnect, so I am going to put very fast processors on a much slower interconnect and hope for better! Uhh...?

5 On Cost and Effort: It is NEVER the hardware cost! The industry (which has a much greater incentive for being cost effective) realized this a long time back that software costs are atleast an order of magnitude higher than associated hardware costs for large projects. Even in universities, research grants for software and algorithm development are much more than associated instrumentation costs. Taking away bandwidth (clusters instead of tightly coupled machines) makes algorithm and software design much more difficult. So here is what we are saying now: I will save money on hardware by getting a cluster and in doing so, make software development even more difficult, when the latter was the dominant cost in the first place. Uhh...?

6 On Scaling, Distributed Platforms, Services Infrastructure: Nothing scales to a thousand processors unless proven otherwise. If nothing scales to a thousand processors on a tightly coupled machines, could it possibly scale on geographically dispersed platforms where latencies are much higher and bandwidths lower. The industry realized the value of service infrastructures and distributed platforms a while back. We should learn from WebSphere, WebLogic, much of the existing P2P services insfrastructure (SLP, SDS, JXTA, Ninja) and distributed platforms (CORBA, RMI, JINI, DCOM, DOT-NET). Do we really need to reinvent all of these technologies?

7 Thoughts to Ponder.. There are largely two kinds of problems - embarassingly parallel and tightly coupled. While much of the current effort on systems development has focused on the former (and the easier one), the latter (more challenging one) has largely been orphaned. There are some tremendous emerging applications that can use all the help they can get from systems and infrastructure developers. As a research community and as funding agencies supporting this research community, we must introspect on how to address these harder problems, otherwise, we shall doom parallelism to a very quick death.