A glance at Distributed Computing

There are a lot of difficult problems out in the real world which require massive computational power to solve. Problems in genomics, scheduling, drug design, weather and the stock market. IBM refers to the class of tools required to address these complex problems Deep Computing. Basically, Deep Computing is the intelligent application of computational power to these problems. Distributed computing is a way of gathering the raw processing power for these solutions.

A popular example is SETI@home. SETI is the Search for Extra-Terrestrial Intelligence, which is looking at radio telescope data for signs of intelligent life beyond our planet. Radio telescopes produce a bunch of data and crunching through these signals looking for indicators of ET would take any one computer (even the most powerful supercomputers) hundreds of years to process. So, you break the problem into smaller pieces and distribute these pieces to thousands of computers for them to solve and report back their answers to a central server. Instant raw processing power!

So, how are people using distributed computing for the stock market? Well, at least one public project, MoneyBee, is trying to leverage distributed computing to tackle the stock market. They have a screen saver that you can download and it will use your computer when it is idle to process jobs. These jobs are training neural networks to predict the movements of individual stocks and indexes. Okay, so I am not a big fan of neural networks, but we will have the PRO and CON debate about them in the future. The point here is MoneyBee is gathering large amounts of processing power and focusing it on a problem -> predicting individual stock movements. Not many other public projects out there, although Intelligent Broker appeared a couple years ago (and is now gone – this link points to an archive in archive.org), but never really seemed to do anything. It seems the concept was to have an open platform for developers to use the processing power of 100′s of machines to create stock market related trading systems that could then be sold to traders interested in the systems. A cool idea, but it seems to be dead now.

So, think about distributed computing when you happen upon a problem that requires a good amount of processing power to solve. There are plenty of distributed computing platforms that allow you to develop applications to take advantage. Another great resource that lists other public non-market related projects is Internet-base Distributed Computing Projects web page.