Ignition Grant to Build a Distributed Desktop Supercomputer

[magorian]

Sent this to Diana, but in case anybody else is interested in the project. Dan

Title: Ignition Grant to build a prototype campus distributed desktop supercomputer

Short description: Dynamic distributed Beowulf subclusters are created by user
scheduling which ifup/ifdown subinterface address ranges of 0-36 slave Parallella
boards to 12 user master boards.

Long description:

Users log into phpScheduleIt (open-source calendaring program) running
on a Cubox (tiny headless box similar to the Parallella: arm, 1G mem,
gige, Ubuntu 12.10) to reserve blocks of time and how many slave boards
they'll need. I am writing a php plug-in to take those calendar entries
and turn them into cronjobs on the scheduler. When those cronjobs run,
they invoke PDSH which ifup and ifdowns the appropriate static eth0:X
subifs to create Beowulf subclusters for the duration of their
experiments. Each node has 10 subifs defined in
/etc/network/interfaces, so effectively the cluster can run 10
experiments simultaneously on different subsets of boards.

Each user will have a master board that they ssh into to run their code
and 3 slave boards that can be requested by theirs or any other master
node. With the 12 initial project clusters on campus, that will be 36
available slave boards. The network is a flat dedicated non-campus
gigabit ethernet from a central surplus Cisco 6503 layer 2 switch,
fibered to all users with unmanaged gige media converters connecting
their unmanaged 5-port desktop gige switches. That private network is
firewalled to the campus network as a routed labnet, so users can ssh to
their clusters from their campus net-attached desktop workstations. The
intent is to provide console as well since they may want to run GUIs
from their master boards instead.

SAN for large data sets is provided by a Synology appliance with 8 fast
4T enterprise-quality disks in a RAID 5 array. It's connected via 4 giges
etherchannelled together and mounted on all Parallellas via NFSv4
using jumbo frame 9K mtus to maximize performance. The local 64G
SD cards are just for OS and cache during jobs (not permanent data
storage) since slave boards will be frequently reassigned. Researcher
users are expected to use MPICH2 libraries for EZ parallelization of
their code (and yes, I do provide a Fortran as well as C compiler, but
they can also apt-get additional packages they need on their master
nodes) and I encourage them to write to the Epiphanies directly. When
the Epiphany interconnect cables arrive they will be experimented with
as well. The collaborating researchers have experimental numeric
applications in the areas of satellite development, weather modelling,
missile defense, code cracking, and other funded projects.

[shodruk]

What university?

[Sundance]

Hi #magorian

1. Who/whom is "Diana" - sorry, but I have no idea

2. What about finishing your [The same for you #shodruk - and that matter everybody ] profile - memberlist.php?mode=viewprofile&u=5144? Makes it easier to communicate, I think.

3. - and this is my view on the idea.

The "Parallalla" is not robust enough to become a "Distributed Desktop HPC" for daily use and the "Parallalla" has limited I/O, as the 1G Ethernet is not fast enough, but the Epiphany DSP concept surely is.

If you are based in the European Union, then Sundance would be very interested in helping getting a grant inside a H2020 Project

- http://ec.europa.eu/programmes/horizon2020/

for building such a machine/solution.

FC.