Distributed computing query

[juliet]

One of my users has asked if there's any possibility of setting something up to use our local network for distributed computing purposes - basically so that people who have Very Large Calculations to run can use unused cycles (e.g. overnight) on other people's machines to run those calculations rather than relying on their own machines.

I have googled briefly but am a bit lost... (I think I'm at the stage where I don't know enough about what's out there to google usefully). Any of you lot know anything about this? (We're running Linux.)

I have looked at BOINC but from what I can see that works by you setting up a server & then getting individuals out in the World At Large to subscribe to your project; which isn't quite what I'm after (I only want it to run on our local machines).

Er. Anyway. Any useful comments or pointers would be greatly appreciated - I'm failing to get a mental handle on this at all atm!

Comments

[rgl.livejournal.com]

I've not been involved with it myself (either as implementor or user), but I know that one of UCL's clustering systems uses Condor (http://www.cs.wisc.edu/condor/), which sounds a bit like what you're after.

[juliet]

Aha - they do explicitly say there that Condor has mechanisms that make it deal well with unused desktop systems. Thanks! I'll have a bit more of a look at it tomorrow.

[martling.livejournal.com]

Our department runs Condor on all its workstations to do exactly this, so probably worth a look. It has commands for locally logged in users to suspend it if it bothers them and so on. (This may or may not have something to do with me improvising my own scripts for distributing several months worth of simulations in time for my honours project deadline.)

[easterbunny.livejournal.com]

Are you still at IC? Do you know the guys on the ICENI project? They may be able to help.

[juliet]

I am still at IC; but I don't know the ICENI people. However! I have found their website & will take a look at their stuff now as well. Thanks!

[uon.livejournal.com]

GRID WILL EAT YOU

[mzdt.livejournal.com]

There's obviously SETI, and the climate change calculation thing, which may be the same as ICENI above...?

[juliet]

SETI runs using BOINC - as above, it doesn't quite match what I want because the way it works (as far as I can ascertain right now) is that you can run a project server, but then the World At Large does the running of things for you; we want something that uses our local desktops when they're not being run.

It might be possible to make BOINC do what I want but it doesn't seem designed for that sort of local use :-/

[weatherpixie.livejournal.com]

If you made the signup url only available on your intranet, then that would be one simple way of limiting where it got installed. If you see what I mean.

[weatherpixie.livejournal.com]

And you could do clever network stuff limiting only IPs on your network to your project server, or something...

[weatherpixie.livejournal.com]

This also implys that you can do it in a limited fashion.

http://boinc.berkeley.edu/vcsc.php

[juliet]

Fab, thank you! I think I'll try Condor first, but will keep this in mind.