For large HTC (or HPC) computations on the cloud, ‘spot instances’ (AWS-speak), ‘low-priority VM’ (Azure-speak) or ‘preemptible VM instances’ (GoogleCloud-speak) are the low cost options for compute. Of course, the challenge here is that these instances/VM can vanish at anytime. If you’re doing a large HTC task, you want to make sure you save your result (and/or checkpoint) files as soon as they are generated to persistent storage. Otherwise you lost the computation you just paid for....
cost effective high-throughput computing in the cloud
High-throughput computing (HTC) is the term to describe the use of many computing resources over long periods of time to accomplish a computational task or job. Robustness and reliability of jobs over a long-time scale is a key concern. The metric for HTC is jobs per month. Typically the jobs are loosely-coupled....