|
I think one of the things that set HPC apart is the way storage is handled. As far as I know, most cloud services tend to have local or near-local/nearline storage for each of the racks they use, not unlike mainframes. HPC uses distributed storage, and none of their compute nodes have ANY storage aside from as much RAM as they can possibly fit. Heck, Lustre on ZFS is the entire reason Linux has ZFS - I'm not sure it would've happened otherwise.
|
#
?
Feb 24, 2021 08:56
|
|
|
|
| # ? Feb 25, 2021 07:38 |
|
|
Most cloud services are engineered to use object storage as well. That's entirely unworkable when your model requires fast random access across a 10TB dataset. A 10TB dataset isn't even large. e: This gets funny when I tell people that no, "the cloud" is a terrible fit for us. I've run into so many people who can not comprehend that anything outside the realm of web crap exists. Like this one dude who kept asking me about our "app" and how we need to refactor our "app" and I just could not get through to him that we don't have an "app", we have a bunch of fuckoffhuge datasets and people write code as required to run against it. We can't put that in s3, we need fast low-latency access that can saturate 10Gpbs ethernet for testing before moving to big iron for the final run. Antigravitas fucked around with this message at 10:58 on Feb 24, 2021 |
|
#
?
Feb 24, 2021 10:52
|
|
|
AWS are trying! But of course this is with HPC focused datacentres not just buzzwords. https://www.nas.nasa.gov/publications/ams/2020/12-17-20.html
|
|
#
?
Feb 24, 2021 12:27
|
|
|
Antigravitas posted:A 10TB dataset isn't even large. NAME USED AVAIL REFER MOUNTPOINT storage 25.0G 38.4T 2.65G /mnt/storage
|
|
#
?
Feb 24, 2021 20:34
|
|
|
pfft if your columns don't end with P or E is it even really data??
|
|
#
?
Feb 24, 2021 20:42
|
|
|
xzzy posted:pfft if your columns don't end with P or E is it even really data?? really more like a loose collection of anecdotes
|
|
#
?
Feb 24, 2021 22:42
|
|
|
xzzy posted:Pretty much every government HPC cluster was given a mandate to start running protein folding jobs in the last year too. I bet you can't guess why! I would assume Alphafold.
|
|
#
?
Feb 25, 2021 05:17
|
|
|
|
| # ? Feb 25, 2021 07:38 |
|
|
xzzy posted:pfft if your columns don't end with P or E is it even really data?? That's this pool on my server. All drives are a minimum of 2TB, and will be replaced with bigger drives as money permits (minimum 8TB drives, as those aren't SMR). BlankSystemDaemon fucked around with this message at 07:35 on Feb 25, 2021 |
|
#
?
Feb 25, 2021 07:24
|
|