Why Memory and GPUs Have to Be Co-Located “If you want to do large-scale training like we do, you need the memory to be co-located with a large cluster of GPUs. I can’t just rent from Google, Microsoft, or even Base10 and run training at the scale I want because I need a gigantic memory card next to the GPUs, containing all my data and accessible by the entire cluster.” @cliffweitzman Love to hear your thoughts on this @ctnzr @tri_dao @matei_zaharia @NaveenGRao