performance - Is it more efficient to use file IO or swap space for large temporary storage?

Question

I have a program that needs to produce a large quantity of temporary data (in the 10-100 gigabyte range) that I don't expect to fit in memory. The data is only used internally to the program, like a stack: it writes the file and then reads it once in reverse, and then it's done. Given such a usage pattern, is it more efficient to use file IO (because I know it's going to be hitting the disk), or write entirely to a huge virtual memory allocation and rely on the OS to perform paging (which should not be inefficient because the working set is small, due to the linear usage of memory)?

question from:https://stackoverflow.com/questions/65840905/is-it-more-efficient-to-use-file-io-or-swap-space-for-large-temporary-storage

Answer 1

Waitting for answers