"The amount of money that it would take to hold our rawest forms of data is insane - I don't even know where we would fit that many computers. So, we have to make some compromises."
"This is a problem that libraries have been dealing with for as long as libraries have existed. We cannot physically collect all the books that we want to collect, and in 50 years, the book may not be useful any more."
The Square Kilometre Array Observatory will consist of telescopes in South Africa and Australia, generating about 700 petabytes of data annually. This is only 1% of its potential output of 60 exabytes per year. The vast data volume presents storage challenges, as retaining all data is financially and logistically unfeasible. The growth of machine learning and AI increases the desire to retain data, but this is not sustainable. Historical parallels exist with libraries, which have always faced limitations in data collection and retention.
Read at Nature
Unable to calculate read time
Collection
[
|
...
]