It makes me think about how low background steel has become a precious commodity. Steel that was made prior to the first atomic bombs has a unique value because it’s uncontaminated.

We have archives of the internet prior to AI as we currently know it coming into widespread use. It seems like the future of all LLM model designers are going to need to be very crafty about their source of data and not just ingest everything they crawl.