Tweet by Dan Hendrycks:

This is worth checking out.

Minor criticisms:
I think industry's "algorithmic secrets" are not a very natural leverage point to greatly restrict.
FlashAttention, Quiet-STaR (q*), Mamba/SSMs, FineWeb, and so on are ideas and advances from outside industry. These advances will… https://t.co/46JawK9K8j