Risks from Learned Optimization in Advanced Machine Learning Systems

We analyze the type of learned optimization that occurs when a learned model
(such as a neural network) is itself an optimizer – a situation we refer to as
mesa-optimization, a neologism we introduce in this paper. We believe that the
possibility of mesa-… Read more

Similar