“Three half-baked research ideas” by Owain_Evans

Here's three research ideas that seems valuable to pursue. My team is not working on this right now and we haven’t done serious research on it. So take what I say with a grain of salt. I've also not taken the time to write up all my thoughts on these ideas or to provide links and related work. These aren't the highest priority directions but just directions I haven't seen other people pursuing. DIRECTION 1: Do models learn to reward hack even if they are never rewarded for hacking in training? There's an effort in the AI labs to remove the “bad” RL environments from post-training. This includes environments that are impossible (so the only way to get reward is to do some unintended hack) or that are easy to hack (so models will often favor hacking). I expect that if the post-training was cleaned up in this way, models would have less propensity to hack at deployment. But how much does the propensity go down? Does it actually go to ~zero? And how does it scale with model capabilities? It would be good to know if reward hacking can emerge even if the model is [...] --- Outline: (00:34) DIRECTION 1: Do models learn to reward hack even if they are never rewarded for hacking in training? (02:10) DIRECTION: Data poisoning during RSI: risks and mitigations (06:22) Direction: What practitioners should know about subliminal learning and subtle generalization (phantom transfer) and related phenomena. --- First published: November 8th, 2026 Source: https://www.lesswrong.com/posts/vAniKbagqhyzpha6r/three-half-baked-research-ideas --- Narrated by TYPE III AUDIO .