Memory before Goal: Role of Pre-History Memory for Humanoid Loco-Manipulation