It is a good idea if some stones are loose or missing altogether. It is possible to design a wall to include planters.
In any case you might want to make sure the wall is protected from damp. Once you fill the gaps with soil and start growing plants there, the surfaces won’t be able to dry themselves naturally. Hence, the dampness from the soil might cause problems.
Also, depending on what you grow, the roots might try to take the wall apart by penetrating into the seams. That’s why it’s better to seal the inside of the wall planter with natural tar or similar heavy-duty sealant.
I never thought this could be a remedy for my backyard. Thanks!