The Children's Society said it discovered some "heart-breaking" stories, including a nine-year-old who had moved house eight times and attended four different schools.
Interviews with scores of children found problems worsened as they get older, with teenagers going hungry and being punished for breaking school uniform rules.
One nine-year-old girl said she and her brothers took it in turns to beg strangers or friends and family for money when family finances reached breaking point.
Matthew Reed, chief executive of The Children's Society, said: "The heart-breaking testimonies of children in this report offer a glimpse into the harsh realities of life for the four million children growing up in poverty in the UK.
"Moving from place to place, living in neighbourhoods where they are frightened to go outside, and travelling for hours to get to school are pressures that no child should have to deal with.
"Yet for some of the children we interviewed, these have become normal parts of their lives.
"This lack of stability and security is hugely damaging to children's wellbeing and could have long-term repercussions for their mental health."
The charity called for the Government to ensure that financial support for housing costs increases in line with local rents for families who are renting privately, to help poorer families secure their children's homes over the long term.
A Government spokesman said: "We've seen the number of children living in workless households fall by over half a million since 2010.
"By introducing Universal Credit, the National Living Wage and increasing the tax-free Personal Allowance, we're ensuring it always pays to be in work.
"Nevertheless, we continue to spend Â£24billion a year on housing benefit to support those families who need it the most."