Tom Brown is energetic, stubborn, kind-hearted, and athletic more than intellectual. He acts according to his feelings and the unwritten rules of the boys around him more than adults' rules.

The early chapters of the novel deal with his childhood at his home in the Vale of White Horse (including a nostalgic picture of a village feast). Much of the scene setting in the first chapter is deeply revealing of Victorian England's attitudes towards society and class, and contains a comparison of so-called Saxon and Norman influences on England. This part of the book, when young Tom wanders the valleys freely on his pony, serves as a sort of Eden with which to contrast the later hellish experiences in his first years at school.

His first school year is at a local school. His second year starts at a private school, but due to an epidemic of fever in the area, all the school's boys are sent home, and Tom is transferred mid-term to Rugby School, where he makes acquaintance with the adults and boys who live at the school and in its environs.

On his arrival, the eleven-year-old Tom Brown is looked after by a more experienced classmate, Harry "Scud" East. Soon after, Tom and East become the targets of a bully named Flashman. The intensity of the bullying increases, and, after refusing to hand over a sweepstake ticket for the favourite in a horse race, Tom is deliberately burned in front of a fire. Tom and East eventually defeat Flashman with the help of a kind (though comical) older boy, Diggs. In their triumph they become unruly.

In the second half of the book, Dr. Thomas Arnold, the historical headmaster of the school at the time, gives Tom the care of George Arthur, a frail, pious, academically brilliant, gauche, and sensitive new boy. A fight that Tom gets into to protect Arthur, and Arthur's nearly dying of fever, are described in loving detail. Tom and Arthur help each other and their friends develop into young gentlemen who say their nightly prayers, do not cheat on homework, and play in a cricket match. An epilogue shows Tom's return to Rugby and its chapel when he hears of Arnold's death.