Pay a visit to two of Montego Bay’s most storied mansions and discover their fascinating stories, as well as the region’s history of slavery. Only a seven-minute drive from the resort, the Rose Hall Great House is Jamaica’s most famous plantation house, boasting beautiful 18th-century Jamaican Georgian architecture and panoramic views of the Caribbean coast. Once an incredibly profitable sugarcane plantation, today it has two celebrated golf courses and both day and night tours (the latter of which focuses on the spine-tingling tale of the so-called “White Witch of Rose Hall.” For a different look into Jamaica’s past, head to the Greenwood Great House instead. Constructed in the 1780s by Richard Barrett, cousin of famed English poet Elizabeth Barret Browning, the mansion remains one of the country’s best-preserved homes from the plantation era—even all of the furnishings are original. Its preservation is largely credited to Barrett’s kind treatment of the enslaved people under his purview (helping them to become literate, a criminal offense at the time), which ultimately saved the home from being razed in the slave uprising of 1831.

Journey below the ground and into a piece of Jamaica’s history at the Green Grotto Caves. Located a leisurely, hourlong drive east along the island’s North Shore, this cave system was once home to the Arawak Indians known as the Taíno. In following years, the caverns were a Spanish hideaway when the island was first invaded by the British and, later, as a shelter for escaped slaves during the 18th century. They were also used as a hiding place for everyone from pirates to Cuban arms smugglers to the Jamaican government, holding treasure, guns, and barrels of rum. More recently, Green Grotto Caves served as the villain’s lair in 1973’s James Bond film Live and Let Die.

Leave the beach behind for a trip down the Martha Brae River, just an easy drive from Montego Bay. Stretching for 48 miles from the North Shore of Jamaica and the Caribbean Sea, this waterway is said to have gotten its name from a Taíno witch who, as legend has it, was tortured by Spanish settlers over the location of some hidden gold. Stories say that after revealing the treasure’s hiding place, she reversed the river’s course, blocking the cave where the riches were hidden. During the slave trade, the Martha Brae was also used to transport “white gold” when sugarcane was floated down the river on 30-foot rafts. Today, those same bamboo rafts carry visitors instead, each piloted by an experienced river guide.