The upside-down and lopsided replica of the White House, deliberately built to look like it had partially collapsed, was eye-catching, but it was not why I came to the small town of Wisconsin Dells, Wisconsin, with a population of just 3,000. Rather than the overabundance of fudge and t-shirt shops, go-karts and bumper cars, haunted houses, and at least half a dozen waterparks, I wanted to see the thing that started all that: the Dells of the Wisconsin River.

Taken from the French dalles (slabs), the Dells is a five-mile gorge on the Wisconsin River, only about an hour’s drive north of Madison, where I was staying at the Mendota Lake House B&B when I was not out admiring one of the most beautiful capitol buildings in the United States, which also boasts one of the world’s top five domes.

Escaping the mass tourism of the town, I hopped on a two-hour cruise on the river through the Upper Dells. An entirely different world of preserved land and unspoiled nature unfolded around me. Sailing north between craggy palisades, some over 100’ tall (particularly attractive in autumn, when the trees atop them flaunt their brilliant reds, oranges, and yellows), it was easy to see why the French christened it what they did. The cream-colored rock formations look like sandstone pancakes piled up on top of one another.

The curiously scenic beauty of the region bears an equally curious geologic history. Although the Dells was formed during the Ice Age 15,000 years ago, the rock I was admiring is much older, well over 500 million years old, and was submerged at the bottom of a lake. At the edge of a glacier, the Dells remained uncovered by ice sheets. The melting glacier formed a lake that was retained by an ice dam, and when that dam burst, an apocalyptic flood was unleashed, lowering the depth of the lake by 100 feet, exposing the existing rock, and cutting the deep, narrow gorges and unusual rock formations that captivate visitors today.

I had the opportunity to inspect this close up during the first of my cruise’s two landings. Accessible only by boat, Witches Gulch is a narrow canyon carved by water and wind that injects itself into the eastern shore of the river. Along paths and wooden footbridges, I walked through the narrow spaces between the rock towers, some biscuit-colored, others green with a healthy moss. Shadowy passages lend a creepy air to the place, and I kept an eye out for cliff cudweed, a plant that grows in the Dells, nearby Kickapoo Valley, and nowhere else on earth.

Back on the boat, I learned that the Dells also holds significant cultural history, in addition to the rich traditions of the Native American tribes who roamed this area and left behind burial mounds, campsites, and rock art. At our second landing, we walked to the base of an imposing pillar of sandstone with a flat mushroom-cap top called Stand Rock, the birthplace of Wisconsin Dells tourism. In 1886, photographer H.H. Bennett took what would become a historic photograph—the world’s first stop-action image. The picture captures Bennett’s son, frozen in a mid-air jump between the top of a cliff and the top of Stand Rock. Not only was this moment in black and white a boon for photography and photo journalism (it proved that Bennett’s advanced shutter technology actually worked), it also begat the local tourism industry. People who saw the widely circulated and revolutionary photo began to flock to the area, eager to see the rock formations for themselves. Bennett’s studio in town, which he had opened a decade earlier and used to sell postcards and souvenir portraits to travelers, blossomed.

The event that Bennett captured on film continues in a live re-creation, although the human performing the stunt has been replaced by a trained dog. Today, a German shepherd named Achtung! made the famous leap from cliff to Stand Rock and then back again, spanning the void between the two formations in a single bound.

On the cruise back into town, I again admired the slabs of sedimentary rock on both banks of the Wisconsin River that vary in color from, depending on time of day and weather conditions, pale beige to rich orange to dark brown. It’s a far cry from what would be awaiting me back in the city, and a magnificent one at that.