In 2018, as back-to-back Hurricanes Florence and Michael threatened Chapel Hill, N.C., where I live and work, I faced a simple, binary decision like millions of others: Stay or go?

Nowadays data science is the hottest thing around. Companies cannot hire enough practitioners. There are books and online courses, and many universities are launching some flavor of a data science degree or center. Classes can barely accommodate the demand. One would hope that this golden age would mean we can make better decisions. But the deluge of data can, paradoxically, make decision-making harder: it doesn’t easily translate into useful information. The democratization of access and the proliferation of expert commentary can make things even thornier. Finally, measurement itself is not a neutral process.

The days leading up to the landfall of both hurricanes, for example, were dominated by their number on the familiar Saffir-Simpson scale of 1 to 5, corresponding to sustained wind speeds, along with the “cones” of the storms’ probable trajectories. Outside mandatory evacuation zones, it was up to everyone to decide for themselves what to do. As management consultant Peter Drucker is credited with saying: “If you can’t measure it, you can’t improve it.” I’d add: “If you do measure it, you’ll be trapped by the number.” That’s the problem with wind intensities: wind damage is obviously relevant, but the worst impact can come from flooding. Florence came ashore as a mere category 1, then dumped three feet of rain in some places—including away from the cone.

Seeking clarity, I checked in on the local TV meteorologists, who could pinpoint local impacts beyond one number. But arguably they had a bias toward emphasizing the dangers, which is better both for ratings and for self-preservation: it’s much dicier if people don’t evacuate when they should than if they flee unnecessarily. So I geared up to find more data. I sought out weather experts on social media and found well-curated lists. It seemed like a great idea at first. These were genuine experts. The commentary was respectful and intelligent. There were links to sources, and the discussion was rich.

But I quickly remembered why I never want to watch sausage being made. I learned a lot about European versus North American weather models—fascinating but fairly useless when you’re trying to decide whether to pack up a few sentimental photographs and leave. One model predicted devastation, the other just some heavy rain. A storm could turn north, for a direct hit, or south—a miss. Worse, each model updated periodically, each run generating more expert discussion.

Now I knew too much but had even less clarity for decision-making. This is sometimes referred to as the “paradox of choice”—too many options can paralyze people trying to make a decision. It’s that feeling you get standing in front of the ketchup shelves in the supermarket, overwhelmed. Organic or not? Low sugar? Sweetened with honey? With artificial sweeteners—and if so, sucralose or aspartame? Low sodium? I have resorted to grabbing one at random—I just want a bottle of ketchup. (Well, glass or plastic?)

So if more data, better science and mightier computation can give us a hurricane’s trajectory so many days in advance, why can’t anyone make better predictions of impacts at the hyperlocal level? Unfortunately, broad predictions don’t easily trickle down, because individual outcomes retain big error ranges—too many false positives and false negatives to be easily actionable.

So should we give up on data-driven decision-making in our own lives? Like many things in the age of big data, the way forward requires paying attention to things beyond the data—from how and what to measure to how to communicate about it. We need more frank talk about the shortcomings, so we can refine our understanding of the difference between a lot of data and useful information. And we especially need to build independent intermediaries to help guide us. Data science by itself can’t do all that.

As for the hurricanes, I had just moved to my street so I did the simplest thing I could think of: I asked my neighbors who’d been there for a long time. They advised me to stock up on batteries. They stayed put, and so did I.