Andrew Jackson, the country’s seventh president, was famous for railing against the financiers of the early 1800s. They speculated on “the breadstuffs of the country,” he warned. “Should I let you go on, you will ruin 50,000 families and that will be my sin! You are a den of vipers and thieves. I intend to rout you out and by the eternal God, I will rout you out.”

The quote, a favorite of bloggers who fret about plots to establish a new world order and such, would be at home today in the superheated arguments over high-frequency trading. The latest diatribe, I’m sad to say, comes from a dear friend and former colleague at Bloomberg Businessweek. Peter Coy writes, “The bigger the financial sector, the more dangerous it becomes.” He bemoans the flood of smart people going into the business, noting that a quarter of Harvard’s brainiacs in the early 2000s were drawn into investment banking and like fields. And he complains about banks “cranking up their trading operations in a way that imperils the financial system once again.”

His indictment, based on the May 6 flash crash, is headlined “What’s the Rush?” And his subhed warns “The American financial system is erratic and voracious, and keeps score in milliseconds. Here’s how to rein in the beast.” Among his prescriptions: a transactions tax of a few cents per $100 to “throw sand into the gears of high-frequency trading,” higher margin and collateral requirements, and steps such as new taxes to reduce corporate debt (on the idea that we’re being assailed by waves of “debt-fueled speculation.”)

Oh, come now, Peter. Let’s dial it down a bit. First, while the Great Recession was in part the fault of Wall Street, it was not a high-frequency phenomenon. Rather, we can blame bad securitization practices, flawed housing policies in Washington, poor market oversight and a raft of other well-documented problems. Superfast trading may have helped stocks crater, but it was not the force that drove them down.

Yes, one must admit that May 6 was not a good day for the high-frequency set. No matter how short-lived, the $800 billion plunge in the value of U.S. stocks that day was worrisome. Stocks such as Accenture slipped to a penny from $40 (before bouncing back) in trading patches as short as eight seconds. Clearly, something was amiss in the superfast computers at the likes of Getco.

But let’s keep a few things in perspective. First, after going haywire the market did correct itself. Prices came back, in most cases rapidly. The Dow lost 1,138.69 points from its high in crazed intraday trading on May 6, but closed just 341.9 points down, and regained all that and then some by May 10. Erratic? No doubt. Voracious. Okay, but when have traders been anything but?

Let’s concede that there’s something bizarre about high-frequency trading. Its relationship to real value in stocks is remote at best. So, too, is its connection to fundamentals such as corporate strategy, earnings power, savvy management. All that good stuff that financial journalists, MBAs and CEOs – and maybe even the odd stockbroker — prize is a few solar systems away from the zippy stock-swapping at Hard Eight Futures, Quantlab Financial and such. Those guys, snapping to the beat of their own algorithms, don’t give a hoot about such things. It’s all numbers, bro.

Let’s concede, too, that the liquidity the HFT pack supposedly brings is an illusion. It is most likely gone when most needed. The simile Peter uses – “like a swimming pool that dries up just as you jump off the high dive” – is apt (hat’s off to his wordsmithing). It’s hard to see just what value the high-freqs bring to anyone but themselves.

But, so what? Speculators, those oft-reviled folks who put the zing in stock markets, have always been in the game for the gamble. They see Wall Street as a massive roulette wheel and believe that any way they can tilt the spin to their favor – legally – is fair play. In an odd way, they are cousins to technical analysts who have long played markets free of the burden of fundamentals. Are we to ban the technical folk because their charts are more like astrology than investment? They, too, are an odd subculture of market players whose powers over stock movements one could decry.

Surely, there needs to be policing to make sure high-freqs don’t misuse the power they have to move markets. They do swap millions of shares in ridiculously short periods of times, all but blind to fundamental values. At times, they account for disturbingly high amounts of volume. If they intentionally – or through glitches – knock stocks down to absurd levels to profiteer in some market-cornering way, they need to be rapped hard for that. Fines, perhaps, or suspensions of trading privileges could be used to rein them in.

But imposing transactions taxes or worse seems like overkill. Such steps would penalize all players for the perfidy of a few. Let’s use the scalpel instead of the meat-axe and target the bad boys, not just the folks looking for an edge of a few milliseconds on the next guy.

By the way, it’s passably ironic that Peter’s employer, Bloomberg, as well as Dow Jones and other data-providers are tripping over themselves to serve up market data ever more quickly to the high-freq bunch. Some go so far as to rent space to traders — at premium prices — so they can house their computers cheek-by-jowl with providers’ machines and save milliseconds of transmission time. What these providers know, just as traders do, is that timely information is still everything in this game.

Every technological advance that changes the playing field makes folks nervous. Luddism is a natural reaction. Moreover, the markets have long been the playground of innovators and, as a consequence, the targets of critics. In 1887 the head of the Chicago Board of Trade forcibly removed telegraph gear from the floor of the CBOT because he couldn’t abide the electronic links to notorious Chicago bucket shops, as recounted by Rutgers historian David Hochfelder. One NYSE broker in 1889 complained that the “indiscriminate distribution of stock quotations to every liquor-saloon and other places has done much to interfere with business.”

We may not like the high-speed folks. We may deride them as little more than turbocharged gamblers, as Rain Man-like idiot savants unfairly using their powers to enrich themselves while adding nothing to the game. But they will be players so long as there’s money to be made. We can take the profit out if they don’t play by the rules (and, by the way, maybe some of those smart Harvard types in finance can cook up better rules to keep market ripples from becoming tsunamis). Let’s not, however, make life onerous for everyone in the process.