Pentagon attack hoax illustrates investor pitfalls of AI-driven fake news

This post was originally published on this site

https://content.fortune.com/wp-content/uploads/2023/05/GettyImages-1247307955-e1684853223498.jpg?w=2048

Generative artificial intelligence is often considered a threat to white-collar workers employed in the knowledge sector. But what about capital markets? 

Investors got a taste of what may be to come after fake images of smoke billowing next to the Defense Department’s headquarters spread via Elon Musk’s social media platform Twitter. 

“There was a very brief selloff in markets yesterday after unconfirmed reports circulated on Twitter about an explosion near the US Pentagon,” wrote Jim Reid, Deutsche Bank’s head of global economics and thematic research, on Tuesday. 

“Given the suggestions that the initial photo might have been AI-generated, it just shows the potential pitfalls for markets if fake news driven by AI can cause concrete movements in asset prices.”

No one knows just who exactly created the images or why, but they were released at a time coinciding with the last-minute game of chicken over the debt ceiling playing out within Washington, DC’s Beltway. 

That means investors are already on edge as they attempt to price in contingencies of a default that Moody’s argues will not spare any corner of the global economy.

Yields on the one-month Treasury bill maturing in June hit a record high of 5.9% after bondholders demanded a higher compensation in light of a potential missed payment by the federal government.

Musk’ attempt to crack down on fake accounts

Deepfakes have proliferated of late as virtually anyone willing to pay money to generative AI platform Midjourney can now quickly and easily publish deceptive images such as the Pope sporting an all-white Balenciaga puffer jacket. 

But Monday’s brief sell-off could be the first time generative A.I. has had a demonstrable effect on financial markets.

The episode could prove damaging to Musk, who has sought to portray Twitter as the most accurate source of information on earth by charging customers for verification.

“The goal is to make this platform maximum truth-seeking or, said another way, the least untrue compared to everything else,” he wrote at the end of April.

Musk, who purchased the social media company last year for $44 billion, has argued his premium subscription service Twitter Blue serves as an effective deterrent for bots and fake accounts since it “significantly increases the cost”. 

That however has not proven to be the case, as numerous accounts have successfully impersonated people and brands, angering advertisers in the process.

Critics argue it is in reality an attempt to milk Twitter customers for much-needed revenue since all legacy accounts already verified lost their status if they refused to pay Musk.

Even if Monday’s market reaction was brief after the Pentagon confirmed there had been no explosion and the tweet was quickly deleted, a potential bad actor had plenty of time to pocket ill-gotten gains from manipulating the market—if that had in fact been the plan all along.

This threat “could be a growing issue over the months and years ahead,” Deutsche Bank’s Reid argued on Monday, “particularly if the technology is able to provide increasingly convincing images.”