SBJ Tech: Elevating the fan experience at the U.S. Open

It’s fitting that the USTA Billie Jean King National Tennis Center’s only frictionless concessions store is sponsored by American Express , giving it a pun-tastic name to brandish across the façade: American Express Shop.

The location, which is exclusive to American Express cardholders and retrofit with Zippin’s AI-powered checkout tech , is in its fourth year nestled under Court 7 to the southwest of Arthur Ashe Stadium. It is one of 150 automated shops Zippin has across industries such as entertainment and hospitality, 90 of which sit in sports venues (such as the AmEx-branded stores at the Nets/Liberty’s Barclays Center and Jazz’s Delta Center).

“A lot of effort goes into ensuring that, because it’s such a high-profile event, the store has to work no matter what,” said Zippin CEO and co-founder Krishna Motukuri. “Perfecting that has helped us in other high-profile event-driven venues like Roland Garros [where Zippin underpinned the French Open’s first autonomous shop this year].”

One week into the U.S. Open main draw, average foot traffic through the store was up more than 30%, with average daily sales up more than 40% year-over-year, according to Zippin. Those figures are even more impressive compared to 2021, the store’s first year on the grounds, with average daily traffic up more than 195% and sales up more than 330%.

The rising engagement -- and revenue -- is certainly a testament to the U.S. Open’s growing attendance . But in the eyes of Motukuri and American Express VP/Global Brand Sponsorships and Experiential Marketing Shiz Suzuki, it also speaks to consumers’ increasing comfort with cashier-less experiences.

“We always have brand ambassadors there, because we always want a seamless experience. But that first year, people were like, ‘What’s inside? What are you selling?’” Suzuki said. “Now, in our fourth year, most of the time people are aware of what it is and what they [can] buy. That entry point is a lot more seamless.”

Indeed, like other stores of its kind, the American Express Shop stations attendants at the entrance and exit turnstiles to guide customers. Signs at the exit also prompt shoppers to register for a receipt through a QR code.

Connecting the dots

On the surface, the experience is as simple as tapping a payment method to enter the store, selecting items off the shelves and walking out. Receipts are typically available within 5-10 minutes, Motukuri said, but can sometimes take hours if inflow is high.

The technical side is a bit more complicated. Zippin uses a system of 44 ceiling-hung, computer vision-powered cameras and several hundred shelf sensors in this store (others have different setups) to identify which items are taken from the store and which shoppers are taking them.

“Think about [each shopper] as a colored dot moving along the map as you’re walking through the store,” Motukuri explained. “It’s a network of cameras, and they’re tracking multiple people. Multiperson, multicamera tracking is the technical term. Are [the people] standing in one place? Are they picking something up? Are they putting something back? That action is identified. It also identifies the products that you’re dealing with and quantities. How many are picked up? How many are put back?

“Depending on the type of product and size of the product, sometimes there’s augmentation of the signals. We have sensors that will amplify the signal, because if it’s a very small product, it’s maybe hard to see how many you picked up. All of that together helps us build the shopping cart as you walk along, and the shopping cart gets closed once you exit the store.”

Suzuki proudly touted the local businesses represented in the store’s inventory -- New Jersey-based Popcorn for the People and Astoria, Queens-based Cookie Crumz, to name two -- in addition to nationwide brands for bottled drinks and packaged snacks.

As of 2023, the store also began offering soft goods (i.e., hats, shirts and towels) and this year unveiled exclusive Ralph Lauren merchandise. Zippin has experience tracking such inventory, particularly in airports, but Motukuri said certain arrangements are required to ensure they are logged correctly, such as rolling up shirts to so that they maintain a firm shape. Zippin offers an RFID-based solution for soft products but uses the same computer vision-and-sensor system as for snacks and drinks at the U.S. Open store.

Balancing speed and accuracy

I grabbed an Evian water bottle -- lot of hot days out here -- and received my receipt a few hours later (I visited the shop during the interlude between the day and night sessions, which is typically a busy period). Motukuri said the occasional long transfer times for receipts was an intentional decision on where to focus computing power.

“We’ve been doing this for four or five years now, and we always have choices,” Motukuri said. “The one thing where we don’t have a choice is accuracy. The accuracy has to be extremely high. And then we say, ‘OK, should we reduce the cost of deployment and compute for the store, so we can do more stores, or should we reduce the receipt time for the shopper?’ We almost always end up saying that the shoppers actually don’t care once they understand this is how it works. [The timing] doesn’t matter to them. But what you can’t do is ever betray their trust.”

IBM’s data center in the bowels of Arthur Ashe Stadium looks unremarkable to the outside observer: a plain blue door leading into a windowless bunker of screens and servers. But the U.S. Open sponsor of more than three decades hopes that the technology hub powers will transform the fan experience for tennis.

Generative AI plays a role here, through use-cases like the AI-assisted match preview and recap articles that the USTA’s editorial team is producing for all 254 matches in the men’s and women’s singles draws, and the AI commentary applied to highlight packages across the tournament’s digital channels.

But another application of IBM’s tech that caught my eye lies within the U.S. Open app . The IBM SlamTracker serves up AI-generated insights and a match-viewer window that displays shot data and live ball flights in 3D. The latter feature, derived from Hawk-Eye data , is new and one the USTA hopes to expand on in the future.

“Obviously, we’re not trying to compete with broadcast. It’s something you can watch along with broadcast,” said USTA Senior Director/Digital Strategy Brian Ryerson. “But also, when you’re not able to watch and you’re on the go, everybody has a phone in their pocket, and matches can happen until 3 in the morning. It’s like, ‘How can you follow in a way that brings that action to life?’”

The first iteration of the match viewer is not yet a seamless second-screen experience. When watching matches on YouTube TV, the app is typically one or two points ahead of the feed. In Ashe itself, it was 1 or 2 seconds behind.

Still, I found myself returning to the page at various points throughout the first week-and-a-half of the main draw -- and for various reasons. Late for a dinner reservation last Friday evening, I watched Ben Shelton touch 147 mph on an ace in the match viewer while standing on a subway platform. Sitting in Ashe for Sunday afternoon’s bout between Emma Navarro and Coco Gauff, I pulled up 3D stats to investigate Gauff’s serving woes.

“As that ball-trajectory data comes in, we’re essentially displaying it,” Ryerson said. “What we hope to do is sync it up, so you can sync it with your broadcaster, or sync it up with your actual screen, and then it’s truly a mirror of what’s going on.”

As of now, the match viewer offers viewpoints from the baseline and side but only tracks ball flight and placement. The players are portrayed as floating icons.

In the future, possibly as soon as next year, Ryerson hopes to integrate skeletal data of the athletes on the virtual court. “[The Hawk-Eye data feed] gives you the speed, shot type, trajectory, ball spin, everything -- it’s a pretty complex data feed that we’re then consolidating down to build that experience off of,” he said. “Then you can start layering in limb movements, actual player movements.”

All in pursuit of an even more immersive experience.

• Greenfly, the digital content management platform that retired MLBer Shawn Green co-founded with his cousin Daniel Kirschner in 2014, turned 10 years old this summer, and in this journal profile of the company, I chronicle how it's become an essential tool for short-form multimedia content for more than 1,300 leagues, teams and organizations.

• Amazon’s Just Walk Out technology is opening 14 new stores in three NFL stadiums this fall, reports SBJ's Joe Lemire: the Ravens' M&T Bank Stadium, the Seahawks' Lumen Field and the Commanders' Northwest Stadium.

• TMRW Sports named longtime PGA Tour exec Brian Cleek as its SVP/business development, writes SBJ's Josh Carpenter.

• MLB is reviving its virtual ballpark for free weekly Wednesday watch parties this September, beginning with today’s 6:50pm ET Twins-Rays game, notes Lemire.

• In his most recent SBJ Betting newsletter, Bill King is watching the performance of apps from ESPN Bet and Fanatics at the dawn of a new football season.

• King also in this week's magazine test-drives the improvements to Genius Sports' BetVision app -- which bettors can use to watch games that cuts the lag of over-the-air and cable/online broadcasts and offers more interactive betting -- and breaks down what it means for sportsbooks who use it.