Three questions to separate AI from marketing hype

Vendors everywhere pitch that their tools include some form of artificial intelligence or machine learning. Here are three questions to ask to separate the technology from the marketing hype.

One of the myriad challenges of being a modern technology leader is separating the marketing hype from reality when it comes time to procure new hardware or software. Product marketing often tends toward hyperbole and focuses on the positive rather than the negative. With technology products, there’s the added wrinkle of complex technical elements that require specialized understanding.

Mix the historical hyperventilation of most product marketing with a hot technology, and you’re forced to wallow through a dense wall of promises, buzzwords and claims to determine if a product will work for your organization. This is especially true in the era of artificial intelligence, where it seems everything from supply chain software to office furniture claims to have some element of AI embedded. One could almost imagine a late-night infomercial host shouting that the product he’s shilling: “Now includes 30% more machine learning!”

SEE: Artificial Intelligence Ethics Policy (TechRepublic Premium)

The problem in evaluating products that include AI is that definitions of what constitutes AI can vary widely. If your definition assumes learning algorithms that intelligently categorize new data, and your vendor considers AI to include little more than a bit of fancy computation, you’ll be disappointed. In order to pin down what your vendor means when they tell you there are elements of AI included in their product, here are three simple questions that can help separate the hype from reality.

How does the AI model learn?

A foundational element of most true AI technologies is that they improve based on the data they receive or include technologies that test potential future outcomes and strengthen their calculations based on those outcomes. Game-playing AI’s are a classic example of this technology, where the AI can simulate playing thousands of iterations of a game and improves its performance based on the outcome of each game.

Ask your vendor for some specifics about how the AI learns and improves. What data does it use? Does it simulate potential scenarios and use that to learn? How many simulations can it run? As you ask these types of questions, you may quickly discover that the “AI-driven learning” touted by the vendor is really just running some basic calculations on your existing data set rather than truly adjusting its algorithms based on a learning capability.

How is the AI monitored and adjusted?

Actual AI systems adapt their predictions based on some combination of the inputs they receive and their ability to run different simulations to test potential outcomes. As such, the AI will need to be monitored and potentially retrained or have additional input data.

Asking your vendor how the AI is monitored and adjusted will indicate whether their product actually contains a degree of intelligence versus some fancy standard algorithms. Suppose your vendor claims no monitoring or adjustment will ever be required. In that case, you can safely assume that AI is marketing hyperbole rather than an embedded and beneficial technology in the product under consideration.

Do you share customer data to train the AI?

Another critical question to ask your vendors actually covers two essential topics. First, it’s worth knowing whether your data are intermingled with other customers’ data to train the AI in a product. This may or may not be beneficial. For example, if you’re considering a supply chain management solution, allowing your data to inform the AI in exchange for gaining the benefit of other companies’ data could be a worthwhile trade since a more extensive data set should make the product more effective. Conversely, if you’re working with unique and highly specific data, having the AI influenced by other data could be a handicap.

This question also should cause your vendor to explain how customer data inform and improve the AI. Suppose they don’t have an answer to this question, or mention that no customer data actually impact the AI’s ability to make predictions. In that case, it’s a likely indicator that the product in question doesn’t actually include any proper AI technology.

It’s easy to tout “AI Inside” as a benefit for a tech tool. However the imprecise definition of artificial intelligence makes a tech leader’s job difficult. Using these questions to determine the extent to which AI powers your technology can be an important differentiator as you select technology.